Response text generation method and device

Abstract

The invention discloses a reply text generation method and device, and relates to the technical field of model optimization, and the method comprises the steps: carrying out the quantization tensor column decomposition of a feature training matrix in a model training process, and carrying out the calculation of a weight matrix and bias; in a pre-filling stage of model reasoning, carrying out matrix multiplication and addition operation by utilizing a weight matrix and bias in a tensor decomposition form after model training and a feature matrix after matrix tensor decomposition operation; in the decoding stage, tensor decomposition is carried out on the one-dimensional feature vector, matrix multiplication and addition are carried out through a weight matrix and bias in a tensor decomposition form, and a target reply text is generated based on a corresponding calculation result, so that the problems that in the related technology, the selection requirement of a rank in the matrix decomposition calculation process is high, the complexity is difficult to effectively reduce, and the decoding efficiency is high are solved. Or the error between the calculation result obtained by utilizing the decomposition form and the original result is large is solved, and the technical effects of greatly reducing the storage and calculation complexity and effectively improving the reasoning performance are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model optimization, and particularly relates to a reply text generation method and device. BACKGROUND

[0002] The calculation complexity and memory occupation of a self-attention module in a Transformer model grow with the length of an input sequence, which poses a great challenge to the calculation performance and energy efficiency ratio of a traditional general hardware platform.

[0003] At present, the Transformer model needs to perform dense matrix multiplication calculation in the training stage and the inference stage, and the time length of the matrix multiplication calculation is relatively long. Therefore, the related technology can accelerate the multiplication calculation of the matrix by using a matrix decomposition method to reduce the complexity of storage and calculation.

[0004] However, in the related technology, the selection of the rank in the matrix decomposition calculation process has high requirements, which makes it difficult to effectively reduce the complexity, or easily leads to a large error between the calculation result obtained by using the decomposition form and the original result, and thus needs to be urgently solved. SUMMARY

[0005] The present application provides a reply text generation method and device to at least solve the technical problem that the selection of the rank in the matrix decomposition calculation process has high requirements in the related technology, which makes it difficult to effectively reduce the complexity, or easily leads to a large error between the calculation result obtained by using the decomposition form and the original result.

[0006] The application provides a reply text generation method, comprising the following steps: obtaining initial text information of a target user, and inputting the initial text information into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and performing a preset matrix quantization tensor column decomposition operation on the feature matrix to obtain a corresponding feature matrix array decomposition result; obtaining a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model, and performing a matrix multiplication and addition operation on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in a pre-padding stage of an inference process to obtain a pre-padding matrix multiplication and addition result corresponding to the autoregressive language model; in a decoding stage of the inference process, generating a one-dimensional feature vector of a current word element based on the pre-padding matrix multiplication and addition result, and performing a preset quantization process on the one-dimensional feature vector to obtain a target feature tensor meeting a preset dimension requirement, and performing a preset vector quantization tensor column decomposition operation on the target feature tensor to obtain a corresponding vector column decomposition result; multiplying the vector column decomposition result and the weight inference matrix to obtain a corresponding vector matrix product result, and calculating a decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the vector matrix product result and the bias inference tensor to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0007] The application also provides a reply text generation device, comprising: a pre-padding module configured to obtain initial text information of a target user, and input the initial text information into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and perform a preset matrix quantization tensor column decomposition operation on the feature matrix to obtain a corresponding feature matrix array decomposition result; a matrix multiplication and addition module configured to obtain a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model, and perform a matrix multiplication and addition operation on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in a pre-padding stage of an inference process to obtain a pre-padding matrix multiplication and addition result corresponding to the autoregressive language model; a decoding module configured to, in a decoding stage of the inference process, generate a one-dimensional feature vector of a current word element based on the pre-padding matrix multiplication and addition result, and perform a preset quantization process on the one-dimensional feature vector to obtain a target feature tensor meeting a preset dimension requirement, and perform a preset vector quantization tensor column decomposition operation on the target feature tensor to obtain a corresponding vector column decomposition result; and a text generation module configured to multiply the vector column decomposition result and the weight inference matrix to obtain a corresponding vector matrix product result, and calculate a decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the vector matrix product result and the bias inference tensor to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0008] The application further provides an electronic device, comprising a memory for storing a computer program, and a processor for implementing the steps of any of the reply text generation methods when executing the computer program.

[0009] The application further provides a non-volatile computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the reply text generation methods.

[0010] The application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the reply text generation methods.

[0011] Through the application, the initial text information of the target user can be obtained, and the initial text information is input into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and a preset matrix quantization tensor column decomposition operation is performed on the feature matrix to obtain a corresponding feature matrix array decomposition result; a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model are obtained, and a matrix multiplication and addition operation is performed on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in the pre-padding stage of the inference process to obtain a pre-padding matrix multiplication and addition result corresponding to the autoregressive language model; in the decoding stage of the inference process, based on the pre-padding matrix multiplication and addition result, a one-dimensional feature vector of the current word element is generated, and a preset quantization processing is performed on the one-dimensional feature vector to obtain a target feature tensor meeting a preset dimension requirement, and a preset vector quantization tensor column decomposition operation is performed on the target feature tensor to obtain a corresponding vector column decomposition result; the vector column decomposition result and the weight inference matrix are multiplied to obtain a corresponding vector matrix product result, and based on the vector matrix product result and the bias inference tensor, a decoding matrix multiplication and addition result corresponding to the autoregressive language model is calculated to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result, so that the technical problem that in the related art, the selection requirement of the rank in the matrix decomposition calculation process is high, the complexity is difficult to effectively reduce, or the error between the calculation result obtained by using the decomposition form and the original result is large can be solved, and the technical effects that the storage and calculation complexity are greatly reduced, the occupation of the data on the storage space is reduced, and the inference performance is effectively improved are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 A flow chart of a reply text generation method according to an embodiment of the application is provided. Figure 2 A schematic diagram of a one-dimensional feature vector quantized tensor column decomposition process according to an embodiment of the application is provided. Figure 3 An example diagram of a reply text generation device according to an embodiment of the application is provided.

[0014] Among them, 10 is a reply text generation device, 100 is a pre-filling module, 200 is a matrix multiplication and addition module, 300 is a decoding module, and 400 is a text generation module. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0016] It should be noted that in the description of the application, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms "first", "second" and the like in the application are used to distinguish similar objects, not to describe a specific order or sequence.

[0017] In order for those skilled in the art to better understand the application, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0018] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the reply text generation method depends, the specific application environment architecture or specific hardware architecture is described here.

[0019] An embodiment of the application provides a reply text generation method.

[0020] As shown in Figure 1 A flow chart of a reply text generation method according to an embodiment of the application is provided, wherein the reply text generation method comprises the following steps: In step S101, initial text information of a target user is acquired, and the initial text information is input into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and a preset matrix quantization tensor column decomposition operation is performed on the feature matrix to obtain a corresponding feature matrix array decomposition result.

[0021] As appreciated by those skilled in the art, at present, the model based on the Transformer structure needs to perform dense matrix multiplication calculation in both the training phase and the inference phase, and the time consumption of the matrix multiplication calculation is relatively long. In the training phase, the calculation complexity required by the dense matrix calculation is the third power of the matrix dimension, and the calculation complexity and the occupied memory in the inference phase increase in the second power with the input sequence length.

[0022] However, in the related art, although the storage and calculation complexity are reduced when performing the matrix multiplication calculation, the selection of the rank will affect the final calculation result. If the rank is selected to be too large, the complexity reduction amplitude is relatively small, and the matrix decomposition form for solving the weight matrix is additionally added, which will cause the total calculation amount to increase. If the rank is selected to be too small, the error between the calculation result obtained by using the decomposition form and the original result will be relatively large, and with the increase of the training or inference iteration number, the error will gradually accumulate, and finally the calculation result will be seriously wrong.

[0023] For the autoregressive language model, it usually includes two parts of calculation processes, i.e., the self-attention mechanism and the feedforward neural network. Both of the two parts of calculation processes are matrix multiplication calculation in the pre-padding phase of training and inference, and are vector multiplication matrix calculation in the decoding phase of inference. Therefore, based on the quantization tensor column decomposition operation of the vector and the matrix, the embodiments of the present application can solve the large consumption of the calculation resources caused by the matrix multiplication step in the training and inference of the large model, and simultaneously relieve the pressure of the demand for large storage space in the inference process.

[0024] Specifically, in the actual execution process, the embodiments of the present application can first acquire the original text of the user and input it into the pre-trained autoregressive language model to generate a corresponding feature matrix, and perform quantization tensor column decomposition on the feature matrix to obtain a corresponding feature matrix array decomposition result.

[0025] Therefore, the embodiments of the present application can extract the feature matrix corresponding to the text by the autoregressive language model, and perform quantization decomposition thereon, so as to efficiently process the text information input by the user, simplify the feature dimension, and improve the efficiency and accuracy of the subsequent text analysis or application.

[0026] Optionally, in one embodiment of the present application, the feature matrix is subjected to a preset matrix quantization tensor column decomposition operation to obtain a corresponding feature matrix array decomposition result, comprising: performing a preset zero padding operation on the feature matrix to obtain a feature zero padding matrix satisfying a preset matrix dimension requirement, and performing a preset rearrangement processing on the elements in the feature zero padding matrix to generate a target dimension tensor operator corresponding to the feature matrix; obtaining dimension information of the target dimension tensor operator, and determining a matrix quantization tensor column decomposition strategy according to the dimension information, and decomposing the target dimension tensor operator through the matrix quantization tensor column decomposition strategy to obtain the feature matrix array decomposition result.

[0027] In the specific implementation process, the steps of the feature matrix quantization tensor column decomposition operation in the embodiment of the present application are as follows: Step 1: the feature matrix is subjected to a zero padding operation, and its dimension is expanded to (i.e. a preset matrix dimension requirement) to obtain a corresponding feature zero padding matrix (wherein, represents the feature matrix is a matrix with rows and columns in a real number field; represents the dimension of the feature zero padding matrix in the row direction; represents the dimension of the feature zero padding matrix in the column direction; represents the basic unit size of the target dimension tensor operator in the row direction decomposition; represents the basic unit size of the target dimension tensor operator in the column direction decomposition; represents the number of decomposition levels or the order of the tensor); Step 2: the feature zero padding matrix is subjected to an element rearrangement processing to obtain a high-dimensional tensor operator with a row dimension of and a column dimension of or , and the mathematical description of this step is as follows: the feature zero padding matrix is composed of elements , as shown in the following formula:

[0028] After the element rearrangement of , the high-dimensional tensor operator A is obtained, which is composed of elements , as shown in the following formula:

[0029] and the element corresponds to the element of the feature zero padding matrix ​and the corresponding relationship between them is shown in the following formula:

[0030] wherein, denotes a feature zero-padded matrix , and the element in the i-th row and the j-th column of , i ranges from 1 to ; denotes an element in the high-dimensional tensor operator A ; denotes the index of each level after the row direction decomposition of the high-dimensional tensor operator, and ranges from 1 to ; denotes the index of each level after the column direction decomposition of the high-dimensional tensor operator, and ranges from 1 to ; Step 3: decompose the high-dimensional tensor operator A using the tensor column decomposition method to obtain the feature matrix array decomposition result, as shown in the following formula:

[0031] wherein, denotes a small-scale 4-dimensional tensor operator with a size of denotes an intermediate index in the decomposition process, and ranges from .

[0032] That is to say, if the feature matrix is expressed in the form of matrix array decomposition, the feature matrix array decomposition result is referred to as the rank of the feature matrix , and the quantized tensor column decomposition expression form of . Through the quantized tensor column decomposition, the complexity of the feature matrix can be reduced from to , wherein denotes the maximum value in the rank parameters of each level.

[0033] It can be understood that has a smaller value, so compared with the original data size of the tensor, the storage complexity of the data after the quantized tensor column decomposition is greatly reduced, thereby occupying less storage space and greatly reducing the demand for large storage space in the inference process.

[0034] In addition, the quantized tensor column decomposition operation on the feature matrix in the above inference process is also applicable to different types of matrices (such as weight matrices) in the inference or training process.

[0035] Therefore, the embodiments of the present application decompose the matrix (such as a feature matrix or a weight matrix) involved in the corresponding matrix multiplication operation in the model inference or training process based on the matrix tensor decomposition method to obtain a decomposition result with a smaller storage complexity, thereby greatly reducing the storage and calculation complexity and effectively improving the inference performance.

[0036] Optionally, in an embodiment of the present application, before obtaining the initial text information of the target user and inputting the initial text information into the pre-trained autoregressive language model to generate the feature matrix corresponding to the initial text information, it further includes: collecting different types of text data, and performing format conversion operation on the text data to generate corresponding standard text data, and constructing a text training data set through the standard text data; inputting the text training data in the text training data set into the autoregressive language model to generate a feature training matrix corresponding to the text training data; performing a pre-set zero padding operation on the feature training matrix to obtain a corresponding feature zero padding training matrix, and performing a pre-set element rearrangement processing on the feature zero padding training matrix to generate a corresponding feature training tensor operator, and performing a pre-set matrix quantization tensor column decomposition operation on the feature training tensor operator to obtain a corresponding feature training decomposition matrix; obtaining a pre-set weight matrix and a pre-set bias tensor in the form of tensor decomposition corresponding to the autoregressive language model, and performing a matrix multiplication and addition operation on the feature training decomposition matrix, the pre-set weight matrix and the bias tensor to obtain a corresponding training process matrix multiplication and addition result; based on the feature training matrix and the feature training decomposition matrix, calculating the matrix multiplication and addition error corresponding to the training process matrix multiplication and addition result before and after the decomposition of the feature training matrix, and respectively based on the matrix multiplication and addition error and the training process matrix multiplication and addition result, iteratively training the autoregressive language model.

[0037] In actual execution process, the embodiments of the present application can first collect multi-domain text data (such as news, dialogue, professional literature, etc.), and uniformly convert the multi-domain different types of text data into standard text data in a pre-set encoding format to construct a text training data set through the standard text data.

[0038] Secondly, the embodiments of the present application can input the text training data into the autoregressive language model to generate a feature training matrix containing semantic features and context association, and perform a dynamic zero padding (such as adaptive padding according to the longest text length) processing on the feature training matrix to obtain a corresponding feature zero padding training matrix.

[0039] Thirdly, the embodiments of the present application can perform an element rearrangement operation on the feature zero padding training matrix to generate a feature training tensor operator, and perform a matrix quantization tensor column decomposition operation on the feature training tensor operator to obtain a feature training decomposition matrix.

[0040] Afterwards, the embodiments of the present application can pre-set the weight matrix and bias tensor of the autoregressive language model into a quantized tensor column decomposition form (such as a combination of low-dimensional matrices with a fixed rank number) in advance, so as to directly perform matrix multiplication and addition operation on the feature training decomposition matrix and the pre-set weight matrix and pre-set bias tensor of the quantized tensor column decomposition form during the model training process, to obtain the corresponding training process matrix multiplication and addition result.

[0041] Finally, the embodiments of the present application can calculate the training process matrix multiplication and addition result before and after the feature training matrix decomposition, and calculate the matrix multiplication and addition error corresponding to the training process matrix multiplication and addition result before and after the matrix decomposition, to calculate the corresponding loss function based on the matrix multiplication and addition error and the training process matrix multiplication and addition result, and perform back propagation, so that the autoregressive language model learns the compensation mechanism of the quantized decomposition error in the iteration.

[0042] It can be understood that the weight of the autoregressive language model obtained after the training (i.e. the weight inference matrix) is highly compressed lightweight format data, which can be directly used for corresponding matrix multiplication and addition calculation in the pre-padding stage and the decoding stage of the model inference, not only greatly reducing the storage requirement and the memory access bandwidth, but also better adapting to the hardware accelerator with low rank operation optimization requirement, and improving the end-to-end model inference efficiency.

[0043] Therefore, the embodiments of the present application train the weight and bias data into a tensor decomposition form during the training process, so that the error caused by the quantized decomposition can be learned and adapted at the beginning of the model training, greatly simplifying the model deployment process and avoiding the accuracy loss caused by the additional decomposition in the inference stage.

[0044] Optionally, in an embodiment of the present application, the preset weight matrix and the preset bias tensor corresponding to the tensor decomposition form of the autoregressive language model are obtained, including: obtaining hierarchical structure information of the autoregressive language model, and parsing the hierarchical structure information to determine input and output feature dimensions and inter-level connection relationships corresponding to the autoregressive language model according to the parsed hierarchical structure information; determining a context window length of the autoregressive language model, and determining an action range and a matrix operation dimension constraint of an initial weight matrix and an initial bias tensor corresponding to the autoregressive language model based on the input and output feature dimensions, the context window length and the inter-level connection relationships; determining a number of hidden layer units and a number of attention heads of the autoregressive language model, and determining a row and column scale of the initial weight matrix according to the number of hidden layer units and the number of attention heads, to construct the initial weight matrix and the initial bias tensor based on the row and column scale, the action range and the matrix operation dimension constraint; performing a preset dimension expansion operation on the initial weight matrix and the initial bias tensor respectively to obtain corresponding expanded matrices, and performing a quantized tensor column decomposition operation on the expanded matrices to obtain a weight decomposition matrix and a bias decomposition matrix corresponding to the expanded matrices, and determining the preset weight matrix and the preset bias tensor according to the weight decomposition matrix and the bias decomposition matrix.

[0045] It should be noted that, in the embodiment of the present application, the hierarchical structure information of the autoregressive language model is first obtained, and the input and output feature dimensions (such as 640 dimensions of the word embedding layer 640 and 1280 dimensions of the attention layer) and the hierarchical connection relationships (such as residual connection and layer normalization sequence) are extracted by a structured hierarchical parsing algorithm; then, the embodiment of the present application can generate a dimension operation dimension constraint graph in combination with a preset context window length (such as 1024 tokens), to clearly define the action boundary (such as acting only within the attention head or cross-layer interaction) and the dimension matching rule (such as ensuring the dimension compatibility of matrix multiplication) of the initial weight matrix and the bias tensor at different levels.

[0046] Secondly, the embodiment of the present application can calculate a basic row and column scale (such as 16 heads x 64 dimensions / head = 1024 columns) of the initial weight matrix according to the number of model hidden layer units (such as 2048) and the number of attention heads (such as 16), and dynamically adjust the scale (such as expanding the cross-layer connection matrix to 2048 x 1280) in combination with the dimension constraint graph, to construct the initial weight matrix and the initial bias tensor.

[0047] Afterwards, the embodiment of the present application can dynamically zero-fill the initial weight matrix and the initial bias tensor (adaptively fill zero elements based on the feature dimension difference of each layer), and generate an initial tensor operator through diagonal block rearrangement (aggregating attention head related parameters into a diagonal block structure) and other operations, and adopts a hierarchical quantization decomposition strategy to decompose the attention layer weight into 16 low-rank matrices according to the head, and decomposes the feedforward layer weight according to the hidden layer unit grouping, so as to obtain the weight decomposition matrix and the bias decomposition matrix, and take them as the preset weight matrix and the preset bias tensor of the autoregressive language model training.

[0048] Therefore, the embodiment of the present application can accurately match the quantization decomposition initial parameters corresponding to the autoregressive language model structure, guarantee the compatibility of model training, reduce the storage and calculation cost, and lay a solid foundation for efficient deployment of the model.

[0049] Optionally, in an embodiment of the present application, the initial weight matrix and the initial bias tensor are respectively subjected to preset dimension expansion operations to obtain a weight expansion matrix and a bias expansion tensor, and the weight expansion matrix and the bias expansion tensor are subjected to a quantized tensor column decomposition operation to obtain a corresponding weight decomposition matrix and a bias decomposition matrix, comprising: determining the dimension expansion requirement corresponding to the initial weight matrix and the initial bias tensor, and performing a preset zero-fill operation on the initial weight matrix and the initial bias tensor according to the dimension expansion requirement to obtain a weight expansion matrix corresponding to the initial weight matrix and a bias expansion tensor corresponding to the initial bias tensor; performing a preset element rearrangement operation on the weight expansion matrix and the bias expansion tensor to obtain a weight tensor operator corresponding to the weight expansion matrix and a bias tensor operator corresponding to the bias expansion tensor, and performing a quantized tensor column decomposition operation on the weight tensor operator and the bias tensor operator to obtain a corresponding weight decomposition matrix and a bias decomposition matrix.

[0050] In actual execution process, the embodiment of the present application can adopt the quantized tensor column decomposition processing mode of the above-mentioned feature matrix to perform tensor column decomposition processing on the initial weight matrix and the initial bias tensor.

[0051] Specifically, the embodiment of the present application can first perform a 0-fill operation on the initial weight matrix and the initial bias tensor respectively to obtain a weight expansion matrix corresponding to the initial weight matrix and a bias expansion tensor corresponding to the initial bias tensor; secondly, the embodiment of the present application can perform a preset element rearrangement operation on the weight expansion matrix and the bias expansion tensor to obtain a corresponding weight tensor operator and a bias tensor operator.

[0052] Afterwards, the embodiment of the present application can perform tensor column decomposition on the weight tensor operator and the bias tensor operator to obtain a corresponding weight decomposition matrix and a bias decomposition matrix.

[0053] For example, embodiments of this application may first pad the initial weight matrix B with zeros to expand its dimensionality. The weight expansion matrix (where, This indicates the row number and directional dimension of the weight expansion matrix; (This represents the column number and directional dimension of the weight expansion matrix). Secondly, in this embodiment, the elements of the zero-padded weight expansion matrix can be rearranged to obtain a matrix of size (...). Weight tensor operator .

[0054] Subsequently, embodiments of this application can be applied to the weight tensor operator. Perform quantization tensor column decomposition to obtain the corresponding weight decomposition matrix, as shown in the following equation:

[0055] in, Indicates the dimension size as Small-scale 4D tensor operators; Represents the elements in the weight tensor operator; This represents the indexes at each level after column-wise decomposition; This represents the index of each level after decomposition along another dimension; Indicates an intermediate index during the initial weight matrix decomposition process; This represents the rank parameter of each level in the column decomposition of the initial weight matrix tensor.

[0056] Therefore, the embodiments of this application process the initial weight matrix and initial bias tensor through zero-padding, element rearrangement and tensor column decomposition operations to obtain parameters in a low-rank decomposition form, thereby reducing model storage and computational overhead and providing reliable data and technical support for efficient model training and deployment.

[0057] Optionally, in one embodiment of this application, the autoregressive language model is iteratively trained based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process, respectively. This includes: calculating the feature error gradient of each iteration of the training process based on the matrix multiplication and addition results of the training process, and determining the gradient direction of the feature error gradient, so as to adjust the preset weight matrix and preset bias tensor according to the gradient direction, so as to obtain the weight adjustment matrix and bias adjustment tensor after each iteration of training; iteratively training the autoregressive language model through the weight adjustment matrix and bias adjustment tensor until the autoregressive language model meets the preset convergence requirements, so as to obtain the final weight adjustment matrix and bias adjustment tensor, and determining the weight inference matrix and bias inference tensor according to the final weight adjustment matrix and bias adjustment tensor, and generating the trained autoregressive language model.

[0058] As an implementable manner, the embodiment of the present application can calculate the feature error gradient of each iteration by using the adaptive gradient clipping algorithm based on the matrix multiplication and addition result of the training process, and identify and smooth the abnormal gradient (such as the mutation value) by gradient direction vector clustering.

[0059] Secondly, the embodiment of the present application can dynamically adjust the preset weight matrix and bias tensor according to the modified gradient direction, combined with the quantization decomposition constraint (such as the low-rank matrix row and column number unchanged), to use a hierarchical learning rate for the weight decomposition matrix (the attention head parameter learning rate is higher than the feedforward layer), and apply L1 regularization constraint to the bias decomposition tensor, to suppress redundant parameters.

[0060] After each iteration, the embodiment of the present application can evaluate the model state by the weighted sum of the feature reconstruction error (the difference between the feature matrix before and after decomposition) and the prediction loss value, and if the weighted sum of the continuous 5 iterations decreases by less than a threshold, the learning rate decay mechanism is triggered.

[0061] When the model meets the preset convergence requirement (such as the prediction loss value is stable for 3 iteration periods), the final weight adjustment matrix and bias adjustment tensor are calibrated for quantization accuracy (such as mapping the floating-point parameters to the 8-bit integer range), to generate the weight inference matrix and the bias inference tensor, thereby obtaining the trained autoregressive language model. Thus, the embodiment of the present application optimizes the iterative training process, thereby balancing the quantization decomposition constraint and the model precision, and improving the efficiency and accuracy of model inference.

[0062] Optionally, in an embodiment of the present application, based on the feature training matrix and the feature training decomposition matrix, the matrix multiplication and addition error corresponding to the training process matrix multiplication and addition result before and after the decomposition of the feature training matrix is calculated, and based on the matrix multiplication and addition error and the training process matrix multiplication and addition result respectively, the self-recurrent language model is iteratively trained, including: calculating the decomposition-before matrix multiplication result of the feature training matrix and the initial weight matrix, and calculating the decomposition-before matrix multiplication and addition result according to the decomposition-before matrix multiplication result and the initial bias tensor; calculating the decomposition-after matrix multiplication result of the feature training decomposition matrix and the preset weight matrix, and calculating the decomposition-after matrix multiplication and addition result according to the decomposition-after matrix multiplication result and the preset bias tensor; based on the decomposition-before matrix multiplication and addition result and the decomposition-after matrix multiplication and addition result, the corresponding matrix multiplication and addition error is calculated, and the matrix multiplication and addition error is input into the pre-constructed error evaluation model to output the corresponding element-level difference information; the error size and the error distribution position are determined according to the element-level difference information, and the error vector is constructed through the error size and the error distribution position, and the operation link that meets the preset error contribution requirement in the error vector is marked to generate a labeled error vector; based on the labeled error vector, the element mapping rule of the preset element rearrangement operation is adjusted, and the adjusted element mapping rule is used to perform the preset matrix quantization tensor column decomposition operation on the feature training matrix, the initial weight matrix and the initial bias tensor again to obtain corresponding decomposition data, and based on the decomposition data, the self-recurrent language model is trained.

[0063] Specifically, the embodiments of the present application can also include the error generated in the decomposition process into the back propagation link of the model training, realizing the cooperation of the decomposition accuracy and the model parameter optimization, and the specific process is as follows: 1、obtain the feature training matrix, the initial weight matrix and the initial bias tensor, perform the conventional matrix multiplication on the feature training matrix and the initial weight matrix, and superimpose the initial bias tensor to obtain the decomposition-before matrix multiplication and addition result; 2、perform the preset element rearrangement operation on the feature training matrix and then perform the quantization tensor column decomposition to obtain the feature training decomposition matrix; perform the preset matrix quantization tensor column decomposition on the initial weight matrix and the initial bias tensor respectively to obtain the preset weight matrix and the preset bias tensor; 3、calculate the decomposition-after matrix product of the feature training decomposition matrix and the preset weight matrix, superimpose the preset bias tensor to obtain the decomposition-after matrix multiplication and addition result; 4、calculate the difference value of the decomposition-before and decomposition-after matrix multiplication and addition results to obtain the matrix multiplication and addition error, input the matrix multiplication and addition error into the error evaluation model, and output the element-level difference information; 5、determine the error size and the distribution position according to the element-level difference information, construct the error vector, and mark the operation link whose error contribution degree exceeds the preset threshold to generate a labeled error vector; 6. The labeled error vector is transmitted in the reverse direction along the original operation link to the quantized tensor column decomposition step of the vector and the quantized tensor column decomposition step of the matrix, and the error label is located to the corresponding quantized rearrangement and decomposition link to output the error reverse distribution result; 7. Based on the error reverse distribution result, the element mapping rule of the quantized rearrangement is adjusted (such as optimizing the element grouping method and adjusting the rearrangement order), and the initialization parameters of the decomposition core are corrected (such as adjusting the initial value distribution of the low-rank core), and the corrected quantization rule and the decomposition core initialization parameter are output as the input of the next round of vector and matrix quantized tensor column decomposition steps, and the decomposition error is dynamically corrected to realize the iterative training of the autoregressive language model. It should be noted that in the iterative training process of the autoregressive language model of the embodiment of the present application, the model training process based on the matrix multiplication error and the model training process based on the matrix multiplication result of the training process complement each other, thereby greatly improving the training performance of the model; for example, after the matrix quantized tensor column decomposition operation is performed on the feature training matrix, the initial weight matrix and the initial bias tensor based on the matrix multiplication error, the corresponding decomposition data is obtained, and the embodiment of the present application can calculate a new matrix multiplication result according to the decomposition data; further, the embodiment of the present application can calculate the corresponding feature error gradient using the new matrix multiplication result, to adjust the corresponding weight matrix and bias tensor, and train the autoregressive language model through the adjusted weight matrix and bias tensor (i.e. complete an iterative training process), until the autoregressive language model meets the preset convergence requirement, to obtain the final weight adjustment matrix and bias adjustment tensor, and generate the trained autoregressive language model.

[0064] Therefore, the embodiment of the present application dynamically adjusts the mapping rule by analyzing the quantized decomposition error, thereby improving the operation accuracy after decomposition and enhancing the stability and efficiency of model training.

[0065] In step S102, the weight inference matrix and the bias inference tensor in the tensor decomposition form corresponding to the autoregressive language model are obtained, and the matrix multiplication operation is performed on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in the pre-padding stage of the inference process to obtain the pre-padding matrix multiplication result corresponding to the autoregressive language model.

[0066] Further, the embodiment of the present application also needs to perform the matrix multiplication operation on the feature matrix array decomposition result and the weight inference matrix and the bias inference tensor in the tensor decomposition form after model training in the pre-padding stage of the model inference process to obtain the pre-padding matrix multiplication result corresponding to the autoregressive language model.

[0067] Specifically, the embodiment of the present application can calculate the product of the feature matrix A after the quantized tensor column decomposition and the weight inference matrix B in the tensor decomposition form to obtain the matrix multiplication result of the model in the pre-padding stage as shown in the following formula: ; Then, the embodiment of the present application can perform a matrix addition operation on the matrix multiplication result of the pre-padding stage and the bias inference tensor in the tensor decomposition form to obtain the pre-padding matrix multiplication and addition result corresponding to the autoregressive language model.

[0068] It can be understood that the embodiment of the present application can reduce the calculation complexity of the matrix product from the original to , wherein, represents the number of rows of the feature matrix, and herein represents the number of rows of the weight inference matrix, represents the number of columns of the weight inference matrix; is the rank of the operator, and its value is small after decomposition, so that the calculation complexity of the matrix product is also greatly reduced due to the tensor decomposition, so that the matrix multiplication after the tensor decomposition is linearly dependent on the dimension of the tensor operator, rather than the dimension of the matrix.

[0069] In step S103, in the decoding stage of the inference process, a one-dimensional feature vector of the current token is generated based on the pre-padding matrix multiplication and addition result, and a preset quantization processing is performed on the one-dimensional feature vector to obtain a target feature tensor satisfying a preset dimension requirement, and a preset vector quantized tensor column decomposition operation is performed on the target feature tensor to obtain a corresponding vector column decomposition result.

[0070] Then, in the decoding stage of the model inference process, the embodiment of the present application takes the matrix multiplication and addition result output in the pre-padding stage (i.e., the pre-padding matrix multiplication and addition result) as the core basis, which is generated by the matrix multiplication and addition operation of the feature matrix array decomposition result of the model in the pre-padding process, the weight inference matrix and the bias inference tensor, and has fused the global context association features and hierarchical semantic information of the text.

[0071] Then, the embodiment of the present application can accurately extract a local feature subset at a corresponding position from the pre-padding matrix multiplication and addition result based on the token index of the current decoding position, and through a dynamic attention weight distribution mechanism, the association strength of the current token and the context key information (such as core nouns and logical conjunctions) is strengthened, and the interference of redundant information is weakened.

[0072] Furthermore, embodiments of this application may also introduce an adaptive feature fusion module, which weights and fuses the extracted local features with the feature vectors of historical decoded words, and then removes invalid dimensions through a dimensionality compression algorithm (such as feature filtering based on information entropy), ultimately generating a one-dimensional feature vector that combines global contextual relevance, historical dependency features, and current word specificity, laying the foundation for subsequent quantization and decomposition operations.

[0073] Finally, embodiments of this application can quantize the one-dimensional feature vector (i.e., perform a dimension expansion operation) to obtain a target feature tensor that meets the preset dimension requirements (such as the same dimension as the weight inference matrix), and perform a preset vector quantization tensor column decomposition operation on it to obtain the corresponding vector column decomposition result.

[0074] Therefore, the embodiments of this application can greatly simplify the feature processing flow of inference decoding, reduce computational complexity and storage overhead, while ensuring the effectiveness of feature representation, which helps to improve the efficiency of model inference.

[0075] Optionally, in one embodiment of this application, a one-dimensional feature vector is subjected to a preset quantization process to obtain a target feature tensor that meets the preset dimension requirements, and a preset vector quantization tensor column decomposition operation is performed on the target feature tensor to obtain the corresponding vector column decomposition result. This includes: determining the vector scale information of the one-dimensional feature vector, and performing a preset element rearrangement operation on the one-dimensional feature vector based on the vector scale information to obtain a target feature tensor that meets the preset dimension requirements; performing a preset tensor column decomposition operation on the target feature tensor to obtain multiple sub-vectors corresponding to the one-dimensional feature vector, and determining the vector column decomposition result based on the multiple sub-vectors.

[0076] As one possible approach, such as Figure 2 As shown, for a dimension of size One-dimensional feature vector The embodiments of this application can first perform quantization (i.e., element rearrangement) to obtain a scale of (Total) indivual A high-dimensional tensor (i.e., the target feature tensor) that is multiplied (i.e., meets the preset dimension requirements). As shown in the following formula:

[0077] in, This indicates the size of the basic unit for quantization decomposition; Representing a high-dimensional tensor Elements in; Represents a one-dimensional eigenvector Elements in; And we can obtain the following formula:

[0078] Secondly, the embodiments of the present application can perform tensor column decomposition (TTD) on high-dimensional tensors to obtain a plurality of sub-vectors (one-dimensional tensors); wherein the high-dimensional tensors The mathematical expression of tensor column decomposition is as follows:

[0079] wherein, represents a three-dimensional tensor with a size of ; represents an intermediate index in the decomposition process, and the value range is 1, 2, , ; is the rank of quantized tensor column decomposition; is the core of quantized tensor column decomposition; represents a dimension parameter related to the basic unit size of quantized decomposition.

[0080] It can be understood that, by performing quantized tensor column decomposition on one-dimensional feature vectors , the complexity of one-dimensional feature vectors can be reduced from to , wherein, , so that the storage complexity of data after quantized tensor column decomposition is greatly reduced, and the demand of the model on large storage space in the inference process is reduced.

[0081] In step S104, the vector column decomposition result and the weight inference matrix are multiplied to obtain the corresponding vector matrix product result, and based on the vector matrix product result and the bias inference tensor, the decoding matrix multiplication and addition result of the autoregressive language model is calculated to generate the target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0082] Further, the embodiments of the present application can calculate the vector matrix product result between the vector column decomposition result and the weight inference matrix, and operate the product result with the bias inference tensor to obtain the decoding matrix multiplication and addition result of the autoregressive language model, and generate the target reply text corresponding to the initial text information of the user based on the decoding matrix multiplication and addition result.

[0083] Thus, the embodiments of the present application can guarantee the efficient connection of the model decoding process through the accurate operation of vectors and matrices, so as to control the calculation cost of the model while ensuring the accuracy and fluency of the reply text generation.

[0084] Optionally, in an embodiment of the present application, the vector column decomposition result and the weight inference matrix are multiplied to obtain a corresponding vector matrix product result, including: obtaining a plurality of historical vector matrix product results and a plurality of historical vectors of the historical input data of the self-recurrent language model in the historical decoding stage, and performing semantic correlation degree evaluation on the plurality of historical vector matrix product results and the plurality of historical vectors to obtain corresponding correlation degree evaluation data, and determining the historical feature correlation factor of the self-recurrent language model through the correlation degree evaluation data; performing element-wise superposition operation on the vector matrix product result and the bias inference tensor to obtain a decoding matrix multiplication result, and calculating the target decoding matrix multiplication result of the self-recurrent language model based on the decoding matrix multiplication result and the historical feature correlation factor; performing feature mapping on the target decoding matrix multiplication result to obtain a feature matching degree sequence corresponding to the preset text database, and selecting at least one reply text satisfying the preset matching degree requirement from the preset text database based on the feature matching degree value of the feature matching degree sequence, and splicing the at least one reply text to generate a target reply text.

[0085] As an implementable way, the embodiments of the present application can first obtain the historical vector corresponding to the historical vector matrix product result and the historical input data of the self-recurrent language model in the historical decoding stage, and evaluate the semantic correlation degree between the two by adopting the way of combining semantic similarity matching with attention weight calculation, that is, by calculating the vector cosine similarity and the key feature overlap rate to generate correlation degree evaluation data, and then dynamically assigning different historical records with differentiated weights based on the evaluation data to determine the historical feature correlation factor of the model.

[0086] Secondly, the embodiments of the present application can perform element-wise superposition operation on each row of the current vector matrix product result and the bias inference tensor to obtain a decoding matrix multiplication result , and the calculation process is as shown in the following formula:

[0087] wherein, represents the element of the first level after the tensor column decomposition of the current vector matrix product result , and L represents the index of the level; represents the element of the first level after the tensor column decomposition of the bias inference tensor ; and represents the element of the first level after the tensor column decomposition of the decoding matrix multiplication result D ; and L represents the total number of decomposition levels.

[0088] ​Again, the embodiment of the present application can weight and fuse the decoding matrix multiplication result and the historical feature correlation factor (the correlation factor weight is dynamically adjusted with the decoding step number) to generate a target decoding matrix multiplication result that takes into account the current semantics and historical dependence.

[0089] After that, the embodiment of the present application can use a multi-level feature mapping algorithm to map the target decoding matrix multiplication result into a feature matching degree sequence consistent with the feature dimension of the preset text database, and sort the matching degree values in descending order to filter out a plurality of candidate reply texts that meet the preset threshold, eliminate conflicting texts through semantic coherence verification (such as syntax fluency, logical consistency detection), and finally splice the target reply text in a smooth and coherent manner according to the matching degree priority.

[0090] Thus, the embodiment of the present application improves the coherence and accuracy of the reply text generation by fusing historical feature correlation and precise feature matching operations, optimizes the feature mapping and text screening efficiency, and guarantees the generation quality of the reply text.

[0091] Optionally, in an embodiment of the present application, after generating the target reply text corresponding to the initial text information according to the decoding matrix multiplication result, it further includes: collecting multi-modal feedback data of the target user on the target reply text, and analyzing the multi-modal feedback data to obtain corresponding analysis data, and extracting a plurality of text evaluation information corresponding to the analysis data, wherein the plurality of text evaluation information includes semantic fit degree, content completeness and logical coherence; extracting a plurality of text features of the target reply text, and performing feature correlation operation on the plurality of text features and the plurality of text evaluation information to determine the accuracy score and feature matching difference level of the target reply text, wherein the plurality of text features include semantic features, syntax structure features and content coherence features; adjusting the weight inference matrix and the bias inference tensor of the autoregressive language model according to the accuracy score and the feature matching difference level to obtain the corresponding weight optimization matrix and the bias optimization tensor, and calculating the new decoding matrix multiplication result corresponding to the autoregressive language model based on the weight optimization matrix and the bias optimization tensor, to regenerate a new target reply text using the new decoding matrix multiplication result.

[0092] Specifically, after generating the target reply text, the embodiment of the present application can first collect multi-modal feedback data such as text evaluation, voice feedback and semantic annotation of the target user based on a multi-modal feedback collection strategy, extract subjective satisfaction scores from text evaluation, extract emotional tendency features from voice feedback, and capture key information missing points from semantic annotation.

[0093] Secondly, the embodiment of the present application can perform cross-modal fusion analysis on multi-modal feedback data, extract text evaluation information such as semantic compatibility, content integrity and logical coherence through natural language processing, and supplement emotional tendency scores through voice emotion recognition technology to form a multi-dimensional evaluation data set; meanwhile, the semantic features, grammatical structure features, content coherence features and information density features of the target reply text are extracted to construct a text feature vector library.

[0094] Thirdly, the embodiment of the present application can use a feature correlation algorithm driven by an attention mechanism to perform dimension-by-dimension matching between the text feature vector and the multi-dimensional evaluation data, and calculate the accuracy comprehensive score of the target reply text through weighted summation, and combine the preset difference threshold to divide the feature matching difference level into three levels of high, medium and low (for example, high-level difference corresponding to semantic feature mismatch).

[0095] After that, the embodiment of the present application can dynamically allocate and adjust the weight based on the difference level; in the actual execution process, the embodiment of the present application can use large-step gradient descent to adjust the core layer parameters of the weight reasoning matrix for high-level difference; for medium and low-level difference, the edge layer parameters and bias reasoning tensor are fine-tuned and optimized through small-step fine-tuning. Recalculate the multiplication and addition results of the decoding matrix after adjustment, and then generate a new target reply text that better meets the user's needs.

[0096] Thus, the embodiment of the present application can rely on multi-modal feedback to dynamically optimize the autoregressive language model, accurately match user needs, and continuously improve the accuracy and compatibility of the target reply text generation, thereby enhancing the self-adaptation ability of the model.

[0097] It should be noted that in the actual execution process, the method for calculating and optimizing and accelerating the autoregressive language model based on the tensor decomposition strategy of the embodiment of the present application not only accelerates the calculation of the autoregressive language model, but also optimizes other models with a large number of matrix multiplication or vector multiplication matrix operations to accelerate their calculation process, thereby achieving the purpose of performance improvement.

[0098] In addition, the embodiment of the present application can also adjust the rank parameter by dynamically adapting the data features, reduce redundant calculation while ensuring decomposition accuracy, and improve decomposition efficiency and adaptability of different feature data, the specific process is as follows: 1. Perform quantization and rearrangement operation on the vector or matrix to convert it into a high-dimensional tensor, and input the high-dimensional tensor as the original object of feature extraction into the feature complexity evaluation module; 2. The feature complexity evaluation module receives the high-dimensional tensor, extracts the element distribution density, non-zero element aggregation degree, local feature variance and other multi-dimensional features corresponding to the high-dimensional tensor, and generates a feature set, which is input into the complexity scoring model; 3. The complexity score model calculates a comprehensive complexity score based on the input feature set through a pre-set feature weight distribution rule, converts the score into a dynamic rank adjustment coefficient according to a mapping relationship, and inputs the coefficient as a reference for setting the dimension of the tensor column decomposition phase; 4. The tensor column decomposition phase sets the initial dimension range of the low-rank core based on the dynamic rank adjustment coefficient, and simultaneously monitors the feature complexity changes of the local block of the tensor in real time to generate local block complexity data, which serves as a judgment basis for dimension adjustment; 5. According to the local block complexity data, if the local block complexity is higher than a pre-set threshold, the dimension of the corresponding low-rank core is increased in proportion to the dynamic rank adjustment coefficient; if the local block complexity is lower than the threshold, the dimension is decreased in proportion, and a dynamic low-rank core set adapted to different regional features is generated, which is used as the input of the subsequent matrix or vector multiplication step to replace the fixed rank core for operation.

[0099] Therefore, the embodiments of the present application can break the fixed low-rank setting, so that the rank in the decomposition process dynamically adapts to the data features, improving the balance between decomposition accuracy and efficiency.

[0100] As an implementable way, the embodiments of the present application can also realize the reuse and migration of decomposition core parameters between different levels of the autoregressive language model, reduce redundant calculation, and the specific process is as follows: Step 1. In the model training phase, the low-rank cores of the weight matrix of each level after completed decomposition are collected, the dimension parameters, element distribution features and operation response modes of each low-rank core are extracted, and a core feature data set is formed, which can be used as the input of step 2 for similarity calculation; Step 2. Based on the core feature data set of step 1, the feature matching degree of the low-rank cores of adjacent levels is calculated, and a matching threshold is set to screen out low-rank cores meeting the sharing conditions, construct a core parameter pool according to the level correlation, and mark the reuse priority, which can be used as the retrieval resource input of step 3; Step 3. When performing quantized tensor column decomposition on the high-level matrix, first search the core parameter pool of step 2 according to the structural features of the current matrix, match and adapt the shared low-rank cores, and output the core matching result, which can be used as the decomposition basis input of step 4; Step 4. Based on the core matching result of step 3, embed the shared low-rank cores into the decomposition framework of the high-level matrix, only perform targeted decomposition on the unmatched feature regions, and generate a composite low-rank core that fuses the shared core and the new decomposition core, which can be used as the operation input of step 5; Step 5. The composite low-rank core of step 4 is used for matrix multiplication operation, the reuse effect of the shared core is recorded, and the priority of the parameter pool is updated, and the updated parameter pool can be used as the resource input for the next round of cross-layer decomposition. Therefore, the embodiments of the present application can reduce repeated decomposition calculation by sharing the decomposed cores of the adaptive layers, reduce the storage requirement, and ensure the continuity of the hierarchical feature transmission.

[0101] In summary, the embodiments of the present application can effectively reduce the storage amount of the matrix through the tensor decomposition operation, and the operator multiplication calculation amount after tensor decomposition is very small. Moreover, with the calculation iteration of the model (such as the autoregressive language model) containing a large number of matrix multiplication calculations in the training and inference process, the effect of reducing the calculation amount is more significant.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0103] The embodiments of the present application also provide a reply text generation device.

[0104] As shown in Figure 3 The reply text generation device 10 includes a pre-filling module 100, a matrix multiplication and addition module 200, a decoding module 300, and a text generation module 400.

[0105] The pre-filling module 100 is configured to obtain initial text information of a target user, input the initial text information into a pre-trained autoregressive language model, generate a feature matrix corresponding to the initial text information, and perform a preset matrix-quantized tensor column decomposition operation on the feature matrix to obtain a corresponding feature matrix array decomposition result.

[0106] The matrix multiplication and addition module 200 is configured to obtain a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model, and perform a matrix multiplication and addition operation on the feature matrix array decomposition result, the weight inference matrix, and the bias inference tensor in a pre-filling stage of an inference process to obtain a pre-filling matrix multiplication and addition result corresponding to the autoregressive language model.

[0107] The decoding module 300 is configured to generate a one-dimensional feature vector of a current token based on the pre-filling matrix multiplication and addition result in a decoding stage of the inference process, perform a preset quantization operation on the one-dimensional feature vector to obtain a target feature tensor satisfying a preset dimension requirement, and perform a preset vector-quantized tensor column decomposition operation on the target feature tensor to obtain a corresponding vector column decomposition result.

[0108] The text generation module 400 is configured to multiply the vector column decomposition result and the weight inference matrix to obtain a corresponding vector matrix product result, and calculate a decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the vector matrix product result and the bias inference tensor, so as to generate the target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0109] Optionally, in an embodiment of the present application, the reply text generation device 10 further comprises a data set construction module, a feature extraction module, a zero padding module, a quantized tensor column decomposition module and an iterative training module.

[0110] The data set construction module is configured to collect different types of text data before obtaining the initial text information of the target user, and perform a format conversion operation on the text data to generate corresponding standard text data, and construct a text training data set through the standard text data, and input the initial text information into the pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information.

[0111] The feature extraction module is configured to input the text training data in the text training data set into the autoregressive language model to generate a feature training matrix corresponding to the text training data.

[0112] The zero padding module is configured to perform a preset zero padding operation on the feature training matrix to obtain a corresponding feature zero padding training matrix, and perform a preset element rearrangement process on the feature zero padding training matrix to generate a corresponding feature training tensor operator, and perform a preset matrix quantized tensor column decomposition operation on the feature training tensor operator to obtain a corresponding feature training decomposition matrix.

[0113] The quantized tensor column decomposition module is configured to obtain a preset weight matrix and a preset bias tensor in a tensor decomposition form corresponding to the autoregressive language model, and perform a matrix multiplication and addition operation on the feature training decomposition matrix, the preset weight matrix and the bias tensor to obtain a corresponding training process matrix multiplication and addition result.

[0114] The iterative training module is configured to calculate a matrix multiplication and addition error corresponding to the training process matrix multiplication and addition result before and after the decomposition of the feature training matrix based on the feature training matrix and the feature training decomposition matrix, and iteratively train the autoregressive language model based on the matrix multiplication and addition error and the training process matrix multiplication and addition result, respectively.

[0115] Optionally, in an embodiment of the present application, the pre-padding module 100 comprises a rearrangement unit and a first decomposition unit.

[0116] The rearranging unit is configured to perform a preset zero padding operation on the feature matrix to obtain a feature zero matrix satisfying a preset matrix dimension requirement, and perform a preset rearranging process on elements in the feature zero matrix to generate a target dimension tensor operator corresponding to the feature matrix.

[0117] The first decomposition unit is configured to obtain dimension information of the target dimension tensor operator, determine a matrix quantized tensor column decomposition strategy according to the dimension information, and decompose the target dimension tensor operator by using the matrix quantized tensor column decomposition strategy to obtain a feature matrix array decomposition result.

[0118] Optionally, in an embodiment of the present application, the decoding module 300 comprises a determining unit and a second decomposition unit.

[0119] The determining unit is configured to determine vector scale information of the one-dimensional feature vector, and perform a preset element rearranging operation on the one-dimensional feature vector based on the vector scale information to obtain a target feature tensor satisfying a preset dimension requirement.

[0120] The second decomposition unit is configured to perform a preset tensor column decomposition operation on the target feature tensor to obtain a plurality of sub-vectors corresponding to the one-dimensional feature vector, and determine a vector column decomposition result according to the plurality of sub-vectors.

[0121] Optionally, in an embodiment of the present application, the text generation module 400 comprises an evaluation unit, a superposition unit and a feature mapping unit.

[0122] The evaluation unit is configured to obtain a plurality of historical vector matrix product results of the autoregressive language model in a historical decoding stage and a plurality of historical vectors of historical input data, perform semantic correlation degree evaluation on the plurality of historical vector matrix product results and the plurality of historical vectors to obtain corresponding correlation degree evaluation data, and determine a historical feature correlation factor corresponding to the autoregressive language model by using the correlation degree evaluation data.

[0123] The superposition unit is configured to perform element-by-element superposition operation on the vector matrix product result and the bias inference tensor to obtain a decoding matrix multiplication result, and calculate a target decoding matrix multiplication result corresponding to the autoregressive language model based on the decoding matrix multiplication result and the historical feature correlation factor.

[0124] The feature mapping unit is configured to perform feature mapping on the target decoding matrix multiplication result to obtain a feature matching degree sequence corresponding to the preset text database, select at least one reply text satisfying a preset matching degree requirement from the preset text database based on a feature matching degree value of the feature matching degree sequence, and splice the at least one reply text to generate a target reply text.

[0125] Optionally, in an embodiment of the present application, the reply text generation device 10 further comprises a collection module, a feature association module and an adjustment module.

[0126] The collection module is configured to collect multi-modal feedback data of the target user on the target reply text after generating the target reply text corresponding to the initial text information according to the decoding matrix multiplication result, analyze the multi-modal feedback data to obtain corresponding analysis data, and extract a plurality of text evaluation information corresponding to the analysis data, wherein the plurality of text evaluation information includes semantic fit degree, content completeness and logical coherence.

[0127] The feature association module is configured to extract a plurality of text features of the target reply text, and perform feature association operations on the plurality of text features and the plurality of text evaluation information to determine an accuracy score of the target reply text and a feature matching difference level, wherein the plurality of text features include semantic features, syntax structure features and content coherence features.

[0128] The adjustment module is configured to adjust the weight inference matrix and the bias inference tensor of the autoregressive language model according to the accuracy score and the feature matching difference level to obtain a corresponding weight optimization matrix and a bias optimization tensor, and calculate a new decoding matrix multiplication result corresponding to the autoregressive language model based on the weight optimization matrix and the bias optimization tensor, to regenerate a new target reply text using the new decoding matrix multiplication result.

[0129] Optionally, in an embodiment of the present application, the quantized tensor column decomposition module comprises an acquisition unit, a window length determination unit, a construction unit and a dimension expansion unit.

[0130] The acquisition unit is configured to acquire hierarchical structure information of the autoregressive language model, and analyze the hierarchical structure information to determine input-output feature dimensions and inter-level connection relationships corresponding to the autoregressive language model according to the analyzed hierarchical structure information.

[0131] The window length determination unit is configured to determine a context window length of the autoregressive language model, and determine an action range and a matrix operation dimension constraint of an initial weight matrix and an initial bias tensor corresponding to the autoregressive language model based on the input-output feature dimensions, the context window length and the inter-level connection relationships.

[0132] The construction unit is configured to determine a number of hidden layer units and a number of attention heads of the autoregressive language model, determine a row and column size of the initial weight matrix according to the number of hidden layer units and the number of attention heads, and construct the initial weight matrix and the initial bias tensor based on the row and column size, the action range and the matrix operation dimension constraint.

[0133] The dimension expansion unit is configured to perform a preset dimension expansion operation on the initial weight matrix and the initial bias tensor respectively to obtain corresponding expanded matrices, perform a quantized tensor column decomposition operation on the expanded matrices to obtain corresponding weight decomposition matrices and bias decomposition matrices, and determine the preset weight matrix and the preset bias tensor according to the weight decomposition matrices and the bias decomposition matrices.

[0134] Optionally, in an embodiment of the present application, the iterative training module comprises a first calculation unit, a second calculation unit, an error evaluation unit, a marking unit and a training unit.

[0135] The first calculation unit is configured to calculate a pre-decomposition matrix product result of the feature training matrix and the initial weight matrix, and calculate a corresponding pre-decomposition matrix multiplication result according to the pre-decomposition matrix product result and the initial bias tensor.

[0136] The second calculation unit is configured to calculate a post-decomposition matrix product result of the feature training decomposition matrix and the preset weight matrix, and calculate a corresponding post-decomposition matrix multiplication result according to the post-decomposition matrix product result and the preset bias tensor.

[0137] The error evaluation unit is configured to calculate a corresponding matrix multiplication error based on the pre-decomposition matrix multiplication result and the post-decomposition matrix multiplication result, and input the matrix multiplication error into a pre-constructed error evaluation model to output corresponding element-level difference information.

[0138] The marking unit is configured to determine a corresponding error size and an error distribution position according to the element-level difference information, construct a corresponding error vector through the error size and the error distribution position, and mark an operation link in the error vector that meets a preset error contribution requirement to generate a marked error vector.

[0139] The training unit is configured to adjust an element mapping rule of a preset element rearrangement operation based on the marked error vector, and perform a preset matrix quantized tensor column decomposition operation on the feature training matrix, the initial weight matrix and the initial bias tensor using the adjusted element mapping rule to obtain corresponding decomposition data, and train the autoregressive language model based on the decomposition data.

[0140] Optionally, in an embodiment of the present application, the iterative training module further comprises a feature error gradient calculation unit and a convergence judgment unit.

[0141] The feature error gradient calculation unit is configured to calculate a feature error gradient of each iteration training process based on a training process matrix multiplication result, and determine a gradient direction of the feature error gradient to adjust the preset weight matrix and the preset bias tensor according to the gradient direction to obtain a weight adjustment matrix and a bias adjustment tensor after each iteration training.

[0142] The convergence judgment unit is configured to iteratively train the autoregressive language model by using the weight adjustment matrix and the bias adjustment tensor until the autoregressive language model meets preset convergence requirements, so as to obtain a final weight adjustment matrix and a final bias adjustment tensor, and determine a weight inference matrix and a bias inference tensor according to the final weight adjustment matrix and the final bias adjustment tensor, and generate the trained autoregressive language model.

[0143] The features of the embodiments of the reply text generation device can refer to the related descriptions of the embodiments of the reply text generation method, which will not be repeated here.

[0144] Embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above reply text generation method embodiments.

[0145] Embodiments of the present application also provide a non-volatile computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above reply text generation method embodiments when running.

[0146] In an exemplary embodiment, the non-volatile computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0147] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above reply text generation method embodiments.

[0148] Embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above reply text generation method embodiments.

[0149] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] The foregoing has provided a detailed description of a response text generation method, apparatus, device, and medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for generating response text, characterized in that, Includes the following steps: The initial text information of the target user is obtained and input into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information. The feature matrix is ​​then subjected to a preset matrix quantization tensor column decomposition operation to obtain the corresponding feature matrix column decomposition result. Obtain the weight inference matrix and bias inference tensor in the tensor decomposition form corresponding to the autoregressive language model, and perform matrix multiplication and addition operations on the feature matrix column decomposition result, the weight inference matrix and the bias inference tensor in the pre-filling stage of the inference process to obtain the pre-filled matrix multiplication and addition result corresponding to the autoregressive language model; During the decoding stage of the reasoning process, a one-dimensional feature vector of the current word is generated based on the multiplication and addition results of the pre-filled matrix, and the one-dimensional feature vector is subjected to preset quantization processing to obtain a target feature tensor that meets the preset dimension requirements. Furthermore, the target feature tensor is subjected to preset vector quantization tensor column decomposition operation to obtain the corresponding vector column decomposition results. The vector column decomposition result is multiplied by the weight inference matrix to obtain the corresponding vector matrix product result. Based on the vector matrix product result and the bias inference tensor, the decoding matrix multiplication result corresponding to the autoregressive language model is calculated to generate the target response text corresponding to the initial text information according to the decoding matrix multiplication result.

2. The response text generation method according to claim 1, characterized in that, Before acquiring the initial text information of the target user and inputting the initial text information into a pre-trained autoregressive language model to generate the feature matrix corresponding to the initial text information, the method further includes: Collect different types of text data, perform format conversion operations on the text data to generate corresponding standard text data, and construct a text training dataset using the standard text data; The text training data in the text training dataset is input into the autoregressive language model to generate the feature training matrix corresponding to the text training data; The feature training matrix is ​​padded with zeros to obtain the corresponding feature zero-padding training matrix. The feature zero-padding training matrix is ​​then rearranged with elements to generate the corresponding feature training tensor operator. Finally, the feature training tensor operator is decomposed into a matrix quantization tensor column to obtain the corresponding feature training decomposition matrix. Obtain the preset weight matrix and preset bias tensor in the tensor decomposition form corresponding to the autoregressive language model, and perform matrix multiplication and addition operations on the feature training decomposition matrix, the preset weight matrix and the bias tensor to obtain the corresponding training process matrix multiplication and addition results; Based on the feature training matrix and the feature training decomposition matrix, the matrix multiplication and addition error corresponding to the matrix multiplication and addition results of the training process before and after the feature training matrix decomposition is calculated, and the autoregressive language model is iteratively trained based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process, respectively.

3. The response text generation method according to claim 1, characterized in that, The step of performing a preset matrix quantization tensor column decomposition operation on the feature matrix to obtain the corresponding feature matrix column decomposition result includes: The feature matrix is ​​padded with zeros in a preset manner to obtain a feature zero-padding matrix that meets the preset matrix dimension requirements. The elements in the feature zero-padding matrix are then rearranged in a preset manner to generate the target dimension tensor operator corresponding to the feature matrix. The dimension information of the target dimension tensor operator is obtained, and a matrix quantization tensor column decomposition strategy is determined based on the dimension information. The target dimension tensor operator is then decomposed using the matrix quantization tensor column decomposition strategy to obtain the feature matrix column decomposition result.

4. The response text generation method according to claim 1, characterized in that, The step of performing a preset quantization process on the one-dimensional feature vector to obtain a target feature tensor that meets the preset dimension requirements, and performing a preset vector quantization tensor column decomposition operation on the target feature tensor to obtain the corresponding vector column decomposition result, includes: Determine the vector size information of the one-dimensional feature vector, and based on the vector size information, perform a preset element rearrangement operation on the one-dimensional feature vector to obtain a target feature tensor that meets the preset dimension requirements; A preset tensor column decomposition operation is performed on the target feature tensor to obtain multiple sub-vectors corresponding to the one-dimensional feature vector, and the vector column decomposition result is determined based on the multiple sub-vectors.

5. The response text generation method according to claim 1, characterized in that, The step of calculating the decoding matrix multiplication result corresponding to the autoregressive language model based on the vector matrix multiplication result and the bias inference tensor, and generating the target response text corresponding to the initial text information according to the decoding matrix multiplication result, includes: The autoregressive language model obtains multiple historical vector matrix product results and multiple historical vectors of historical input data during the historical decoding stage, and performs semantic relevance evaluation on the multiple historical vector matrix product results and the multiple historical vectors to obtain corresponding relevance evaluation data, and determines the historical feature relevance factor corresponding to the autoregressive language model through the relevance evaluation data. The vector matrix multiplication result and the bias inference tensor are superimposed element-wise to obtain the decoding matrix multiplication result. Based on the decoding matrix multiplication result and the historical feature association factor, the target decoding matrix multiplication result corresponding to the autoregressive language model is calculated. The target decoding matrix multiplication and addition results are subjected to feature mapping to obtain a feature matching degree sequence corresponding to a preset text database. Based on the feature matching degree value of the feature matching degree sequence, at least one response text that meets the preset matching degree requirement is selected from the preset text database. The at least one response text is then concatenated to generate the target response text.

6. The response text generation method according to claim 1, characterized in that, After generating the target response text corresponding to the initial text information based on the multiplication and addition results of the decoding matrix, the method further includes: Collect multimodal feedback data of the target user's response text to the target, and parse the multimodal feedback data to obtain corresponding parsed data. Extract multiple text evaluation information corresponding to the parsed data, wherein the multiple text evaluation information includes semantic fit, content completeness and logical coherence. Extract multiple text features from the target response text, and perform feature association operations on the multiple text features and the multiple text evaluation information to determine the accuracy score and feature matching difference level of the target response text. The multiple text features include semantic features, grammatical structure features and content coherence features. The weight inference matrix and the bias inference tensor of the autoregressive language model are adjusted according to the accuracy score and the feature matching difference level to obtain the corresponding weight optimization matrix and bias optimization tensor. Based on the weight optimization matrix and the bias optimization tensor, a new decoding matrix multiplication and addition result corresponding to the autoregressive language model is calculated to regenerate a new target response text using the new decoding matrix multiplication and addition result.

7. The response text generation method according to claim 2, characterized in that, The step of obtaining the preset weight matrix and preset bias tensor of the tensor decomposition form corresponding to the autoregressive language model includes: Obtain the hierarchical structure information of the autoregressive language model and parse the hierarchical structure information to determine the input and output feature dimensions and inter-hierarchical connection relationships of the autoregressive language model based on the parsed hierarchical structure information; Determine the context window length of the autoregressive language model, and based on the input and output feature dimensions, the context window length, and the inter-level connectivity, determine the scope of action and matrix operation dimension constraints of the initial weight matrix and initial bias tensor corresponding to the autoregressive language model. The number of hidden layer units and the number of attention heads of the autoregressive language model are determined, and the row and column size of the initial weight matrix is ​​determined based on the number of hidden layer units and the number of attention heads. Based on the row and column size, the scope of action and the matrix operation dimension constraints, the initial weight matrix and the initial bias tensor are constructed. The initial weight matrix and the initial bias tensor are expanded by a preset dimension to obtain corresponding expanded matrices. The expanded matrices are then decomposed into quantized tensor columns to obtain corresponding weight decomposition matrices and bias decomposition matrices. The preset weight matrix and the preset bias tensor are then determined based on the weight decomposition matrices and the bias decomposition matrices.

8. The response text generation method according to claim 7, characterized in that, The step of calculating the matrix multiplication and addition error corresponding to the matrix multiplication and addition results of the training process before and after the feature training matrix decomposition based on the feature training matrix and the feature training decomposition matrix, and iteratively training the autoregressive language model based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process respectively, includes: Calculate the undecomposed matrix product of the feature training matrix and the initial weight matrix, and calculate the corresponding undecomposed matrix multiplication and addition result based on the undecomposed matrix product and the initial bias tensor; Calculate the product of the decomposed matrix of the feature training decomposition matrix and the preset weight matrix, and calculate the corresponding product of the decomposed matrix based on the product of the decomposed matrix and the preset bias tensor; Based on the matrix multiplication and addition results before and after decomposition, the corresponding matrix multiplication and addition error is calculated, and the matrix multiplication and addition error is input into the pre-constructed error evaluation model to output the corresponding element-level difference information. The corresponding error magnitude and error distribution location are determined based on the element-level difference information, and a corresponding error vector is constructed using the error magnitude and error distribution location. The computational steps in the error vector that meet the preset error contribution requirements are marked to generate a marked error vector. Based on the labeled error vector, the element mapping rules of the preset element rearrangement operation are adjusted, and the preset matrix quantization tensor column decomposition operation is performed again on the feature training matrix, the initial weight matrix, and the initial bias tensor using the adjusted element mapping rules to obtain the corresponding decomposed data. Based on the decomposed data, the autoregressive language model is trained.

9. The response text generation method according to claim 2, characterized in that, The iterative training of the autoregressive language model based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process includes: Based on the matrix multiplication and addition results of the training process, the feature error gradient of each iteration of the training process is calculated, and the gradient direction of the feature error gradient is determined. The preset weight matrix and the preset bias tensor are adjusted according to the gradient direction to obtain the weight adjustment matrix and bias adjustment tensor after each iteration of training. The autoregressive language model is iteratively trained using the weight adjustment matrix and the bias adjustment tensor until it meets the preset convergence requirements, so as to obtain the final weight adjustment matrix and bias adjustment tensor. The weight inference matrix and the bias inference tensor are then determined based on the final weight adjustment matrix and bias adjustment tensor, and the trained autoregressive language model is generated.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the response text generation method as described in any one of claims 1 to 9 when executing the computer program.

Images