Question answering methods and products based on data-driven regularization and large language models

Abstract

This application relates to the field of artificial intelligence technology and discloses a question-answering method and product based on a data-driven regularization-based question-answering large language model. The method includes: obtaining the device parameter values ​​of the target device to which the target question-answering large language model is to be deployed, and determining the channels that need pruning in the pre-trained original question-answering large language model; updating the model parameters of the pre-trained original question-answering large language model based on question-answering sample data using a loss function with a regularization loss term; performing channel pruning on the updated question-answering large language model according to the channels that need pruning, and obtaining the target question-answering large language model based on the channel-pruned question-answering large language model; and obtaining the answer by reasoning through the target question-answering large language model for questions from the target device. This method can reduce the size of the large language model while maintaining model performance, facilitating the deployment of the large language model to lightweight devices.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a question-answering method and product based on a data-driven regularization question-answering large language model. Background Technology

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLMs) products are widely used in various fields such as natural language processing, intelligent customer service, healthcare, and education. Due to the widespread adoption of AI products, LLMs are being deployed on lightweight devices (such as smartphones and tablets) to meet user needs. However, when users perform question reasoning using LLMs on lightweight devices, they encounter problems such as insufficient device hardware configuration, high memory usage, high computational resource consumption, and excessively long question reasoning response times, severely impacting the user experience.

[0003] Therefore, it is necessary to find a way to improve the question-answering efficiency of large language model products on lightweight devices, reduce resource consumption, and improve user experience. Summary of the Invention

[0004] In view of this, this application aims to propose a question-answering method and product based on a data-driven regularization-based question-answering large language model, so as to improve the question-answering efficiency of the question-answering large language model product on lightweight devices, reduce resource consumption, and improve user experience.

[0005] To achieve the above objectives, the technical solution of this application is as follows:

[0006] The first aspect of this application provides a question-answering method based on a data-driven regularization large language model, the method comprising:

[0007] Obtain the device parameter values ​​of the target device for the target question-answering language model to be deployed. The device parameter values ​​include at least one of the following: storage space size and computing speed.

[0008] Based on the device parameter values, determine the channels that need pruning in the pre-trained original question-answering language model. The pre-trained original question-answering language model is used to reason about questions from the client and obtain answers.

[0009] Based on question-and-answer sample data, the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term, wherein the regularization loss term is determined according to the parameter matrix of the channels that need to be pruned.

[0010] According to the channels that need to be pruned, channel pruning is performed on the question-answering large language model after the model parameters have been updated. Based on the question-answering large language model after channel pruning, the target question-answering large language model is obtained.

[0011] For questions from the target device, the answer is obtained by reasoning through the target question-answering big language model.

[0012] Optionally, based on the device parameter values, determine the channels in the pre-trained original question-answering large language model that need pruning, including:

[0013] Based on the device parameter values, determine the number of parameters of the target question-answering large language model;

[0014] The pruning ratio is determined based on the number of parameters and the original number of parameters of the pre-trained original question-answering large language model;

[0015] Based on the pruning ratio, determine the channels that need pruning in the pre-trained original question-answering large language model.

[0016] Optionally, based on the pruning ratio, the channels of the pre-trained original question-answering large language model that need pruning are determined, including:

[0017] Based on the pruning ratio p%, a pseudo-index selection matrix R is constructed. The pseudo-index selection matrix R is a diagonal matrix, and the elements of the pseudo-index selection matrix R are 1 for p% and 0 for 1-p%.

[0018] Based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering big language model, determine the regularization loss value corresponding to the regularization loss term;

[0019] The model parameters of the pre-trained original question-answering large language model are updated using a loss function with a regularization loss term, including:

[0020] For the first question in the question-and-answer sample data, the predicted answer is obtained through the pre-trained original question-and-answer large language model;

[0021] Based on the difference between the predicted answer and the first answer in the question-answering sample data, the original question-answering loss value of the pre-trained original question-answering large language model is determined;

[0022] The model parameters of the pre-trained original question-answering large language model are updated based on the original question-answering loss value and the regularization loss value corresponding to the regularization loss term.

[0023] Optionally, channel pruning is performed on the question-answering large language model after the model parameters have been updated, according to the channels that need pruning, including:

[0024] Based on the pseudo-index selection matrix R and the identity matrix I, construct the channel pruning matrix. S , S=IR ;

[0025] Based on the parameter matrix of the question-answering large language model after model parameter updates The parameter matrix of the question-answering large language model after channel pruning is obtained by combining the channel pruning matrix S. .

[0026] Optionally, based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering large language model, the regularization loss value corresponding to the regularization loss term is determined, including:

[0027] Based on the pseudo-index selection matrix R and the attention mechanism module of the pre-trained original question-answering big language model, i The original parameter matrix of the layer , , , Determine the first i Attention regularization loss value of the layer;

[0028] Based on the pseudo-index selection matrix R and the feedforward neural network module of the pre-trained original question-answering big language model, i The original parameter matrix of the up projection in the layer and the original parameter matrix of the downward projection Determine the first i Regularized loss value of a layered feedforward neural network;

[0029] The regularization loss value of the remaining modules is determined based on the pseudo-index selection matrix R and the original parameter matrix of the remaining modules of the pre-trained original question-answering big language model.

[0030] Optionally, the method further includes:

[0031] Obtain the question-answering sample dataset used by the original question-answering large language model obtained through pre-training;

[0032] Randomly extract a subset of the question-and-answer sample dataset to obtain a subset of the question-and-answer sample data;

[0033] Based on question-and-answer sample data, the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term, including:

[0034] Each question-and-answer sample data in the subset of question-and-answer sample data is input into the pre-trained original question-and-answer large language model, and the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term.

[0035] Based on the question-answering large language model after channel pruning, the target question-answering large language model is obtained, including:

[0036] Using the subset of question-and-answer sample data, the question-and-answer large language model after channel pruning is fine-tuned to obtain the target question-and-answer large language model.

[0037] Optionally, the method further includes:

[0038] For the test question-and-answer sample data, obtain the predicted answer generated by the target question-and-answer large language model and the predicted answer generated by the pre-trained original question-and-answer large language model;

[0039] If the difference between the predicted answer generated by the target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is greater than or equal to a preset difference, the channels that need to be pruned are continuously adjusted, and the process returns to the step: inputting question-answering sample data into the pre-trained original question-answering language model, and using a loss function with a regularization loss term to update the model parameters of the pre-trained original question-answering language model until the difference between the predicted answer generated by the final target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is less than the preset difference;

[0040] For questions from the target device, the answer is obtained by reasoning through the final target question-answering large language model.

[0041] Optionally, continuously adjusting the channels that require pruning includes:

[0042] The number of elements that are 1 in the pseudo-index selection matrix R is continuously adjusted, or the position of the elements that are 1 in the pseudo-index selection matrix R is continuously adjusted.

[0043] According to a second aspect of the embodiments of this application, a question-answering system based on a data-driven regularization question-answering large language model is provided, for implementing the steps of the method provided in the first aspect of the embodiments of this application, the system comprising:

[0044] The preprocessing module is configured to obtain the device parameter values ​​of the target device for the target question-answering language model to be deployed, the device parameter values ​​including at least one of the following: storage space size and computing speed; based on the device parameter values, determine the channels that need to be pruned in the pre-trained original question-answering language model, the pre-trained original question-answering language model being used to reason about questions from the client and obtain answers;

[0045] The information transfer module is configured to update the model parameters of the pre-trained original question-answering large language model based on question-answering sample data using a loss function with a regularization loss term, wherein the regularization loss term is determined according to the parameter matrix of the channels that need to be pruned.

[0046] The pruning module is configured to perform channel pruning on the question-answering language model after the model parameters have been updated, according to the channels that need to be pruned, and obtain the target question-answering language model based on the channel-pruned question-answering language model.

[0047] The reasoning module is configured to perform reasoning on questions from the target device using the target question-answering language model to obtain answers.

[0048] According to a third aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps of the method provided in the first aspect of the present application.

[0049] This application employs a question-answering method based on data-driven regularization and a large language model. First, the device parameter values ​​(storage space size, computing speed, etc.) of the target device (i.e., a lightweight device) where the pruned target question-answering large language model will ultimately be deployed are determined. Based on these device parameter values, the channels that need pruning in the pre-trained original model are determined, ensuring the pruned model can run normally on the target device. After determining the pruned channels, a regularization loss term is determined based on the parameter matrix of the required pruned channels. The pre-trained original model is then trained using a loss function with the regularization loss term to update the model parameters, thereby transferring important information from the model to the retained channels. Next, the original question-answering large language model is pruned according to the required channels to obtain the target question-answering large language model. The pruned lightweight model is then deployed on the lightweight target device to infer answers to user-submitted questions on the target device.

[0050] The pruned lightweight model of this application can run efficiently on lightweight devices, significantly reducing the response time for problem reasoning on the target device, while also reducing the requirements for hardware configuration and computing resources. Furthermore, because the model is pre-regularized to transfer important information to the channels that need to be preserved, the information loss caused by parameter removal during pruning is reduced, and the pruned model can maintain inference performance comparable to the original large language model. In addition, since model performance is preserved, there is no need to use a large amount of data for debugging to restore model performance after pruning, further saving time and computing resources for model deployment. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of a question-answering method based on a data-driven regularization question-answering large language model proposed in an embodiment of this application;

[0053] Figure 2 This is a flowchart of the regularization process of each layer of the large language model in one embodiment of this application;

[0054] Figure 3 This is a schematic diagram illustrating the migration of important information through a regularization process in one embodiment of this application;

[0055] Figure 4 This is a flowchart illustrating the pruning of a large language model according to an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of a question-answering system based on a data-driven regularization question-answering large language model proposed in an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0060] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects as detailed in this application.

[0062] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0063] Currently, large question-answering language models deployed on lightweight devices suffer from excessively long question inference response times and high demands on hardware configuration, memory usage, and computational overhead, severely impacting user experience. To improve model response efficiency, this application employs pruning to compress the model size, reducing computational overhead on lightweight devices and improving question inference response efficiency, thereby enhancing user experience. Traditional model pruning methods, such as structured pruning, reduce the size of large models by first selecting channels or layers to prune based on design metrics, followed by a recovery fine-tuning process. Since important information may reside in the selected pruned portions, direct pruning removes valuable information, leading to irreversible performance degradation; high pruning ratios can even cause model performance crashes. This necessitates extensive training data for recovery fine-tuning (RFT) after pruning to restore model performance, increasing time and computational costs, resulting in insignificant model compression and limiting the effectiveness of pruning schemes in reducing latency and improving throughput.

[0064] This application employs a method of first migrating valuable information to the remaining part of the model, followed by pruning. By transferring important information to the remaining part of the model in advance, it reduces information loss caused by removing some channels during pruning, thereby compressing the model size while maintaining the model's language modeling capabilities. This improves the question reasoning efficiency of large question-answering language models deployed on lightweight devices, reduces the model's requirements for device hardware configuration and computational overhead, and improves the user experience. The application will now be described in detail with reference to the accompanying drawings and embodiments.

[0065] Figure 1 This is a flowchart of a question-answering method based on a data-driven regularization large language model proposed in an embodiment of this application. Figure 1 As shown, the method includes:

[0066] S1: Obtain the device parameter values ​​of the target device for the target question-answering language model to be deployed. The device parameter values ​​include at least one of the following: storage space size and computing speed.

[0067] S2: Based on the device parameter values, determine the channels that need pruning in the pre-trained original question-answering language model, which is used to reason about questions from the client and obtain answers;

[0068] S3: Based on the question-and-answer sample data, update the model parameters of the pre-trained original question-and-answer large language model using a loss function with a regularization loss term, wherein the regularization loss term is determined according to the parameter matrix of the channels that need to be pruned.

[0069] S4: According to the channels that need to be pruned, perform channel pruning on the question-answering language model after the model parameters have been updated, and obtain the target question-answering language model based on the channel-pruned question-answering language model.

[0070] S5: For questions from the target device, reason through the target question-answering big language model to obtain the answer.

[0071] In this embodiment, the channels that need pruning in the original large language model are determined based on the device parameter values ​​of the target device. Specifically, the channels that need pruning in the original question-answering large language model can be determined based on at least one of the actual available storage space size and computing speed in the target device.

[0072] The original question-answering large language model is pre-trained using a widely used benchmark dataset. In this embodiment, question-answering sample data is determined based on the benchmark dataset for subsequent regularization. During regularization, a loss function with a regularization loss term is constructed based on the parameter matrix of the channels to be pruned. The parameters of the original question-answering large language model are updated using the question-answering sample data; that is, regularization is applied to the channels that need pruning using the constructed regularization loss term. This redistributes important information in the channels that need pruning to the unpruned components. After performing the regularization process, channel pruning is performed on the updated question-answering large language model according to the previously determined channels that need pruning, resulting in the target question-answering large language model. The target question-answering large language model is deployed to the target device, and then used to infer answers (i.e., inference results) from questions from the target device. Pruning reduces the size of the original large language model and the computational load of question reasoning. The pruned model is adapted to the hardware configuration, storage space, and computing speed of the target device, thus enabling efficient reasoning for questions from the target device, reducing question-answering response time, and improving user experience.

[0073] By transferring crucial information from the pruned portion of the model to the remaining portion through pre-regularization, information loss during subsequent pruning is reduced, maximizing the preservation of the pruned model's performance. This eliminates the need for extensive performance recovery tuning (RFT) using large amounts of training data, saving computational overhead and time costs associated with RFT and improving the efficiency of deploying large question-answering language models on lightweight devices. This approach maintains comparable inference performance between the pruned and original models even at high pruning ratios, meeting the deployment requirements of large language models in resource-constrained scenarios. It achieves high sparsity while maintaining performance, improving the inference speed and data throughput efficiency of large question-answering language models on lightweight devices.

[0074] As one embodiment of this application, determining the channels requiring pruning in the pre-trained original question-answering large language model based on the device parameter values ​​includes:

[0075] Based on the device parameter values, determine the number of parameters of the target question-answering large language model;

[0076] The pruning ratio is determined based on the number of parameters and the original number of parameters of the pre-trained original question-answering large language model;

[0077] Based on the pruning ratio, determine the channels that need pruning in the pre-trained original question-answering large language model.

[0078] In this embodiment, the device parameter values ​​include storage space size and processing speed. Based on these device parameter values, the channels requiring pruning are determined. First, the parameter count of the target question-answering language model to be deployed on the target device is determined. Then, based on the parameter count, the channels requiring pruning are determined from the original question-answering language model. This ensures that the resulting lightweight model (target question-answering language model) matches the device parameter values ​​of the target device, guaranteeing that the target device can properly run the target question-answering language model.

[0079] The number of parameters in a target question-answering large language model includes: storage parameters and computational parameters. The number of storage parameters is determined based on the actual available storage space on the target device. Specifically, the number of storage parameters for the target question-answering large language model is determined by comparing the memory occupied by each parameter in the model with the actual available storage space on the target device.

[0080] The number of computational parameters is determined based on the processing speed of the target device. The number of computational parameters for the target question-answering large language model is determined according to the processing speed of the target device and the computational load of a single inference operation.

[0081] In this embodiment, at least one of the storage parameters and the computational parameters is used to determine the channels that need pruning. Optionally, the storage parameters and the computational parameters are compared, and the smaller value is subtracted from the original parameters of the original question-answering language model to determine the pruning ratio, thereby ensuring the optimal running efficiency of the target question-answering language model on the target device. Based on the pruning ratio, the channels in the original question-answering language model that need pruning are determined.

[0082] As one embodiment of this application, determining the channels requiring pruning in the pre-trained original question-answering large language model based on the pruning ratio includes:

[0083] Based on the pruning ratio p%, a pseudo-index selection matrix R is constructed. The pseudo-index selection matrix R is a diagonal matrix, and the elements of the pseudo-index selection matrix R are 1 for p% and 0 for 1-p%.

[0084] Based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering big language model, determine the regularization loss value corresponding to the regularization loss term;

[0085] The model parameters of the pre-trained original question-answering large language model are updated using a loss function with a regularization loss term, including:

[0086] For the first question in the question-and-answer sample data, the predicted answer is obtained through the pre-trained original question-and-answer large language model;

[0087] Based on the difference between the predicted answer and the first answer in the question-answering sample data, the original question-answering loss value of the pre-trained original question-answering large language model is determined;

[0088] The model parameters of the pre-trained original question-answering large language model are updated based on the original question-answering loss value and the regularization loss value corresponding to the regularization loss term.

[0089] In one embodiment, the required pruning ratio p% of the pre-trained original question-answering large language model is determined based on device parameter values, and then the channels to be pruned are determined based on the pruning ratio. In this embodiment, the range of p in the pruning ratio p% is [10, 60]. A pseudo-index selection matrix R is constructed based on the pruning ratio p%, and this pseudo-index selection matrix is ​​used to construct the regularization loss term in the loss function. The pseudo-index selection matrix R is a diagonal matrix, and the pruning ratio is determined based on the pruning ratio. p% On the diagonal of the matrix p% The element is 1, and the remaining elements on the diagonal (i.e., 1) -p%The elements of the pseudo-index selection matrix R and the parameter matrix W of the original question-answering large language model are set to 0. Based on this pseudo-index selection matrix R and the parameter matrix W of the original question-answering large language model, the regularization loss term in the loss function is determined, and a loss function with the regularization loss term is constructed. The original question-answering large language model is trained based on this loss function to transfer important information. During training, for the first question in the question-answering sample data, the original question-answering large language model is used to infer and obtain the predicted answer. Furthermore, based on the difference between the predicted answer and the first answer to the first question in the question-answering sample data, the original question-answering loss value of the original question-answering large language model is determined. Based on this original question-answering loss value, the model parameters of the original question-answering large language model are updated.

[0090] As one implementation of this application, the regularization loss value corresponding to the regularization loss term is determined based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering large language model, including:

[0091] Based on the pseudo-index selection matrix R and the attention mechanism module of the pre-trained original question-answering big language model, i The original parameter matrix of the layer , , , Determine the first i Attention regularization loss value of the layer;

[0092] Based on the pseudo-index selection matrix R and the feedforward neural network module of the pre-trained original question-answering big language model, i The original parameter matrix of the up projection in the layer and the original parameter matrix of the downward projection Determine the first i Regularized loss value of a layered feedforward neural network;

[0093] The regularization loss value of the remaining modules is determined based on the pseudo-index selection matrix R and the original parameter matrix of the remaining modules of the pre-trained original question-answering big language model.

[0094] In one embodiment, the loss function of the model during the regularization process is a combination of language modeling loss and regularization loss, wherein the regularization loss includes the loss resulting from the regularization of the attention mechanism module, the forward propagation network and the rest of the model. Figure 2 This is a flowchart of the regularization process for each layer of a large language model in one embodiment of this application. For example... Figure 2 As shown, the regularization loss of the attention module is included in each layer of the model's regularization process. Regularized loss of the forward propagation network in FFN (Feedforward Neural Network) and the regularization loss of the remaining part of the model (including layer normalization) .

[0095] The specific expression for the loss function is as follows:

[0096] ;

[0097] in, The language modeling loss of the model is used to measure the difference between the model's predicted answer and the true answer; For the model number i The attention mechanism module of the layer uses regularized loss; For the model number i Regularization loss of the forward propagation network of the layer; This is the regularization loss for the remaining part of the model excluding the pruned portion; This represents the number of layers in the model.

[0098] Specifically, the model Attention mechanism module regularization loss of layer The expression is as follows:

[0099] ;

[0100] in R This represents the pseudo-index selection matrix (diagonal matrix) with a pruning ratio of... In this case, R There is on the middle diagonal The element is 1. The element is 0; The symbol represents a vector norm applied to the matrix column by column. , , , The first i The query matrix, key matrix, value matrix, and output matrix in the layer.

[0101] Model No. i Regularization loss of a layered FFN forward propagation network The expression is as follows:

[0102] ;

[0103] in, , The first i The upper projection matrix and lower projection matrix in the layer.

[0104] Regularization loss of the remaining part of the model excluding the pruned portion. The expression is as follows:

[0105] ;

[0106] in, Represents the data feature matrix. Represents the position encoding matrix. Representing the Layer normalized vector, Represents the language modeling matrix.

[0107] As one embodiment of this application, channel pruning is performed on the question-answering large language model after model parameter updates, according to the channels that need pruning, including:

[0108] Based on the pseudo-index selection matrix R and the identity matrix I, construct the channel pruning matrix. S , S=IR ;

[0109] Based on the parameter matrix of the question-answering large language model after model parameter updates The parameter matrix of the question-answering large language model after channel pruning is obtained by combining the channel pruning matrix S. .

[0110] In this embodiment, after completing the model regularization process, channel pruning is performed on the updated question-answering language model according to the previously determined channels that need pruning, removing the model parameter channels that have been regularized. Specifically, this is based on a pseudo-index selection matrix. R Define matrix ,in I It is the identity matrix. Using matrices... S Pruning is performed on the model's first... i The attention module of the layer performs pruning using the following expression:

[0111] ; ;

[0112] ; .

[0113] in, , , , The updated model parameters are respectively the first... The query matrix, key matrix, value matrix, and output matrix in the layer; , , , The first and second parts of the pruned model are respectively The query matrix, key matrix, value matrix, and output matrix in the layer.

[0114] Model No. The FFN forward propagation network of the layer is pruned using the following expression:

[0115] ; ;

[0116] in, , The updated model parameters are respectively the first... i The upper projection matrix and lower projection matrix in the layer; , The first and second parts of the pruned model are respectively i The upper projection matrix and lower projection matrix in the layer.

[0117] The remaining part of the model is pruned using the following expression:

[0118] ; ;

[0119] ; ;

[0120] in, The data feature matrix after model parameter updates. The position encoding matrix after model parameter updates. The updated model parameters i Layer normalized vector, Model the language model matrix after updating the model parameters; This is the data feature matrix of the pruned model. This is the position encoding matrix for the pruned model. For the pruned model, the first i Layer normalized vector, The language modeling matrix for the pruned model.

[0121] Figure 3 This is a schematic diagram illustrating the migration of important information through a regularization process in one embodiment of this application. For example... Figure 3 As shown, in this embodiment, before pruning the model, a regularization process is performed. Based on the question-and-answer sample data, the model parameters are updated in advance using a loss function with a regularization loss term, so that the important information contained in the channels to be pruned ( Figure 3 The information (marked with an exclamation mark) is transferred to the rest of the model. After the information transfer is complete, a pruning process is performed, using the S matrix to remove the channels to be pruned. Because the information of the channels to be pruned has been transferred to other channels in advance, the pruned model can still retain the key information of the original question-answering large language model.

[0122] In this embodiment, by performing regularization followed by pruning, the model size is effectively compressed while maintaining performance comparable to the original question-answering large language model. An S-matrix is ​​constructed to prune the regularized question-answering large language model. For the attention module and the forward propagation network, linear transformation and pruning using the S-matrix ensure that selected channels are effectively removed, allowing the pruned model to retain its key functionalities.

[0123] As one embodiment of this application, the method further includes:

[0124] Obtain the question-answering sample dataset used by the original question-answering large language model obtained through pre-training;

[0125] Randomly extract a subset of the question-and-answer sample dataset to obtain a subset of the question-and-answer sample data;

[0126] Based on question-and-answer sample data, the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term, including:

[0127] Each question-and-answer sample data in the subset of question-and-answer sample data is input into the pre-trained original question-and-answer large language model, and the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term.

[0128] Based on the question-answering large language model after channel pruning, the target question-answering large language model is obtained, including:

[0129] Using the subset of question-and-answer sample data, the question-and-answer large language model after channel pruning is fine-tuned to obtain the target question-and-answer large language model.

[0130] In one embodiment, before updating the parameters of the original question-answering large language model using a loss function with a regularization loss term, a question-answer sample dataset (e.g., a benchmark dataset) of the original question-answering large language model is obtained, and this dataset is randomly sampled to obtain a subset of question-answer sample data. The model's regularization process is performed using this extracted subset of question-answer sample data. The question-answer sample data from this subset is then input into the original question-answering large language model, and the model's parameters are updated using a loss function with a regularization loss term to migrate important information to the remaining channels that do not require pruning. Finally, pruning operations are performed on the updated large model.

[0131] In this embodiment, after pruning the question-answering large language model with updated parameters, the pruned model is further fine-tuned using data from the training phase (i.e., a subset of question-answering sample data) to obtain the final target question-answering large language model that can be deployed. Because this scheme first regularizes the transfer of important information before pruning the model, important information is retained in the components of the pruned model, and the model's performance is almost unaffected. Therefore, compared to traditional pruning schemes, this embodiment only requires a very small amount of training data to perform Regression-Free Tuning (RFT) on the pruned model to meet performance requirements. This significantly improves model compression efficiency and saves computational overhead and time costs during regularization and Regression-Free Tuning.

[0132] Figure 4 This is a flowchart illustrating the pruning of a large language model according to an embodiment of this application. Figure 4 As shown, in this embodiment, the parameters of the original question-and-answer large language model are first updated using a loss function with a regularization loss term. This transfers important information from channels to be pruned to the remaining parts of the model. Then, the model is pruned to obtain a pruned question-and-answer large language model. Further, using this subset of question-and-answer sample data, the pruned question-and-answer large language model is fine-tuned to obtain the final target question-and-answer large language model (i.e., the compressed model) deployed on the target device. This completes the compression of the original question-and-answer large language model while maintaining its performance.

[0133] As one embodiment of this application, the method further includes:

[0134] For the test question-and-answer sample data, obtain the predicted answer generated by the target question-and-answer large language model and the predicted answer generated by the pre-trained original question-and-answer large language model;

[0135] If the difference between the predicted answer generated by the target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is greater than or equal to a preset difference, the channels that need to be pruned are continuously adjusted, and the process returns to the step: inputting question-answering sample data into the pre-trained original question-answering language model, and using a loss function with a regularization loss term to update the model parameters of the pre-trained original question-answering language model until the difference between the predicted answer generated by the final target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is less than the preset difference;

[0136] For questions from the target device, the answer is obtained by reasoning through the final target question-answering large language model.

[0137] In one embodiment, test question-answer sample data is input into the target question-answering large language model domain and the original question-answering large language model, respectively. Predicted answers generated by the two models are obtained, and the difference between the two predicted answers is compared with a preset difference. If the difference between the two predicted answers is greater than the preset difference, the process returns to determine the pruning channel for the original large model, and a loss function with a regularization loss term is reconstructed based on the adjusted pruning channel. The parameters of the original question-answering large language model are updated using the reconstructed loss function, and then pruning and fine-tuning are performed to obtain a new target question-answering large language model. The difference between the two predicted answers output by the new target question-answering large language model and the original question-answering large language model is again compared with the preset difference.

[0138] Repeat the above steps until the difference between the two predicted answers output by the target question-answering language model and the original question-answering language model is less than a preset difference. At this point, the performance of the target question-answering language model is considered comparable to that of the original, thus completing the compression of the original question-answering language model and obtaining a lightweight model that can be deployed on the target device. Deploy the target question-answering language model on the target device and use it to infer answers to questions submitted by users on the target device.

[0139] In this embodiment, the inference performance of the target question-answering large language model is measured by a preset difference. If the preset difference requirement is not met, the target question-answering large language model is adjusted by returning to the modification pruning channel to ensure that the lightweight model after pruning has inference performance comparable to the original question-answering large language model.

[0140] As one embodiment of this application, continuously adjusting the channel that needs pruning includes:

[0141] The number of elements that are 1 in the pseudo-index selection matrix R is continuously adjusted, or the position of the elements that are 1 in the pseudo-index selection matrix R is continuously adjusted.

[0142] In one embodiment, if the difference between the predicted answer generated by the target question-answering large language model and the predicted answer generated by the pre-trained original question-answering large language model is greater than or equal to a preset difference, then the pruned channels need to be adjusted. Adjusting the pruned channels can involve adjusting the position of the pruned channels or the number of pruned channels. Specifically, the number of pruned channels is adjusted by changing the number of elements with a value of 1 on the diagonal of the pseudo-index selection matrix R; the more elements with a value of 1, the more channels need to be pruned. The corresponding positions of the pruned channels are adjusted by changing the position of the elements with a value of 1 on the diagonal of the pseudo-index selection matrix R.

[0143] Optionally, if the reasoning performance of the target question-answering large language model is not comparable to that of the original question-answering large language model (i.e., the difference between predicted answers is not less than a preset difference), perform at least one of the following operations to regenerate a new target question-answering large language model:

[0144] (1) Randomly adjust the position of element 1 in the pseudo-index selection matrix R;

[0145] (2) Randomly adjust the number of elements 1 in the pseudo-index selection matrix R.

[0146] Based on the same inventive concept, one embodiment of this application provides a question-answering system based on a data-driven regularization question-answering large language model. Figure 5 This is a schematic diagram of a question-answering system 100 based on a data-driven regularization-based large language model, as proposed in an embodiment of this application. Figure 5 As shown, the system includes:

[0147] The preprocessing module 101 is configured to obtain the device parameter values ​​of the target device of the target question-answering large language model to be deployed, the device parameter values ​​including at least one of the following: storage space size and computing speed; and determine the channels that need to be pruned in the pre-trained original question-answering large language model based on the device parameter values, the pre-trained original question-answering large language model being used to reason about questions from the client and obtain answers.

[0148] Information transfer module 102 is configured to update the model parameters of the pre-trained original question-answering large language model based on question-answering sample data using a loss function with a regularization loss term, wherein the regularization loss term is determined according to the parameter matrix of the channel that needs to be pruned.

[0149] The pruning module 103 is configured to perform channel pruning on the question-answering language model after the model parameters have been updated, according to the channels that need to be pruned, and obtain the target question-answering language model based on the question-answering language model after channel pruning.

[0150] The reasoning module 104 is configured to perform reasoning on questions from the target device using the target question-answering big language model to obtain answers.

[0151] As one embodiment of this application, the preprocessing module 101 is configured to determine the channels of the pre-trained original question-answering large language model that need pruning based on the device parameter values, specifically including:

[0152] Based on the device parameter values, determine the number of parameters of the target question-answering large language model;

[0153] The pruning ratio is determined based on the number of parameters and the original number of parameters of the pre-trained original question-answering large language model;

[0154] Based on the pruning ratio, determine the channels that need pruning in the pre-trained original question-answering large language model.

[0155] As one embodiment of this application, the preprocessing module 101 is configured to determine the channels of the pre-trained original question-answering large language model that need to be pruned according to the pruning ratio, specifically including:

[0156] Based on the pruning ratio p%, a pseudo-index selection matrix R is constructed. The pseudo-index selection matrix R is a diagonal matrix, and the elements of the pseudo-index selection matrix R are 1 for p% and 0 for 1-p%.

[0157] Based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering big language model, determine the regularization loss value corresponding to the regularization loss term;

[0158] The information transfer module 102 is configured to update the model parameters of the pre-trained original question-answering large language model using a loss function with a regularization loss term, specifically including:

[0159] For the first question in the question-and-answer sample data, the predicted answer is obtained through the pre-trained original question-and-answer large language model;

[0160] Based on the difference between the predicted answer and the first answer in the question-answering sample data, the original question-answering loss value of the pre-trained original question-answering large language model is determined;

[0161] The model parameters of the pre-trained original question-answering large language model are updated based on the original question-answering loss value and the regularization loss value corresponding to the regularization loss term.

[0162] As one embodiment of this application, the pruning module 103 is configured to perform channel pruning on the question-answering large language model after model parameter updates, according to the channels that need to be pruned, specifically including:

[0163] Based on the pseudo-index selection matrix R and the identity matrix I, construct the channel pruning matrix. S , S=IR ;

[0164] Based on the parameter matrix of the question-answering large language model after model parameter updates The parameter matrix of the question-answering large language model after channel pruning is obtained by combining the channel pruning matrix S. .

[0165] As one embodiment of this application, the preprocessing module 101 is configured to determine the regularization loss value corresponding to the regularization loss term based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering large language model, including:

[0166] Based on the pseudo-index selection matrix R and the attention mechanism module of the pre-trained original question-answering big language model, i The original parameter matrix of the layer , , , Determine the first i Attention regularization loss value of the layer;

[0167] Based on the pseudo-index selection matrix R and the feedforward neural network module of the pre-trained original question-answering big language model, i The original parameter matrix of the up projection in the layer and the original parameter matrix of the downward projection Determine the first i Regularized loss value of a layered feedforward neural network;

[0168] The regularization loss value of the remaining modules is determined based on the pseudo-index selection matrix R and the original parameter matrix of the remaining modules of the pre-trained original question-answering big language model.

[0169] As one embodiment of this application, the preprocessing module 101 is further configured to perform the following operations: obtain the question-answer sample dataset used by the original question-answering large language model obtained through pre-training; and randomly extract a subset of question-answer sample data from the question-answer sample dataset.

[0170] The information transfer module 102 is configured to update the model parameters of the pre-trained original question-and-answer large language model based on question-and-answer sample data using a loss function with a regularization loss term, specifically including:

[0171] Each question-and-answer sample data in the subset of question-and-answer sample data is input into the pre-trained original question-and-answer large language model, and the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term.

[0172] The pruning module 103 is configured to obtain the target question-answering large language model based on the question-answering large language model after channel pruning, specifically including:

[0173] Using the subset of question-and-answer sample data, the question-and-answer large language model after channel pruning is fine-tuned to obtain the target question-and-answer large language model.

[0174] In one embodiment of this application, the system further includes a callback module configured to perform the following steps:

[0175] For the test question-and-answer sample data, obtain the predicted answer generated by the target question-and-answer large language model and the predicted answer generated by the pre-trained original question-and-answer large language model;

[0176] If the difference between the predicted answer generated by the target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is greater than or equal to a preset difference, the channels that need to be pruned are continuously adjusted, and the process returns to the step: inputting question-answering sample data into the pre-trained original question-answering language model, and using a loss function with a regularization loss term to update the model parameters of the pre-trained original question-answering language model until the difference between the predicted answer generated by the final target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is less than the preset difference;

[0177] The reasoning module 104 is further configured to reason about questions from the target device using the final target question-answering language model to obtain answers.

[0178] In one embodiment of this application, the callback module is configured to continuously adjust the channel that needs pruning, specifically including:

[0179] The number of elements that are 1 in the pseudo-index selection matrix R is continuously adjusted, or the position of the elements that are 1 in the pseudo-index selection matrix R is continuously adjusted.

[0180] Based on the same inventive concept, one embodiment of this application provides an electronic device. Figure 6 This is a schematic diagram of an electronic device according to an embodiment of this application. Figure 6 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps of the question-answering method based on a data-driven regularization question-answering large language model as described in any of the above embodiments of this application.

[0181] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0182] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0183] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and components involved are not necessarily essential to this application.

[0184] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0186] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0188] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, this application is to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0189] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0190] The above provides a detailed description of the question-answering method and product based on a data-driven regular expression-based large language model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A question-answering method based on a data-driven regularization-based large language model, characterized in that, The method includes: Obtain the device parameter values ​​of the target device for the target question-answering language model to be deployed. The device parameter values ​​include at least one of the following: storage space size and computing speed. Based on the device parameter values, determine the channels that need pruning in the pre-trained original question-answering language model. The pre-trained original question-answering language model is used to reason about questions from the client and obtain answers. Based on question-and-answer sample data, the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term. The regularization loss term is determined based on the parameter matrix of the channels that need to be pruned. The loss function includes the regularization loss of each layer in the original question-and-answer large language model, as follows: ； in, The language modeling loss of the model is used to measure the difference between the model's predicted answer and the true answer; For the model number i The attention mechanism module of the layer uses regularized loss; For the model number i Regularization loss of the forward propagation network of the layer; This is the regularization loss for the remaining part of the model excluding the pruned portion; The number of layers in the model; Among them, the model number Attention mechanism module regularization loss of layer for: ； in R This represents the pseudo-index selection matrix, with a pruning ratio of p%. R Elements with p% on the diagonal are 1, and elements with 1-p% are 0; The symbol represents a vector norm applied to the matrix column by column. , , , The first i The query matrix, key matrix, value matrix, and output matrix in the layer; Model No. i Regularization loss of a layered FFN forward propagation network The expression is as follows: ； in, , The first i The upper projection matrix and lower projection matrix in the layer; Regularization loss of the remaining part of the model excluding the pruned portion. The expression is as follows: ； in, Represents the data feature matrix. Represents the position encoding matrix. Representing the Layer normalized vector, Represents the language modeling matrix; According to the channels that need pruning, channel pruning is performed on the question-answering language model after model parameter updates. Based on the channel-pruned question-answering language model, the target question-answering language model is obtained. Specifically, channel pruning of the question-answering language model after model parameter updates according to the channels that need pruning includes: constructing a channel pruning matrix S, S=IR, based on the pseudo-index selection matrix R and the identity matrix I; and based on the parameter matrix of the question-answering language model after model parameter updates... The parameter matrix of the question-answering large language model after channel pruning is obtained by combining the channel pruning matrix S. ; For questions from the target device, the answer is obtained by reasoning through the target question-answering big language model.

2. The question-answering method based on a data-driven regularization-based large language model according to claim 1, characterized in that, Based on the device parameter values, determine the channels that need pruning in the pre-trained original question-answering large language model, including: Based on the device parameter values, determine the number of parameters of the target question-answering large language model; The pruning ratio is determined based on the number of parameters and the original number of parameters of the pre-trained original question-answering large language model; Based on the pruning ratio, determine the channels that need pruning in the pre-trained original question-answering large language model.

3. The question-answering method based on a data-driven regularization-based large language model according to claim 2, characterized in that, Based on the pruning ratio, determine the channels in the pre-trained original question-answering large language model that need pruning, including: Based on the pruning ratio p%, a pseudo-index selection matrix R is constructed. The pseudo-index selection matrix R is a diagonal matrix, and the elements of the pseudo-index selection matrix R are 1 for p% and 0 for 1-p%. Based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering big language model, determine the regularization loss value corresponding to the regularization loss term; The model parameters of the pre-trained original question-answering large language model are updated using a loss function with a regularization loss term, including: For the first question in the question-and-answer sample data, the predicted answer is obtained through the pre-trained original question-and-answer large language model; Based on the difference between the predicted answer and the first answer in the question-answering sample data, the original question-answering loss value of the pre-trained original question-answering large language model is determined; The model parameters of the pre-trained original question-answering large language model are updated based on the original question-answering loss value and the regularization loss value corresponding to the regularization loss term.

4. The question-answering method based on a data-driven regularization-based large language model according to claim 3, characterized in that, Based on the pseudo-index selection matrix R and the original parameter matrix W of the pre-trained original question-answering large language model, the regularization loss value corresponding to the regularization loss term is determined, including: Based on the pseudo-index selection matrix R and the attention mechanism module of the pre-trained original question-answering big language model, i The original parameter matrix of the layer , , , Determine the first i Attention regularization loss value of the layer; Based on the pseudo-index selection matrix R and the feedforward neural network module of the pre-trained original question-answering big language model, i The original parameter matrix of the up projection in the layer and the original parameter matrix of the downward projection Determine the first i Regularized loss value of a layered feedforward neural network; The regularization loss value of the remaining modules is determined based on the pseudo-index selection matrix R and the original parameter matrix of the remaining modules of the pre-trained original question-answering big language model.

5. The question-answering method based on a data-driven regularization-based large language model according to claim 1, characterized in that, The method further includes: Obtain the question-answering sample dataset used by the original question-answering large language model obtained through pre-training; Randomly extract a subset of the question-and-answer sample dataset to obtain a subset of the question-and-answer sample data; Based on question-and-answer sample data, the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term, including: Each question-and-answer sample data in the subset of question-and-answer sample data is input into the pre-trained original question-and-answer large language model, and the model parameters of the pre-trained original question-and-answer large language model are updated using a loss function with a regularization loss term. Based on the question-answering large language model after channel pruning, the target question-answering large language model is obtained, including: Using the subset of question-and-answer sample data, the question-and-answer large language model after channel pruning is fine-tuned to obtain the target question-and-answer large language model.

6. The question-answering method based on a data-driven regularization-based large language model according to claim 3, characterized in that, The method further includes: For the test question-and-answer sample data, obtain the predicted answer generated by the target question-and-answer large language model and the predicted answer generated by the pre-trained original question-and-answer large language model; If the difference between the predicted answer generated by the target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is greater than or equal to a preset difference, the channels that need to be pruned are continuously adjusted, and the process returns to the step: inputting question-answering sample data into the pre-trained original question-answering language model, and using a loss function with a regularization loss term to update the model parameters of the pre-trained original question-answering language model until the difference between the predicted answer generated by the final target question-answering language model and the predicted answer generated by the pre-trained original question-answering language model is less than the preset difference; For questions from the target device, the answer is obtained by reasoning through the final target question-answering large language model.

7. The question-answering method based on a data-driven regularization-based large language model according to claim 6, characterized in that, Continuously adjust the channels that require pruning, including: The number of elements that are 1 in the pseudo-index selection matrix R is continuously adjusted, or the position of the elements that are 1 in the pseudo-index selection matrix R is continuously adjusted.

8. A question-answering system based on a data-driven regularization-based large language model, characterized in that, For performing the method as described in any one of claims 1-7, comprising: The preprocessing module is configured to obtain the device parameter values ​​of the target device for the target question-answering language model to be deployed, the device parameter values ​​including at least one of the following: storage space size and computing speed; based on the device parameter values, determine the channels that need to be pruned in the pre-trained original question-answering language model, the pre-trained original question-answering language model being used to reason about questions from the client and obtain answers; The information transfer module is configured to update the model parameters of the pre-trained original question-answering large language model based on question-answering sample data, using a loss function with a regularization loss term. The regularization loss term is determined according to the parameter matrix of the channels that need to be pruned. The loss function includes the regularization loss of each layer in the original question-answering large language model, as follows: ； in, The language modeling loss of the model is used to measure the difference between the model's predicted answer and the true answer; For the model number i The attention mechanism module of the layer uses regularized loss; For the model number i Regularization loss of the forward propagation network of the layer; This is the regularization loss for the remaining part of the model excluding the pruned portion; The number of layers in the model; Among them, the model number Attention mechanism module regularization loss of layer for: ； in R This represents the pseudo-index selection matrix, with a pruning ratio of p%. R Elements with p% on the diagonal are 1, and elements with 1-p% are 0; The symbol represents a vector norm applied to the matrix column by column. , , , The first i The query matrix, key matrix, value matrix, and output matrix in the layer; Model No. i Regularization loss of a layered FFN forward propagation network The expression is as follows: ； in, , The first i The upper projection matrix and lower projection matrix in the layer; Regularization loss of the remaining part of the model excluding the pruned portion. The expression is as follows: ； in, Represents the data feature matrix. Represents the position encoding matrix. Representing the Layer normalized vector, Represents the language modeling matrix; The pruning module is configured to perform channel pruning on the updated question-answering language model according to the channels that need pruning, and to obtain the target question-answering language model based on the channel-pruned question-answering language model. Specifically, performing channel pruning on the updated question-answering language model according to the channels that need pruning includes: constructing a channel pruning matrix S, S=IR, based on the pseudo-index selection matrix R and the identity matrix I; and pruning based on the parameter matrix of the updated question-answering language model. The parameter matrix of the question-answering large language model after channel pruning is obtained by combining the channel pruning matrix S. ; The reasoning module is configured to perform reasoning on questions from the target device using the target question-answering language model to obtain answers.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps in the method as described in any one of claims 1-7.

Images