Method and apparatus for question and answer model processing

Abstract

The application discloses a method and equipment for question and answer model processing, and relates to the technical field of artificial intelligence, and comprises the following steps: calculating a reward value change characteristic quantity by counting the activation conditions and corresponding reward values of each expert subnetwork, screening out invalid redundant experts which are activated but have no obvious change in the reward value in combination with a preset threshold, and pruning the experts, so that the technical problem of waste of computing resources existing in the pre-trained question and answer model in the related art is solved, the technical effect of effectively reducing the structural redundancy of the question and answer model, significantly improving the performance of the question and answer model, and reducing the waste of computing resources is achieved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to methods and devices for question-answering model processing. Background Technology

[0002] With the development of large-scale language models, question-answering models, and multimodal models, Mixture of Experts (MoE) models have become an important technical route for building ultra-large-scale models because they can significantly improve the scale of model parameters and representation capabilities while keeping computational costs under control.

[0003] In pre-trained question-answering models in related technologies, some expert subnetworks are activated, but their processing results contribute little or no to the output sequence. The presence of these experts increases the computational load during question-answering model inference without improving model performance, resulting in a waste of computational resources. Summary of the Invention

[0004] This application provides a method and apparatus for question answering model processing, so as to at least solve the problem of wasted computing resources in pre-trained question answering models in related technologies.

[0005] This application provides a method for question answering model processing, including: acquiring multiple question samples, inputting the multiple question samples into a pre-trained question answering model, and generating response sequences corresponding to the question samples; determining the reward value of the target response sequence based on the target question sample and the target response sequence corresponding to the target question sample; statistically analyzing whether each expert subnetwork in the pre-trained question answering model is activated during the generation of multiple response sequences and the reward value of the activated expert subnetwork, wherein the reward value of the target response sequence is associated with the activated expert subnetwork to determine the reward value of the activated expert subnetwork; based on the statistical results of each expert subnetwork, determining the change feature of the reward value of the activated expert subnetwork, judging whether the change feature is less than a preset threshold, and if the target change feature is less than the preset threshold, pruning the expert subnetwork corresponding to the target change feature to obtain a structurally adjusted question answering model.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described question-answering model processing methods.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described question-answering model processing methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described question-answering model processing methods.

[0009] This application solves the technical problem of wasted computational resources in pre-trained question-answering models by statistically analyzing the activation status and corresponding reward values ​​of each expert subnetwork, calculating the reward value change feature, and combining it with a preset threshold to filter out invalid and redundant experts that are activated but whose reward values ​​do not change significantly, and then pruning such experts. This achieves the technical effect of effectively reducing the structural redundancy of question-answering models, significantly improving the performance of question-answering models, and reducing the waste of computational resources. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating the question-answering model processing method provided in this application is shown; Figure 2 Another flowchart illustrating the question-answering model processing method provided in this application is shown; Figure 3 A schematic diagram of the composition structure of the question-answering model processing device provided in this application is shown; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0013] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] With the development of large-scale language models and multimodal models, Mixture of Experts (MoE) models have become an important technical approach for building ultra-large-scale models because they can significantly improve the scale of model parameters and representation capabilities while keeping computational costs under control. MoE models typically use gating networks to dynamically select several experts to participate in the computation for input samples, thereby achieving a balance between inference efficiency and model capacity.

[0016] The model training process typically includes a pre-training phase and a post-training phase. The post-training phase, especially Reinforcement Learning from Human Feedback (RLHF), has become a crucial step in improving model alignment, safety, and controllability. The optimization objective in this phase is no longer the traditional language modeling loss, but rather uses rewarding model output or human preference feedback as the core optimization signal.

[0017] However, with the continuous growth of model size and the number of experts, how to effectively manage experts in hybrid expert models in the post-training stage to ensure training efficiency, stability and alignment has become an urgent technical problem to be solved.

[0018] Most model pruning and structural optimization schemes applied in the post-training stage of related technologies are designed with the goal of fine-tuning specific downstream tasks. Their core assumptions include: the training objective is fixed and singular; the data distribution is relatively stable; and the optimization signal is clear and directly attributable. Under these assumptions, model pruning is mostly regarded as a model compression method before deployment or within a task, rather than a dynamic structural optimization mechanism that runs through the training process and is geared towards multi-task or policy learning.

[0019] In contrast, post-training across all tasks, especially RLHF, has the following significant differences: the training objective is implicitly defined by the reward model or human preferences; the reward signal is sparse, noisy, and dynamically changes with training; and the model needs to maintain generalization ability across multiple tasks and instruction distributions. Therefore, post-training pruning schemes in related technologies are inherently difficult to directly transfer to RLHF scenarios in terms of design assumptions and applicable scenarios.

[0020] In related technologies, the pruning results are highly dependent on the specific task distribution, resulting in a significant decrease in generalization ability and difficulty in transferring to other tasks; a single indicator-driven approach makes it difficult to fully reflect the state of all experts; furthermore, it cannot simultaneously meet the needs of multi-task or instruction generalization, making it unsuitable for multi-task post-training; in addition, it lacks a task importance differentiation mechanism, making it impossible to dynamically adjust the pruning strategy based on tasks or signals.

[0021] Model compression methods in related technologies are usually performed after the model reinforcement learning training is completed, or applied in offline scenarios. Compression is not performed during model training. In addition to the problem that model compression does not participate in the model policy learning process and cannot improve model training efficiency, it also lacks the ability to dynamically adjust the structure, cannot adapt to changes in reward distribution during training, and focuses on overall performance optimization, making it difficult to evaluate the contribution of individual experts and failing to design for reward models or human feedback signals.

[0022] The application environment architecture upon which the question-answering model processing method provided in this application depends includes: a data input layer, a model running layer, a statistical calculation layer, and a model tuning layer. The data input layer is used to acquire and preprocess multiple question samples, providing standardized data for subsequent question-answering model input. The model running layer is used to run the pre-trained question-answering model, supporting the execution of the response sequence generation step. The statistical calculation layer supports the execution of reward value calculation, expert subnetwork statistics, change feature calculation, and threshold judgment steps in the method. The model tuning layer supports the execution of the steps in the method to prune the expert subnetwork and obtain the structurally adjusted question-answering model.

[0023] The embodiments of this application provide a question-answering model processing method that can be applied to terminal devices, such as desktop computers, laptops, or servers. Figure 1 A flowchart illustrating the question-answering model processing method provided in this application is shown, as follows: Figure 1 As shown, the question-answering model processing method provided in this application includes: Step S101: Obtain multiple question samples, input the multiple question samples into the pre-trained question answering model respectively, and generate a response sequence corresponding to the question samples.

[0024] The question-answering model represents an AI model that receives user questions and outputs natural language answers, such as a large-scale dialogue model or a retrieval-based question-answering model. The question-answering model can be a hybrid expert model. Question samples include text, images, or a combination of both. The pre-trained question-answering model represents a question-answering model that has achieved a certain level of accuracy after pre-training. The pre-trained question-answering model possesses basic question-answering capabilities, but its structure contains some redundancy. The response sequence represents the textual responses obtained after inputting the question samples into the pre-trained question-answering model.

[0025] Specifically, in step S101, each question sample is input into the pre-trained question-answering model to obtain the response sequence generated by the question-answering model.

[0026] Step S102: Based on the target question sample and the target response sequence corresponding to the target question sample, determine the reward value of the target response sequence.

[0027] The reward value is a quantitative indicator used to characterize the quality of the evaluation response sequence. It can be calculated based on a reward model or human feedback to determine the reward value corresponding to the target response sequence.

[0028] Specifically, for each "target question sample - target response sequence" pair, the quality score of the target response sequence is calculated through a reward model or human feedback.

[0029] Step S103: Statistically determine whether each expert subnetwork in the pre-trained question-answering model is activated during the generation of multiple response sequences and the reward value of the activated expert subnetwork. Specifically, associate the reward value of the target response sequence with the activated expert subnetwork to determine the reward value of the activated expert subnetwork.

[0030] The pre-trained question-answering model can be a hybrid expert model, which includes at least one hybrid expert layer. Each hybrid expert layer comprises at least one gating network and multiple expert sub-networks. When generating response sequences based on the hybrid expert model, only some of the expert sub-networks within the hybrid expert layer are activated. The gating network is used to generate the routing probability distribution of the corresponding expert sub-network based on the input question sample or lexical features.

[0031] Specifically, when generating each response sequence, we statistically analyze which expert subnetworks are selected by the gating network to participate in the forward computation, and the reward value for generating the response sequence, establishing a correlation between the activation behavior of expert subnetworks and the response performance of the question-answering model. Each question sample, after being input into the pre-trained question-answering model, generates a response sequence with a corresponding reward value. During the generation of the response sequence, several expert subnetworks are activated, and the reward value is associated with all activated expert subnetworks, i.e., the reward value of the activated expert subnetwork.

[0032] Step S104: Based on the statistical results of each expert sub-network, determine the change feature of the reward value of the activated expert sub-network, and determine whether the change feature is less than a preset threshold. If the target change feature is less than the preset threshold, prune the expert sub-network corresponding to the target change feature to obtain the question-answering model with adjusted structure.

[0033] The Reward Sensitivity is a statistical measure representing the contribution of the expert subnetwork to the reward value after multiple activations. Based on the Reward Sensitivity, it can be used to determine whether an expert subnetwork is redundant or should be retained. The preset threshold is a pre-defined critical value used to determine whether an expert subnetwork is redundant. Pruning represents freezing and bypassing the expert subnetwork corresponding to the target Reward Sensitivity. The restructured question-answering model represents the optimized question-answering model that, after pruning, has a more streamlined structure, faster inference, and higher stability.

[0034] Specifically, based on whether each expert subnetwork is activated and the reward value obtained, the contribution of the activated expert subnetwork to the reward value is determined. If the contribution of the activated expert subnetwork to the reward value is lower than a preset threshold, it indicates that the contribution of the activated expert subnetwork to the reward value is low, and expert subnetworks whose contribution to the reward value is lower than the preset threshold can be pruned. The preset threshold can be determined by determining the mean and standard deviation of the change features of all expert subnetworks in the hybrid expert layer. The preset threshold can be configured as the mean plus 0.5 times the standard deviation. In this way, by statistically analyzing the change features of the reward value after multiple activations of expert subnetworks, redundant expert subnetworks that are activated but whose contribution to the reward value does not change significantly can be automatically identified and pruned, which can effectively reduce the structural redundancy of the question answering model. At the same time, after removing redundant expert subnetworks, the question answering model is more lightweight, which can reduce inference latency, significantly improve the performance of the question answering model, and reduce the waste of computing resources. In addition, by using change features instead of call frequency for expert pruning, the expert management strategy can be highly consistent with the optimization goal of reinforcement learning training based on human feedback.

[0035] By pruning redundant experts, the restructured question-answering model has reduced parameter size and computational cost, requiring less processor resources during training and lower memory overhead during inference. Simultaneously, by removing useless experts, the model does not need to allocate computational resources to redundant experts, resulting in faster decision-making by the gating network during response generation, more focused gradient updates on core parameters during training, and a shorter iteration cycle. Furthermore, by only pruning experts that do not contribute to the reward value while retaining core experts, the quality of the model's generated responses does not decrease. Less resource consumption translates to lower server or computing power costs, especially for large-scale question-answering models such as intelligent customer service and chatbots, which can significantly reduce costs for long-term deployment. Additionally, the restructured question-answering model has faster inference speed and more stable response quality, enabling faster responses to user questions and improving user experience.

[0036] In some optional implementations, the pre-trained question-answering model includes a multi-layer neural network, wherein at least one layer is a hybrid expert layer. The hybrid expert layer includes at least one gating network and multiple expert sub-networks. Multiple question samples are input into the pre-trained question-answering model to generate response sequences corresponding to the question samples. This includes: segmenting the question samples to obtain text or words; converting the text or words into tokens; generating token embedding features based on the tokens; inputting the token embedding features into the pre-trained question-answering model; allocating the token embedding features to multiple target expert sub-networks based on the gating network; performing a first-level linear transformation on the allocated target token embedding features based on the target expert sub-networks; mapping the target token embedding features to a preset dimension based on the first-level linear transformation to obtain an intermediate feature vector; performing a non-linear transformation on the intermediate feature vector based on an activation function to obtain a non-linear transformation result; performing a second-level linear transformation on the non-linear transformation result; performing feature filtering and integration on the non-linear transformation result based on the second-level linear transformation to obtain the processed result of the target token embedding features; and aggregating the processed results based on the gating network to generate a response sequence.

[0037] The pre-trained question-answering model comprises a multi-layered neural network, with each layer responsible for processing input features from different dimensions. These layers work together to achieve the process from question input to response generation. A gating network receives lexical embedding features and, based on the semantic characteristics and task suitability of the features, assigns them to target experts in multiple expert sub-networks. It also aggregates the processing results from each target expert sub-network. Lexical units represent the basic data units that the question-answering model can directly recognize and process, converted from text or words. Lexical embedding features represent the vector representation of lexical units, converting discrete lexical units into continuous vector features and capturing their semantic information.

[0038] The first-layer linear transformation converts the input lexical embedding features to a preset feature dimension via linear mapping. This preset feature dimension matches the structure and task requirements of the expert sub-network and is the target dimension of the linear transformation, ensuring that the transformed intermediate features can be effectively processed by the expert sub-network. The intermediate feature vector represents the intermediate representation of the lexical embedding features after dimension alignment. It preserves the semantic information of the lexical embedding features while adapting to the processing dimension of the expert sub-network, serving as an intermediate carrier connecting the linear and nonlinear transformations. The activation function represents the function that implements the nonlinear transformation. The nonlinear transformation result represents the feature representation incorporating nonlinear semantic associations. Compared to the intermediate feature vector, it better reflects the deep semantic information of the lexical units and enhances the expressive power of the features.

[0039] The second linear transformation is used to perform feature selection, integration, and dimensionality adjustment on the nonlinear transformation results, filtering redundant features and retaining those useful for response generation. Feature selection is used to characterize and retain the core features relevant to the question-answering task in the nonlinear transformation results and eliminate redundant and invalid features. Feature integration is used to characterize the fusion of the selected features to form a unified feature representation with strong expressive power, ensuring that the processing results can be effectively aggregated by the gating network.

[0040] The processing results of target lexical embedding features are used to characterize the core features of the expert sub-network's deep analysis of lexical semantics. Aggregation is used to characterize the aggregation of the processing results of multiple expert sub-networks into a unified feature representation through methods such as weighted summation or feature fusion.

[0041] Specifically, the input single question sample is split into single characters or multiple words; the split characters / words are then mapped into tokens that the model can process; then, through embedding operations, the tokens are converted into fixed-dimensional token embedding features. The generated token embedding features are input into a pre-trained question-answering model; based on the gating network in the hybrid expert layer within the model, the input token embedding features are analyzed, and according to the semantic characteristics and adaptability of the features, they are assigned to a portion of the expert subnetworks, namely the target expert subnetwork.

[0042] The target expert subnetwork performs a first-level linear transformation on the input features. This transformation converts the dimension of the target word embedding features to a pre-defined dimension, resulting in a dimension-aligned intermediate feature vector. This intermediate feature vector is then input into an activation function. Through the non-linear mapping of the activation function, the intermediate feature vector is processed to obtain a non-linear transformation result incorporating non-linear semantic associations. A second-level linear transformation is then performed on this non-linear transformation result. During this second-level linear transformation, feature selection and integration are simultaneously performed on the non-linear transformation result, yielding the processed target word embedding features.

[0043] Based on the gated network in the hybrid expert layer, the processing results output by all target expert sub-networks are received. Through aggregation methods such as weighted summation and feature fusion, the processing results of multiple experts are integrated into a unified feature representation. Based on the aggregated features, the response sequence corresponding to the input question sample is generated through the subsequent structure of the model, such as the decoding layer.

[0044] In this way, the hierarchical processing flow, which uses the first-layer linear transformation for dimension alignment, the activation function for non-linear transformation, and the second-layer linear transformation for feature selection and integration, not only solves the problem of mismatch between the dimensionality of the word embedding features and the features processed by the expert sub-network, but also captures complex semantic associations and filters redundant features, ensuring that each expert sub-network can efficiently and accurately process the assigned features and output high-quality processing results.

[0045] In some optional implementations, the variation characteristics include volatility variation, which characterizes the difference in reward values ​​when the activated expert subnetworks participate in the decision-making process of generating response sequences for different problem samples. Based on the statistical results of each expert subnetwork, the variation characteristics of the reward values ​​of the activated expert subnetworks are determined, including: determining the mean of the reward values ​​of the activated expert subnetworks based on the statistical results; and determining the variance of the reward values ​​of the activated expert subnetworks based on the reward values ​​and the mean of the activated expert subnetworks, thereby obtaining the volatility variation of the reward values ​​of the activated expert subnetworks.

[0046] In this embodiment, the reward variance can be calculated as follows: ; in, For characterization experts The amount of fluctuation change, For characterization experts Participate in the decision-making of the question-answering model; For characterization experts Rewards earned from participating in the question-answering model's decision-making process; For characterization experts The variance of the reward value obtained by participating in the question-and-answer model decision-making.

[0047] It can be done The calculation results measure the experts The difference in reward values ​​obtained by the question-answering model when participating in response sequence decision-making. During the reinforcement learning training phase based on human feedback after pre-training, the reward values ​​themselves are noisy and unstable. However, the variance of the reward values ​​does not depend on their absolute size and can measure their volatility and uncertainty. If the variance of the reward value distribution of the question samples participated in by the activated expert subnetwork is large, it indicates that the expert subnetwork determines the direction of the reward values; therefore, this expert subnetwork is relatively important and needs to be retained. If the variance of the reward value distribution of the question samples participated in by the activated expert subnetwork is small, it indicates that the reward values ​​hardly change with the expert subnetwork, meaning that the expert subnetwork has a small impact on the reward values; therefore, this expert subnetwork can be pruned.

[0048] In related technologies, it is difficult to accurately assess the true contribution of a single expert to reward value optimization during the post-training stage of hybrid expert models. This application, by defining the variable feature as a fluctuating variable, can accurately assess the differences in reward values ​​of expert subnetworks participating in decision-making across multiple problem samples. This accurately reflects the true contribution of expert subnetworks to reward value optimization and provides an accurate basis for pruning expert subnetworks.

[0049] In some optional implementations, the variation features include gradient sensitivity variation, which characterizes the difference in reward values ​​caused by parameter changes in the activated expert subnetwork. Based on the statistical results of each expert subnetwork, the variation features of the reward values ​​of the activated expert subnetwork are determined, including: determining the set of problem samples processed by the activated expert subnetwork; determining the gradient vector of the reward value of the activated expert subnetwork with respect to the parameters in the activated expert subnetwork; determining the norm of the gradient vector; and taking the expected value of the norm corresponding to the problem samples in the problem sample set to obtain the gradient sensitivity variation of the reward values ​​of the activated expert subnetwork.

[0050] In this implementation, based on statistical results, all problem samples processed by the currently activated expert subnetwork can be filtered to obtain a problem sample set. For each problem sample in the problem sample set, the gradient vector of all parameters within the currently activated expert subnetwork is calculated. Then, the norm of the gradient vector is calculated, and the expected value of the norm of all problem samples in the problem sample set is taken. This expected value is the gradient sensitivity change of the reward value of the activated expert subnetwork. Based on the gradient sensitivity change, the extent to which changes in the parameters of the activated expert subnetwork affect the reward value can be measured.

[0051] Specifically, the gradient sensitivity change of the reward value of the activated expert subnetwork can be calculated using the following formula: ; in, Used to characterize the gradient sensitivity change of the reward value of the activated expert subnetwork. For characterization experts The parameters, The gradient vector of the reward value of the activated expert subnetwork with respect to the parameters in the activated expert subnetwork is used to characterize the reward value of the activated expert subnetwork. The norm used to characterize the gradient vector measures how sensitive the reward value is to the parameters in the activated expert subnetwork. Used to characterize a set of problem samples Taking the overall expectation can eliminate noise from single-problem samples.

[0052] Thus, a larger change in gradient sensitivity indicates a higher sensitivity of the parameters in the expert subnetwork to reward optimization, while a smaller change indicates a lower impact of the parameters on the reward. The computational complexity of gradient sensitivity change is higher than that of volatility change.

[0053] In related technologies, the value of expert subnetworks is measured only by their performance fluctuations in handling different problems, ignoring the impact of changes in the expert subnetwork's own parameters on the reward value. This application's embodiments, by determining the gradient vector, norm, and expectation, can accurately quantify the degree of influence of parameter changes in the expert subnetwork on the reward value, enabling a more comprehensive evaluation of the expert subnetwork and further improving the accuracy of expert pruning and the accuracy of question-answering model responses.

[0054] In some optional implementations, the change characteristics include a comprehensive change, which is obtained by weighted summation based on volatility change and gradient sensitivity change. Based on the statistical results of each expert subnetwork, the change characteristics of the reward value of the activated expert subnetwork are determined, including: determining the volatility change and gradient sensitivity change of the reward value of the activated expert subnetwork; determining the first normalized result of the volatility change and the second normalized result of the gradient sensitivity change; and obtaining the comprehensive change of the reward value of the activated expert subnetwork by weighted summation based on the first normalized result and the second normalized result.

[0055] In this embodiment, the volatility change is calculated using the aforementioned method, and the gradient sensitivity change is calculated using the aforementioned method. The volatility change is then normalized layer by layer to obtain a first normalization result. The gradient sensitivity change is then normalized layer by layer to obtain a second normalization result. A first weight for the first normalization result and a second weight for the second normalization result are pre-configured. The sum of the first normalization result multiplied by the first weight and the second normalization result multiplied by the second weight is the overall change in the reward value of the activated expert subnetwork.

[0056] Specifically, the overall change in reward value of the activated expert subnetwork (Reward Sensitivity) can be calculated using the following formula: ; in, Used to characterize the overall change in the reward value of the activated expert subnetwork. Used to characterize the first normalization result, Used to characterize the second normalization result, Used to characterize the first weight, Used to characterize the second weight. It can be configured. Greater than To adapt to engineering implementation, variance calculation is more stable and easier to implement. The impact of activated expert subnetworks on reward values ​​can be evaluated by comprehensively assessing changes. Configuration is possible. , The value can be configured as follows: , The value can be configured as follows: .

[0057] In related technologies, evaluating change features using only a single indicator, such as volatility change, fails to measure the stability of the expert's own parameters; using only gradient sensitivity change fails to measure the stability of the expert's performance in handling different problems. Single-indicator evaluation leads to biased expert assessments, resulting in misjudgments or invalid retention. In this embodiment, the comprehensive change measure integrates both volatility change (assessing performance stability) and gradient sensitivity change (assessing parameter stability). Determining the change feature based on the comprehensive change measure solves the problem of bias in single-indicator evaluation, allowing for a more comprehensive and objective evaluation of the expert subnetwork.

[0058] In some optional implementations, the selection of specific statistical parameters for the changing feature quantity can be determined according to the actual application scenario. For example, in simple scenarios, the changing feature quantity can be configured as a fluctuating change quantity to improve computational efficiency; in complex scenarios, the changing feature quantity can be configured as a comprehensive change quantity, taking into account both fluctuating and gradient-sensitive change quantities to improve statistical stability; when parameter stability is a concern, the changing feature quantity can be configured as a gradient-sensitive change quantity. This allows for scenario adaptation and on-demand selection, improving the versatility and flexibility of the solution.

[0059] Specifically, the change characteristic quantity can include only one statistical parameter, such as the change characteristic quantity can be determined as the volatility change quantity, the gradient sensitivity change quantity, or the combined change quantity; it can also include two statistical parameters, such as the change characteristic quantity can be determined as the volatility change quantity and the gradient sensitivity change quantity, or as the volatility change quantity and the combined change quantity, or as the gradient sensitivity change quantity and the combined change quantity; it can also include the volatility change quantity, the gradient sensitivity change quantity, and the combined change quantity.

[0060] In some optional implementations, the aforementioned question-answering model processing method further includes: based on a preset detection time interval, statistically analyzing whether each expert subnetwork in the pre-trained question-answering model is activated and the reward value obtained during the generation of multiple response sequences.

[0061] In this embodiment, the preset detection duration is used to characterize a pre-defined fixed statistical period, which can be determined based on the deployment scenario and task requirements of the question-answering model. The activation of the statistical expert subnetwork and the reward value for generating the response sequence can be repeatedly evaluated based on the time interval of the preset detection duration.

[0062] During the long-term operation of a question-answering model, the types of question samples it processes may change, which may cause dynamic changes in the activation frequency and reward value of the expert sub-network in the hybrid expert layer. By periodically collecting statistics, we can regularly obtain the dynamic changes in the activation status and reward value of the expert sub-network, providing the latest data support for subsequent updates to the changing feature quantities and pruning of the expert sub-network. This ensures that the question-answering model structure can dynamically adapt to task changes and maintain a streamlined and efficient state in the long term. At the same time, periodic statistics can also balance resource consumption and optimization effect.

[0063] In some optional implementations, the aforementioned question-answering model processing method further includes: acquiring routing distribution data of each expert subnetwork in the pre-trained question-answering model during the generation of multiple response sequences; determining the routing distribution change rate of each expert subnetwork based on the routing distribution data; determining whether there is a routing structure change in each expert subnetwork based on the comparison result of the average routing distribution change rate of each expert subnetwork and the routing change rate threshold within adjacent preset detection durations; wherein, if the average routing distribution change rate is less than or equal to the routing change rate threshold, it is determined that there is no routing structure change; if the average routing distribution change rate is greater than the routing change rate threshold, it is determined that there is a routing structure change; if it is determined that the first expert subnetwork has a routing structure change, and the change feature quantity of the first expert subnetwork is less than the preset threshold, the first expert subnetwork is pruned; if it is determined that the second expert subnetwork has a routing structure change, and the change feature quantity of the second expert subnetwork is greater than or equal to the preset threshold, the second expert subnetwork is retained; if it is determined that the third expert subnetwork has no routing structure change, and the change feature quantity of the third expert subnetwork is less than the preset threshold, the third expert subnetwork is pruned.

[0064] In this embodiment, the routing change rate threshold can be used. The configuration is set to 0.2. For the expert subnetwork in the pre-trained question-answering model, the rate of change in route distribution is statistically analyzed, that is, the change in the average route allocation probability ratio of the expert subnetwork is compared between two adjacent detection durations.

[0065] For example, the hybrid expert layer of the question-answering model includes three expert sub-networks: expert sub-network 1, expert sub-network 2, and expert sub-network 3. During the first detection period, the average route distribution of the hybrid expert layer is: expert sub-network 1 30%, expert sub-network 2 20%, and expert sub-network 3 50%. During the second detection period, the average route distribution is: expert sub-network 1 10%, expert sub-network 2 60%, and expert sub-network 3 30%. The rate of change in route distribution is (20% + 20% + 40%) ÷ 3 ≈ 0.27. Since 0.27 is greater than the route change rate threshold, the route result is determined to have changed.

[0066] In scenarios involving changes in routing structure, some expert subnetworks may become redundant and inefficient due to adjustments in routing allocation logic. If not pruned in time, these subnetworks will continuously consume the model's storage and computing resources. This application's embodiments, through statistical verification and redundancy pruning of expert subnetworks, can promptly adjust the structure of expert subnetworks when the question-answering model's routing structure changes due to task scenario changes. Redundant experts that cannot adapt to the new routing allocation logic are eliminated, allowing the model structure to dynamically adapt to the operational state after routing changes, thus improving the model's long-term adaptability to task and feature changes.

[0067] In some optional implementations, if the target change feature quantity is less than a preset threshold, the expert sub-network corresponding to the target change feature quantity is pruned, including: sorting the activated expert sub-networks based on the change feature quantity from high to low, and determining the expert sub-networks in the top preset percentage of the sorting results as retained expert sub-networks; determining the expert sub-networks in the sorting results other than the retained expert sub-networks as expert sub-networks to be adjusted; if there are expert sub-networks to be pruned among the expert sub-networks to be adjusted whose change feature quantity is less than the preset threshold for multiple consecutive preset detection durations, the expert sub-networks to be pruned are pruned.

[0068] In this embodiment, the preset percentage can be configured to be between 20% and 30%. For example, the preset percentage can be configured to 25%. Based on the order of the changing feature values ​​from high to low, the activated expert subnetworks are sorted, and the top 25% of the expert subnetworks in the sorting results are determined as retained expert subnetworks. The retained expert subnetworks are the core experts and will not be pruned. The remaining expert subnetworks are expert subnetworks to be adjusted, and further verification is needed to determine whether the expert subnetworks to be adjusted are redundant.

[0069] In the expert subnetworks to be adjusted, if the statistically obtained change feature quantity of one of the expert subnetworks is less than a preset threshold in each of several consecutive preset detection durations, this expert subnetwork can be identified as the expert subnetwork to be pruned, and a pruning operation can be performed on the expert subnetwork to be pruned. After simulating the trajectory (rollout), it can be periodically determined whether the expert subnetwork needs to be pruned.

[0070] Specifically, it can be determined whether a network is a subnetwork of experts to be pruned in the following way: ; in, This is used to characterize a preset threshold. If an expert subnetwork does not belong to the retained expert subnetworks and satisfies the above condition for multiple consecutive preset detection durations, then this expert subnetwork can be determined as an expert subnetwork to be pruned and can be pruned.

[0071] In this way, even if experts are called infrequently, as long as they are highly sensitive to reward signals, they can still be prioritized for retention. This can identify experts who contribute significantly to the reward signals and prevent core experts from being pruned. At the same time, strengthening the influence of the retained expert subnetwork in the RLHF stage can accelerate the question-answering model's learning of human preferences. Furthermore, compared to a one-time expert pruning rule, this application uses a combination of ranking and preset percentages to determine the priority retention of the retained expert subnetwork. The expert subnetwork to be adjusted is further validated to determine the expert subnetwork to be pruned. This can avoid accidental pruning of core experts and accurately focus on redundant experts, ensuring reasonable pruning.

[0072] In some optional implementations, pruning the expert subnetwork corresponding to the target variable feature quantity includes: adjusting the probability of assigning lexical embedding features to the expert subnetwork corresponding to the target variable feature quantity to zero based on a gating network, wherein the problem sample is segmented into words to obtain characters or words, the characters or words are converted into lexical units, and lexical embedding features are generated based on the lexical units; stopping the updating of parameters in the expert subnetwork corresponding to the target variable feature quantity; and retaining the structure of the expert subnetwork corresponding to the target variable feature quantity.

[0073] In this embodiment, during the RLHF stage of the question-answering model, the allocation logic of the gating network in the hybrid expert layer of the question-answering model is modified. The probability of the gating network assigning the word embedding feature to the expert sub-network corresponding to the target variable feature is adjusted to zero. In this way, the expert sub-network corresponding to the target variable feature no longer participates in the feature processing and response generation of the question-answering model, reducing the computational load and resource consumption of the question-answering model.

[0074] Simultaneously, for the expert subnetwork corresponding to the target changing feature, all update operations on its internal model parameters are stopped. Regardless of subsequent model training and iteration, the parameters of this expert subnetwork remain in their current state and do not change further. During model training and iteration, parameter updates require significant computational resources; stopping parameter updates for the experts to be pruned reduces this redundant computation.

[0075] Furthermore, the expert subnetworks that are pruned are not deleted; all their structures and current parameters are fully preserved, allowing them to participate in forward computation. This enables rollback when necessary. Specifically, if anomalies occur during the question-answering model training process, such as a sudden surge in the loss function, numerical instability, or divergence of model parameters, the system restores the model state, optimizer state, or training environment to a previously stable, saved checkpoint to prevent error propagation and ensure training stability.

[0076] In this way, compared to directly physically deleting the expert subnetwork corresponding to the target change features, soft pruning avoids assigning tasks to the expert subnetwork corresponding to the target change features, while preserving its structure and stopping its parameter updates. This approach can optimize the expert subnetwork structure and ensure the stability of the question answering model training process. In addition, preserving the structure of the expert subnetwork corresponding to the target change features also facilitates subsequent corrections.

[0077] In some optional implementations, the aforementioned question-answering model processing method further includes: after obtaining the structurally adjusted question-answering model, stopping the acquisition of statistical results within a preset cooling time; after the preset cooling time, resuming the acquisition of statistical results; and if the target change feature quantity is less than a preset threshold, pruning the expert subnetwork corresponding to the target change feature quantity.

[0078] In this embodiment, after the question-and-answer model structure is adjusted, a cooling time T is preset. cool During the cooldown period, expert pruning is paused to prevent repeated execution within a short time. After the cooldown period, the question-answering model training process resumes. The system statistically analyzes whether each expert subnetwork in the restructured question-answering model is activated during the generation of multi-state response sequences and the reward values ​​of the activated expert subnetworks. Based on the statistical results of each expert subnetwork, the system determines the change feature of the reward values ​​of the activated expert subnetworks in the restructured question-answering model. It then checks whether the change feature is less than a preset threshold. If the target change feature is less than the preset threshold, the expert subnetwork corresponding to the target change feature is pruned, and the structure of the pre-trained question-answering model is adjusted again. The cooldown period can be configured to be five to ten RLHF iteration cycles. After expert subnetwork pruning, methods related to loss function design can be used for stability control to mitigate the problems of policy distribution abrupt changes and reward instability introduced by pruning, thereby ensuring the stability and convergence of the RLHF training process.

[0079] In this way, by setting a preset cooling-off period, both the simplification of the question-answering model and its performance stability can be achieved. This allows the question-answering model sufficient time to adapt to the new load distribution and operating state after structural adjustments, avoiding the collection of distorted statistical data when the question-answering model is not stable. At the same time, it reduces the interference of structural adjustments on the operation of the question-answering model, avoids training oscillations caused by frequent structural adjustments, enhances the stability and controllability of the training process, and keeps the question-answering model in a stable state in the long term, ensuring stable response quality and inference efficiency.

[0080] In some optional implementations, the aforementioned question-answering model processing method further includes: if the target change feature quantity is less than a preset threshold, obtaining the initial weights of the expert sub-network corresponding to the target change feature quantity; adjusting the weights of the expert sub-network corresponding to the target change feature quantity based on the weight reduction step size until the weights of the expert sub-network corresponding to the target change feature quantity are zero, thereby obtaining the question-answering model with adjusted structure; wherein, the weight reduction step size is determined based on the ratio of the initial weights to a first number of preset detection times.

[0081] In this embodiment, the initial weight is used to characterize the gating routing allocation weight of the expert subnetwork corresponding to the target change feature quantity. It is the initial probability of selecting the expert subnetwork corresponding to the target change feature quantity when the gating network allocates lexical embedding features. The first quantity can be configured as a preset integer, such as 3, 57, etc., and can be adjusted based on model requirements.

[0082] Specifically, following a predetermined weight reduction step size, after each preset detection duration, the weights of the expert subnetwork corresponding to the target changed feature quantity are adjusted by decay: after each detection duration, the current weight of the expert subnetwork corresponding to the target changed feature quantity is subtracted from the weight reduction step size to obtain the new weight at the end of the cycle; the above decay operation is repeated, and after each adjustment, the activation status of the expert subnetwork corresponding to the target changed feature quantity during the current detection duration is simultaneously counted to ensure a smooth transition in model routing allocation during weight decay; the decay operation is continuously executed until the weight of the expert subnetwork corresponding to the target changed feature quantity drops to zero. At this point, the expert subnetwork is no longer selected by the gating network and no longer participates in any feature processing tasks, thus completing the weight adjustment; after the weight adjustment is completed, the network structure and all parameters of the target expert subnetwork are retained, and it is only deactivated by setting the weight to zero, finally obtaining the question answering model with adjusted structure.

[0083] In this way, by adopting a gradual weight decay, the weights of the expert subnetworks corresponding to the target change features are gradually reduced to zero, and the routing allocation probability of the gated network is smoothly transitioned. There will be no jump in the output distribution due to a sudden drop in weights, which ensures the stability of the quality of the responses generated by the question-answering model and avoids performance fluctuations.

[0084] Figure 2 Another flowchart illustrating the question-answering model processing method provided in this application is shown, such as... Figure 2 As shown, another process of the question-answering model processing method provided in this application includes: Step S201: In the reinforcement learning training based on human feedback of the question-answering model with a hybrid expert layer, record the reward value corresponding to each expert sub-network in the hybrid expert layer.

[0085] Step S202: Based on the recorded results, periodically calculate the change feature quantity of each expert subnetwork.

[0086] Step S203: Based on the calculation results of the changing feature quantities, periodically trigger the pruning of the expert subnetwork.

[0087] Step S204: In each period, retain the expert subnetwork with high variable feature quantity and prune the expert subnetwork with low variable feature quantity.

[0088] In step S205, after triggering the pruning operation, a cooling-off period is entered, and stability control is performed on the question-answering model.

[0089] Among these, the variable features can be defined as rewards related to text, image-text alignment, or visual understanding, in order to evaluate the contribution of the expert subnetwork to the rewards.

[0090] In this way, during the RLHF stage, the optimization objective of the question-answering model is transformed from language modeling loss to a reward function or reward model output. Based on the pruning of expert subnetworks using varying feature quantities, Layer-wise Adaptive Expert Pruning (LAEP) is upgraded from a computationally driven approach in the pre-training stage to an alignment signal-driven expert management mechanism. This can accurately reflect the true contribution of expert subnetworks to reward optimization, and can be applied to various hybrid expert model structures and different forms of reinforcement learning post-training processes, exhibiting good versatility and scalability.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0092] Embodiments of this application also provide an apparatus for question-answering model processing. Figure 3 A schematic diagram of the structural composition of the question-answering model processing device provided in this application is shown, such as... Figure 3 As shown, the apparatus for question-answering model processing includes: The acquisition module 301 is used to acquire multiple question samples, input the multiple question samples into the pre-trained question answering model, and generate a response sequence corresponding to the question samples.

[0093] The calculation module 302 is used to determine the reward value of the target response sequence based on the target problem sample and the target response sequence corresponding to the target problem sample.

[0094] The statistics module 303 is used to statistically analyze whether each expert subnetwork in the pre-trained question-answering model is activated during the generation of multiple response sequences and the reward value of the activated expert subnetwork. Specifically, the reward value of the target response sequence is associated with the activated expert subnetwork to determine the reward value of the activated expert subnetwork.

[0095] The adjustment module 304 is used to determine the change feature of the reward value of the activated expert sub-network based on the statistical results of each expert sub-network, and to determine whether the change feature is less than a preset threshold. If the target change feature is less than the preset threshold, the expert sub-network corresponding to the target change feature is pruned to obtain the question-answering model after structural adjustment.

[0096] In some optional implementations, the pre-trained question-answering model includes a multi-layer neural network, wherein at least one layer of the neural network is a hybrid expert layer, which includes at least one gating network and multiple expert sub-networks. The acquisition module 301 includes: a first unit for segmenting the question sample to obtain text or words, converting the text or words into tokens, and generating token embedding features based on the tokens; inputting the token embedding features into the pre-trained question-answering model, and allocating the token embedding features to multiple target expert sub-networks based on the gating network; performing a first-layer linear transformation on the allocated target token embedding features based on the target expert sub-networks, mapping the target token embedding features to a preset dimension based on the first-layer linear transformation to obtain an intermediate feature vector; performing a non-linear transformation on the intermediate feature vector based on an activation function to obtain a non-linear transformation result; performing a second-layer linear transformation on the non-linear transformation result, and performing feature filtering and integration on the non-linear transformation result based on the second-layer linear transformation to obtain the processing result of the target token embedding features; and aggregating the processing results based on the gating network to generate a response sequence.

[0097] In some optional implementations, the variable characteristic includes volatility variation, which characterizes the difference in reward values ​​when the activated expert subnetwork participates in the decision-making of different problem samples to generate response sequences; the adjustment module 304 includes: an adjustment module first unit, used to determine the mean of the reward values ​​of the activated expert subnetwork based on statistical results; and to determine the variance of the reward values ​​of the activated expert subnetwork based on the reward values ​​and the mean of the activated expert subnetwork, thereby obtaining the volatility variation of the reward values ​​of the activated expert subnetwork.

[0098] In some optional implementations, the change feature includes gradient sensitivity change, which characterizes the difference in reward value caused by parameter changes in the activated expert subnetwork; the adjustment module 304 further includes: a second adjustment module unit, used to determine the set of problem samples processed by the activated expert subnetwork; determine the gradient vector of the reward value of the activated expert subnetwork with respect to the parameters in the activated expert subnetwork; determine the norm of the gradient vector; and take the expected value of the norm corresponding to the problem samples in the problem sample set to obtain the gradient sensitivity change of the reward value of the activated expert subnetwork.

[0099] In some optional implementations, the change feature includes a comprehensive change, which is obtained by weighted summation based on the volatility change and the gradient sensitivity change. The adjustment module 304 further includes: a third adjustment module unit, used to determine the volatility change and the gradient sensitivity change of the activated expert subnetwork reward value; calculate a first normalized result of determining the volatility change and a second normalized result of determining the gradient sensitivity change; and obtain the comprehensive change of the activated expert subnetwork reward value by weighted summation based on the first normalized result and the second normalized result.

[0100] In some optional implementations, the aforementioned question-answering model processing apparatus further includes: a detection module, used to statistically analyze, based on a preset detection time interval, whether each expert subnetwork in the pre-trained question-answering model is activated and the reward value obtained during the generation of multiple response sequences.

[0101] In some optional implementations, the adjustment module 304 further includes: a fourth adjustment module unit, used to sort the activated expert subnetworks based on the order of the changing feature quantities from high to low, and determine the expert subnetworks in the sorting results that are the top preset percentage as retained expert subnetworks; to determine the expert subnetworks in the sorting results other than the retained expert subnetworks as expert subnetworks to be adjusted; and to prune the expert subnetworks to be pruned if there are multiple consecutive preset detection durations in the expert subnetworks to be adjusted where the changing feature quantities are all less than a preset threshold.

[0102] In some optional implementations, the adjustment module 304 further includes: a fifth adjustment module unit, used to acquire routing distribution data of each expert subnetwork in the pre-trained question-answering model during the generation of multiple response sequences; based on the routing distribution data, determine the routing distribution change rate of each expert subnetwork; within adjacent preset detection durations, based on the comparison result of the average routing distribution change rate of each expert subnetwork with the routing change rate threshold, determine whether there is a routing structure change in each expert subnetwork; wherein, if the average routing distribution change rate is less than or equal to the routing change rate threshold, it is determined that there is no routing structure change; if the average routing distribution change rate is greater than the routing change rate threshold, it is determined that there is a routing structure change; if it is determined that the first expert subnetwork has a routing structure change, and the change feature quantity of the first expert subnetwork is less than the preset threshold, the first expert subnetwork is pruned; if it is determined that the second expert subnetwork has a routing structure change, and the change feature quantity of the second expert subnetwork is greater than or equal to the preset threshold, the second expert subnetwork is retained; if it is determined that the third expert subnetwork has no routing structure change, and the change feature quantity of the third expert subnetwork is less than the preset threshold, the third expert subnetwork is pruned.

[0103] In some optional implementations, the adjustment module 304 further includes: a fifth adjustment module unit, used to adjust the probability of allocating lexical embedding features to the expert subnetwork corresponding to the target variable feature quantity to zero based on the gating network; stop updating the parameters in the expert subnetwork corresponding to the target variable feature quantity; and retain the structure of the expert subnetwork corresponding to the target variable feature quantity.

[0104] In some optional implementations, the aforementioned question-answering model processing apparatus further includes: a cooling module, used to stop acquiring statistical results within a preset cooling time after obtaining the structurally adjusted question-answering model; to resume acquiring statistical results after the preset cooling time; and to prune the expert subnetwork corresponding to the target change feature if the target change feature is less than a preset threshold.

[0105] For a description of the features in the embodiment corresponding to the question-answering model processing device, please refer to the relevant description in the embodiment corresponding to the question-answering model processing method, which will not be repeated here.

[0106] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0107] The following is a detailed reference. Figure 4 This diagram illustrates a structural schematic suitable for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory 402 or a program loaded from memory 408 into random access memory 403. Random access memory 403 also stores various programs and data required for the operation of the electronic device. The processor 401, read-only memory 402, and random access memory 403 are interconnected via bus 404. Input / output interface 405 is also connected to bus 404.

[0108] Typically, the following devices can be connected to the input / output interface 405: input devices 406 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory devices 408 including, for example, magnetic tape, hard disk, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0109] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described question-answering model processing method embodiments when run.

[0110] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0111] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described question-answering model processing method embodiments.

[0112] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described question-answering model processing method embodiments.

[0113] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] The above provides a detailed description of the question-answering model processing method and apparatus provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for processing question-answering models, characterized in that, The method includes: Multiple question samples are obtained, and the multiple question samples are respectively input into a pre-trained question-answering model to generate a response sequence corresponding to the question samples; Based on the target question sample and the target response sequence corresponding to the target question sample, determine the reward value of the target response sequence; The pre-trained question-answering model is statistically analyzed to determine whether each expert subnetwork is activated during the generation of multiple response sequences and the reward value of the activated expert subnetwork. The reward value of the target response sequence is associated with the activated expert subnetwork to determine the reward value of the activated expert subnetwork. Based on the statistical results of each expert sub-network, the change feature of the reward value of the activated expert sub-network is determined, and it is determined whether the change feature is less than a preset threshold. If the target change feature is less than the preset threshold, the expert sub-network corresponding to the target change feature is pruned to obtain the question-answering model with adjusted structure. The variable feature quantity includes one or more of the following: volatility variable quantity, gradient sensitivity variable quantity, or comprehensive variable quantity. The volatility variable quantity is used to characterize the difference in reward value when the activated expert subnetwork participates in the decision-making of generating the response sequence for different problem samples. The gradient sensitivity variable quantity is used to characterize the difference in reward value caused by parameter changes in the activated expert subnetwork. The comprehensive variable quantity is obtained by weighted summation of the volatility variable quantity and the gradient sensitivity variable quantity. The variable feature quantity is determined according to the actual application scenario. In simple scenarios, the variable feature quantity is configured as volatility variable quantity. In complex scenarios, the variable feature quantity is configured as comprehensive variable quantity. When parameter stability is a concern, the variable feature quantity is configured as gradient sensitivity variable quantity.

2. The method for question-answering model processing according to claim 1, characterized in that, When the variable characteristic includes volatile variable, determining the variable characteristic of the reward value of the activated expert sub-network based on the statistical results of each expert sub-network includes: Based on the statistical results, the mean reward value of the activated expert subnetwork is determined; Based on the reward value of the activated expert subnetwork and the mean, the variance of the reward value of the activated expert subnetwork is determined, and the volatility change of the reward value of the activated expert subnetwork is obtained.

3. The method for question-answering model processing according to claim 2, characterized in that, When the change feature includes gradient sensitivity change, determining the change feature of the reward value of the activated expert sub-network based on the statistical results of each expert sub-network includes: Determine the set of problem samples processed by the activated expert sub-network; Determine the gradient vector of the reward value of the activated expert subnetwork with respect to the parameters in the activated expert subnetwork; Determine the norm of the gradient vector; The expected value of the norm corresponding to the problem sample in the problem sample set is taken to obtain the gradient sensitivity change of the reward value of the activated expert subnetwork.

4. The question-answering model processing method according to claim 3, characterized in that, When the change feature includes a comprehensive change, determining the change feature of the reward value of the activated expert sub-network based on the statistical results of each expert sub-network includes: Determine the volatility and gradient sensitivity changes of the reward value of the activated expert subnetwork; Determine the first normalized result of the volatility change and the second normalized result of the gradient sensitivity change; The weighted sum of the first normalization result and the second normalization result yields the comprehensive change in the reward value of the activated expert subnetwork.

5. The method for question-answering model processing according to claim 1, characterized in that, The method further includes: Based on a preset detection time interval, the system statistically analyzes whether each expert subnetwork in the pre-trained question-answering model is activated and the reward value obtained during the generation of multiple response sequences.

6. The method for question-answering model processing according to claim 5, characterized in that, The method further includes: Obtain the routing distribution data of each expert subnetwork in the pre-trained question answering model during the process of generating multiple response sequences, and determine the routing distribution change rate of each expert subnetwork based on the routing distribution data; During adjacent preset detection periods, based on the comparison results between the average route distribution change rate and the route change rate threshold of each expert sub-network, it is determined whether there is a route structure change in each expert sub-network. If the average route distribution change rate is less than or equal to the route change rate threshold, it is determined that there is no route structure change. If the average route distribution change rate is greater than the route change rate threshold, it is determined that there is a route structure change. If it is determined that the routing structure of the first expert subnetwork has changed, and the change characteristic of the first expert subnetwork is less than the preset threshold, the first expert subnetwork is pruned. If it is determined that the routing structure of the second expert subnetwork has changed, and the change characteristic of the second expert subnetwork is greater than or equal to the preset threshold, the second expert subnetwork is retained. If it is determined that the third expert subnetwork does not have the routing structure change, and the change characteristic quantity of the third expert subnetwork is less than the preset threshold, the third expert subnetwork is pruned.

7. The method for question-answering model processing according to claim 5, characterized in that, If the target change feature quantity is less than the preset threshold, the expert subnetwork corresponding to the target change feature quantity is pruned, including: Based on the order of the changing feature quantities from high to low, the activated expert subnetworks are sorted, and the expert subnetworks in the top preset percentage of the sorting results are determined as retained expert subnetworks. The expert subnetworks in the sorting results other than the retained expert subnetworks are identified as expert subnetworks to be adjusted. If, in the expert subnetwork to be adjusted, there exists an expert subnetwork whose change feature quantity is less than the preset threshold within multiple consecutive preset detection durations, then the expert subnetwork to be pruned is pruned.

8. The method for question-answering model processing according to claim 1, characterized in that, The pre-trained question-answering model includes a multi-layer neural network, wherein at least one layer is a hybrid expert layer, the hybrid expert layer including at least one gating network and multiple expert sub-networks, and the expert sub-networks corresponding to the target change features are pruned as follows: Based on the gated network, the probability of allocating the word embedding feature to the expert subnetwork corresponding to the target change feature is adjusted to zero. Specifically, the problem sample is segmented into words to obtain characters or words, the characters or words are converted into word units, and the word embedding feature is generated based on the word units. Stop updating the parameters in the expert subnetwork corresponding to the target change feature; The structure of the expert subnetwork corresponding to the target change feature quantity is preserved.

9. The method for question-answering model processing according to claim 1, characterized in that, The method further includes: After obtaining the question-and-answer model with the adjusted structure, the acquisition of the statistical results is stopped within a preset cooling period; After the preset cooling time, the statistical results are retrieved again. If the target change feature is less than the preset threshold, the expert subnetwork corresponding to the target change feature is pruned.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the question-answering model processing method as described in any one of claims 1 to 9 when executing the computer program.

Images