Collaborative office-oriented intelligent question and answer method and apparatus, and electronic device

Abstract

The embodiment of the invention provides an intelligent question and answer method and device for collaborative office and electronic equipment. The method comprises the steps that question description information input by a target user is received; processing the question description information based on a pre-trained multi-domain knowledge model to obtain a target question and answer result corresponding to the question description information; wherein the multi-domain knowledge model is determined based on an expert model, a language model linear layer and an expert sharing model corresponding to each domain, the expert model comprises a low-rank adaptation matrix, and the low-rank adaptation matrix is used for adjusting parameter quantity after input data processing; the expert sharing model is used for processing all the output first sample data. According to the technical scheme, the cooperative work capability between expert models is improved, the question and answer accuracy is improved, and the resource consumption in the model training process is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an intelligent question answering method and device for collaborative office work and electronic equipment. BACKGROUND

[0002] With the diversification of business, traditional office automation (OA) systems cannot meet the growing business needs, especially for knowledge management involving finance, personnel management, administration and business, etc. It is necessary to build a more intelligent and efficient office system.

[0003] At present, the existing method usually constructs different types of expert models, and classifies and trains similar knowledge. When the user asks a question, a gating mechanism is used to select the answer from different expert models.

[0004] However, in the existing method, each expert model works independently, lacks effective information exchange and knowledge sharing mechanism, the model cannot fully utilize the knowledge of different experts, weakens the collaborative work ability among experts, and reduces the overall performance of the model and the accuracy of the answer. SUMMARY

[0005] The embodiments of the present application provide an intelligent question answering method and device for collaborative office work and electronic equipment to realize efficient intelligent question answering in a collaborative office scenario.

[0006] In a first aspect, the embodiments of the present application provide an intelligent question answering method for collaborative office work, comprising:

[0007] receiving problem description information input by a target user;

[0008] processing the problem description information based on a pre-trained multi-domain knowledge model to obtain a target question answering result corresponding to the problem description information;

[0009] The multi-domain knowledge model is determined based on an expert model corresponding to each domain, a language model linear layer and an expert shared model. The expert model includes a low-rank adaptation matrix, which is used to adjust the parameter quantity after processing the input data, and the expert shared model is used to process all first sample data output.

[0010] In a second aspect, the embodiments of the present application further provide an intelligent question answering device for collaborative office work, comprising:

[0011] a problem description information receiving module configured to receive problem description information input by a target user;

[0012] The target question answering result acquisition module is used to process the question description information based on the pre-trained multi-domain knowledge model in order to obtain the target question answering result corresponding to the question description information;

[0013] The multi-domain knowledge model is determined based on the expert model, the linear layer of the language model, and the expert-shared model corresponding to each domain. The expert model includes a low-rank adaptation matrix, which is used to adjust the parameters after processing the input data. The expert-shared model is used to process all the first sample data output.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0015] At least one processor; and

[0016] A memory that is communicatively connected to at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the intelligent question-answering method for collaborative work provided in any embodiment of the present invention.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute the intelligent question-and-answer method for collaborative office work provided in any embodiment of the present invention.

[0019] This invention receives question description information input by a target user and processes it based on a pre-trained multi-domain knowledge model to obtain the target question-and-answer result corresponding to the question description information. The multi-domain knowledge model is determined based on expert models, linear layers of language models, and expert-shared models corresponding to each domain. Each expert model includes a low-rank adaptation matrix, used to adjust the parameters after processing the input data. The expert-shared model is used to process all the output first sample data. This improves the collaborative working ability between expert models, increases question-and-answer accuracy, and reduces resource consumption during model training.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of an intelligent question-answering method for collaborative office work provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of an expert model architecture based on a low-rank adaptation matrix provided in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of an expert model architecture that incorporates shared experts, provided as an embodiment of the present invention.

[0025] Figure 4 A flowchart of an intelligent question-answering method for collaborative office work provided by an embodiment of the present invention;

[0026] Figure 5 A flowchart of an intelligent question-answering method for collaborative office work provided by an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of a vector hybrid architecture provided in an embodiment of the present invention;

[0028] Figure 7 This is an overall architecture diagram of an intelligent question-answering method for collaborative office work provided by an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of the structure of an intelligent question-and-answer device for collaborative office work provided in an embodiment of the present invention;

[0030] Figure 9 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Figure 1 This invention provides a flowchart of an intelligent question-answering method for collaborative office work, applicable to scenarios involving intelligent question answering in collaborative office environments. The method can be executed by an intelligent question-answering device for collaborative office work, which can be implemented in hardware and / or software and can be configured in a computing device. Figure 1 As shown, the method includes:

[0034] S110. Receive the problem description information input by the target user.

[0035] The target users are those who use the intelligent question-and-answer function in the collaborative office system, such as staff members who need to query information or solve problems. The question description information is the query content entered by the target user in natural language, including but not limited to a question, instruction, or scenario description. The question description information includes, but is not limited to, a piece of audio, an image, or a piece of text; this information is input to the system.

[0036] Specifically, target users input a text message into the collaborative office system regarding the problems they face, and the system will return corresponding answers to support the target users in making relevant decisions.

[0037] For example, if a target user wants to know about the reimbursement policy, they can enter a description of the problem on the terminal, such as: What are the reimbursement policies this month, and what is the reimbursement process?

[0038] Optionally, the problem description information is a description of a problem in an office setting, which includes at least one or more of the following: finance, human resources management, administration, and business operations.

[0039] The specific application scenario for the multi-domain knowledge model is the office environment. The knowledge scope in the finance domain covers activities related to fund operations, including but not limited to expense reimbursement and cost control. The knowledge scope in the human resources management domain covers activities related to internal human resource management, including but not limited to recruitment, training, and performance evaluation. The knowledge scope in the administration domain covers activities related to daily basic operations, including but not limited to procurement, meeting arrangements, and logistical support. The knowledge scope in the business domain includes but is not limited to product technical solutions and project management.

[0040] Specifically, multi-domain knowledge models are applied in scenarios involving internal collaborative work and management. This scenario consists of multiple well-defined domains, including but not limited to finance, human resources management, administration, and business areas.

[0041] The technical solution provided in this invention receives question description information input by a target user. Based on a pre-trained multi-domain knowledge model, the question description information is processed to obtain the target question-answering result corresponding to the question description. The multi-domain knowledge model is determined based on expert models, linear layers of language models, and expert-shared models corresponding to each domain. The expert models include low-rank adaptation matrices, which are used to adjust the parameters after processing the input data. The expert-shared model is used to process all the first samples output. This improves the collaborative working ability between expert models and increases the accuracy of question answering.

[0042] S120. Process the problem description information based on the pre-trained multi-domain knowledge model to obtain the target question-answering result corresponding to the problem description information.

[0043] The multi-domain knowledge model is determined based on the expert model, the linear layer of the language model, and the expert-shared model corresponding to each domain. The expert model includes a low-rank adaptation matrix, which is used to adjust the parameters after processing the input data. The expert-shared model is used to process all the first sample data output.

[0044] In this embodiment, the multi-domain knowledge model is a language model containing multiple domain expert components, such as a language model based on the Transformer architecture. This model collaboratively processes complex questions that target users may encounter in collaborative work, spanning different professional domains. The target question-answering result is the final answer returned to the user after processing by the model. Expert models are sub-models specifically trained and optimized for a particular domain; each expert model possesses the professional knowledge and terminology of its corresponding domain.

[0045] Specifically, the problem description information is input into a pre-trained multi-domain knowledge model, which integrates expert models from multiple domains, the inherent linear layers in the original language model, and an expert-shared model. The expert models incorporate a low-rank adaptation matrix, allowing for parameter updates without requiring a complete overhaul when facing new domains by introducing a small number of trainable parameter increments. The expert-shared model processes all initial sample data after feature extraction from the problem description information, learning and fusing general knowledge from multiple domains.

[0046] As can be seen from the above, the multi-domain knowledge model includes multiple models. Next, each model and its corresponding function will be described in detail.

[0047] In this embodiment, the multi-domain knowledge model is determined based on the expert model, the linear layer of the language model, and the expert-shared model corresponding to each domain. Specific domains include, but are not limited to, finance, human resource management, administration, and business. The linear layer of the language model is a neural network layer located in the original language model. It is at the same level as the expert model and the expert-shared model, working together to comprehensively process the input information. The expert-shared model does not specify a specific domain; it is responsible for processing common, fundamental knowledge and capabilities across all domains, providing knowledge supplementation and adjustment to the expert model for a specific domain. The first sample data is the data obtained after deep feature extraction from the input problem description information through multiple layers of networks in the multi-domain knowledge model. This data contains deep semantics from the problem description information.

[0048] The expert model includes a low-rank adaptation matrix, a model fine-tuning component implemented through low-rank decomposition. Structurally, this component consists of a dimensionality reduction matrix and a recovery matrix. The dimensionality reduction matrix projects the input data into a low-dimensional space, retaining only the core features most relevant to the current task. The recovery matrix maps the filtered low-dimensional features back to the original high-dimensional space. Introducing the low-rank adaptation matrix reduces training parameters and computational costs.

[0049] For example, if an additional low-rank adaptation matrix is ​​initialized on a linear layer in a language model, the output of that layer would be:

[0050]

[0051] in, For input features that include different expert models, such as financial knowledge expert models Administrative Knowledge Expert Model and human resources knowledge expert model ,Right now . The output of the low-rank adaptation matrix, For the original language model linear layer, For predefined lower The weights of the low-rank adaptation matrix.

[0052] Specifically, the problem description information is input into a pre-trained multi-domain knowledge model, which integrates expert models from multiple domains, the inherent linear layers in the original language model, and an expert-shared model. The expert models incorporate a low-rank adaptation matrix, allowing for parameter updates without requiring a complete overhaul when facing new domains by introducing a small number of trainable parameter increments. The expert-shared model processes all initial sample data after feature extraction from the problem description information, learning and fusing general knowledge from multiple domains.

[0053] Optionally, the problem description information is processed based on a pre-trained multi-domain knowledge model to obtain the target question-answering result corresponding to the problem description information, including:

[0054] The input features corresponding to the question description information are extracted using a feature extraction layer in a multi-domain knowledge model. A gated network is then used to determine a target expert model adapted to these input features. The input features are processed by the target expert model, a linear layer of the language model, and an expert-shared model to obtain the target features. The input features are determined based on optimized learning parameters, and the model parameters in the linear layer of the language model are also optimized. Based on the target features, the target question-answering result is determined.

[0055] The input features are the features output by the feature extraction layer in the multi-domain knowledge model after the problem description information has passed through it. The feature extraction layer is the foundational front-end module of the model, composed of multiple network layers. It converts the problem description information into a mathematical representation that the model can understand, such as encoding the vocabulary, grammar, and shallow semantic information in the problem description information into a high-dimensional vector. The gating network acts as a routing scheduler. It receives the input features and outputs a probability distribution through a gating function, which is used to select which target expert model(s) to invoke to handle the current problem.

[0056] In this embodiment, Figure 2 This is a schematic diagram of an expert model architecture based on a low-rank adaptation matrix, provided as an embodiment of the present invention. Figure 2As shown, the target expert model is one or more of the most relevant domain-specific models activated after being calculated based on input features by a gating network. These models are trained on their corresponding domain datasets. After receiving input features assigned by the gating network, they utilize their deep domain knowledge to transform and infer the features, outputting a specialized feature representation. This can be understood as the gating network outputting different probability distributions in real time for different input questions, thereby activating different combinations of target expert models. The target features are feature representations enhanced with specialized semantics, used to generate target question-answering results. The optimized learning parameters are the model weights, fixed after the multi-domain knowledge model completes pre-training and fine-tuning, reaching their optimal state. During the inference phase, these parameters are frozen and used to process user questions. The inference phase is the application phase of the multi-domain knowledge model.

[0057] For example, if a low-rank fitting matrix corresponding to an expert model is designed in the linear layer of a multi-domain knowledge model, each matrix is ​​a low-rank fitting matrix appended to the expert model. When sample input data is input to the feature extraction layer of the multi-domain knowledge model to obtain input features, a gating network is used to determine the target expert model that fits the input features, such as a financial knowledge expert, an administrative knowledge expert, and a human resources knowledge expert. The financial knowledge expert model contains a corresponding low-rank fitting matrix. The administrative knowledge expert model includes a corresponding low-rank adaptation matrix. And the human resources knowledge expert model includes the corresponding low-rank adaptation matrix. When the input features are simultaneously fed into multiple target expert models and the original language model, the output is:

[0058]

[0059] in, , as well as and These are all input features, used only for calibrating the input location. The features are the output of a linear layer weighted fusion of multiple target expert models and the original language model.

[0060] Specifically, after obtaining the problem description information, it is input into a pre-trained multi-domain expert model. The model's feature extraction layer extracts features from the problem description information. These features are then input into a gating network to obtain a probability distribution. Multiple probability scores are sorted in descending order, and a pre-selected number of expert models is chosen. Based on this, the target expert model corresponding to the probability distribution scores of the pre-selected number of expert models is selected. The input features are then fed into the target expert model, the linear layer of the language model, and the expert-shared model to process the input features and obtain the target features used to output the target question-answering result.

[0061] For example, Figure 3 This is a schematic diagram of an expert model architecture that incorporates shared experts, provided as an embodiment of the present invention. Figure 3 As shown, the input is the problem description information, which is then processed by the feature extraction layer in a multi-domain knowledge model, resulting in input features. These features are then processed by a gating network to generate probability distributions for multiple experts, selecting target expert models such as model y and model d. The input features are then used as... and The input to the corresponding expert model includes the corresponding low-rank fitness matrix. and Simultaneously, an additional shared expert model is constructed, containing the corresponding low-rank adaptation matrix. The low-rank adaptation matrix of the additional shared expert synchronization model d and the knowledge base y is as follows:

[0062]

[0063] The target features are obtained by processing the input features based on the target expert model, the linear layer of the language model, and the expert sharing model. :

[0064]

[0065] in, This is a hyperparameter that weighs the ratio between the shared expert model and the target expert model (y and d). This value can be preset, such as being fixed at 0.5 by default. Simultaneous activation of both the shared and target expert models allows for the generation of specific responses for inference steps within a particular domain by adjusting the parameters. Through this operation, the knowledge information passed to the shared expert is also passed to the target expert model with a certain weight, achieving knowledge sharing between experts.

[0066] Figure 4 This is a flowchart illustrating an intelligent question-answering method for collaborative office work, provided by an embodiment of the present invention. This embodiment describes the training process of a multi-domain knowledge model based on the aforementioned optional embodiments. Figure 4As shown, the multi-domain knowledge model to be trained includes a feature extraction layer to be trained, a feature processing model connected to the output of the feature extraction layer, and a gating network, multiple expert models to be trained communicating with the gating network, a linear layer of the language model to be trained, and a shared expert model to be trained. The method for training the multi-domain knowledge model to obtain the multi-domain knowledge model includes:

[0067] S210. For multiple first sample data corresponding to different fields, extract the input features of the sample input data based on the feature extraction layer to be trained.

[0068] The first sample data includes sample input data and sample output data.

[0069] In this embodiment, the first sample data is used to train the multi-domain knowledge model. This sample data includes sample input data and sample output data. The sample input data consists of various expert knowledge corpora, and the sample output data consists of the correct answers corresponding to each expert knowledge corpus, along with their corresponding domain labels. The feature extraction layer to be trained is an untrained feature extraction layer; its model parameters are not fixed and need to be adjusted based on the first sample data during the training process. The sample input data is the input portion of the first sample data, containing unprocessed raw question description information from different domains. The sample output data is the expected output portion of the first sample data, representing the correct target answer corresponding to the sample input data. During training, the purpose of training the multi-domain knowledge model is to make its output infinitely close to the sample output data.

[0070] Specifically, multiple first sample data are obtained, which are collections of sample data from multiple domains. The sample input data from the first sample data is input into an untrained multi-domain knowledge model, and the input features of the sample input data are extracted through the feature extraction layer in the model.

[0071] S220. Based on a gating network, determine at least one processable expert model corresponding to the input feature from multiple expert models to be trained, and output a first feature corresponding to the input feature based on at least one processable expert model.

[0072] In this system, the expert models to be trained are multiple expert models whose parameters are not yet fixed during the model training phase, and each expert model focuses on learning knowledge from a specific domain. The processable expert models are one or more expert models selected from the multiple trainable expert models based on the current input features by a gating network. The number of processable expert models can be set by the user; this value is an integer ranging from [0, N], where N is the total number of expert models, meaning the number of processable expert models is less than or equal to the number of expert models. The first feature is a feature representation containing specific domain semantics, calculated by the processable expert models after processing the input features. This feature is obtained by the combined output of multiple processable expert models, a shared expert model, and a linear layer of the semantic model.

[0073] Specifically, the input features output by the feature extraction layer are obtained, and the input features are input into the gating network to calculate a probability distribution. Based on this distribution, at least one processable expert model is selected from multiple expert models to be trained. The input features are simultaneously input into the processable expert model, and the enhanced first feature is output.

[0074] Optionally, the expert model to be trained includes a low-rank adaptation matrix, and the model parameters of the expert model are adjusted by adjusting the matrix weights of the low-rank adaptation matrix.

[0075] In this context, the model parameters of the expert model are the network parameters in the expert model to be trained. These parameters are frozen during the fine-tuning process, and their values ​​remain unchanged. Adjusting the model parameters of the expert model is equivalent to updating the parameters during model training, which mainly optimizes the weights in the low-rank adaptation matrix.

[0076] Specifically, the network structure of the expert model to be trained contains a low-rank adaptation matrix. During the training process, the parameters of the expert model are not adjusted for all model parameters, but rather for the weights in the low-rank adaptation matrix through gradient descent optimization.

[0077] S230. Determine the second feature corresponding to the input feature based on the expert model to be trained; and determine the third feature corresponding to the input feature based on the expert sharing model.

[0078] The second feature is the feature output by the input features fed into at least one model to be trained during the model training phase, and includes the features output by each model to be trained. The third feature is the specialized semantic information output by the expert-shared model in multiple domain sets after processing the input features.

[0079] Specifically, after inputting the input features into the expert model to be trained, the features output by each expert model are accumulated to obtain the second feature. At the same time, the input features are input into the expert sharing model, which outputs a third feature that focuses on cross-domain general knowledge.

[0080] S240. Based on the first feature, the second feature, the third feature, and the sample output data, at least one of the model parameters in the multi-domain knowledge model to be trained, including the expert model, the linear layer of the language model to be trained, and the shared expert model to be trained, is corrected.

[0081] In this embodiment, the model parameters are the optimizable weights and biases within all trainable components, including the expert model, the linear layer of the language model to be trained, and the expert-shared model to be trained. These parameters are variables determined through model training. The modification involves updating the model parameters using a gradient backpropagation algorithm based on the loss function. By reducing the difference between the predicted values ​​of the multi-domain knowledge model and the sample output data, the accuracy of the multi-domain knowledge model is improved. The predicted values ​​of the multi-domain knowledge model are determined by the first feature, the second feature, and the third feature.

[0082] Specifically, the input features are fed into the multi-domain knowledge model to be trained. The intermediate results generated through multiple paths in the model are the first feature, the second feature, and the third feature. These multiple paths can be understood as the input features passing through a processable expert model, a linear layer of a language model, and a shared expert model, respectively. The sample output data, representing the true answer, is compared with the first, second, and third features. The loss function is used to calculate the loss, and the model parameters in the multi-domain knowledge model are adjusted accordingly.

[0083] Optionally, based on the first feature, second feature, third feature, and sample output data, at least one of the model parameters in the multi-domain knowledge model to be trained—the expert model, the linear layer of the language model to be trained, and the expert-shared model to be trained—is modified, including:

[0084] The first, second, and third features are weighted according to the weight value corresponding to each feature to output the predicted output data corresponding to the weighted features. Based on the predicted output data and the sample output data, the sample loss value is determined, and the model parameters in at least one of the expert models, the linear layer of the language model to be trained, and the expert shared model to be trained are corrected based on the sample loss value.

[0085] In this embodiment, the weight value refers to the trainable parameter matrix itself, which constitutes the low-rank adaptation matrix in the expert model. The aforementioned weights are the targets optimized by the gradient descent algorithm during model training. Weighting is a feature fusion operation that multiplies the first, second, and third features by their respective weight values. All weighted features are summed to form a unified feature representation. The predicted output data is the final prediction result output by the multi-domain knowledge model to be trained after processing the sample input data; this prediction result corresponds to the predicted answer of the sample input data. The sample loss value is a numerical value calculated using a loss function, representing the difference between the predicted output data and the sample output data, and is used to correct the model parameters of the multi-domain knowledge model.

[0086] Specifically, the sample input data from the first sample data is input into the multi-domain knowledge model to be trained, and the input features are obtained through a feature extraction layer. The output probability distribution is then passed through a gating network. At least one usable expert model is determined based on a preset number of experts. The input features are simultaneously input into the linear layer of the language model, at least one usable expert model, and an expert-shared model to obtain the first, second, and third features. The weight corresponding to each feature is then weighted with the aforementioned features, and all weighted features are summed. Based on the predicted output data from the multi-domain knowledge experts and the sample output data from the first sample data, a sample loss value is calculated. Backpropagation is performed based on the sample loss value to correct the parameters in all trainable components of the multi-domain knowledge model.

[0087] S250. The model obtained when the loss function in the multi-domain knowledge model to be trained converges shall be used as the multi-domain knowledge model.

[0088] The loss function includes the result loss function and the loss function corresponding to the label of the sample input data.

[0089] In this embodiment, the loss function is used to quantify the difference between the predicted output data obtained by the multi-domain knowledge model and the sample output data. This function is a composite loss function, including an outcome loss function and a loss function corresponding to the labels of the sample input data. The outcome loss function measures the difference between the model's predicted output data and the sample output data. The labels of the sample input data are domain category labels; each sample input data is labeled with a label for the domain involved in the input data. Labeling can be done manually or automatically using an existing model. The loss function corresponding to the labels of the sample input data is typically the difference between the probability distribution predicted by the gating network in the multi-domain knowledge model and the labels of the sample input data. Convergence refers to the process where, after multiple iterations of training, the value of the loss function decreases to a local minimum and tends to stabilize, without showing a significant decreasing trend.

[0090] Specifically, during training, the sample input data from the first set of sample data is fed into the multi-domain knowledge model to obtain the predicted output data and the probability distribution of the gating network output in the multi-domain knowledge model. Based on the predicted output data, sample output data, labels of the sample input data, and the probability distribution results, the loss value is calculated using a loss function. After multiple iterations, when the loss value tends to stabilize, the model at this point is taken as the trained multi-domain knowledge model.

[0091] The technical solution provided by this invention extracts input features from multiple first sample data corresponding to different domains based on a feature extraction layer to be trained. Using a gating network, at least one processable expert model corresponding to the input features is determined from multiple expert models to be trained. A first feature corresponding to the input features is output based on this at least one processable expert model. Further, a second feature corresponding to the input features is determined based on the expert model to be trained, and a third feature corresponding to the input features is determined based on an expert-shared model. Based on the first, second, and third features and the sample output data, the model parameters in at least one usable expert model, the linear layer of the language model to be trained, and the expert-shared model to be trained are corrected. Finally, after multiple iterations of training, the model obtained when the loss function of the multi-domain knowledge model to be trained converges is taken as the multi-domain knowledge model. This improves the collaborative working ability of expert models, realizes the fusion of knowledge from multiple domains, and reduces resource consumption during model training.

[0092] Figure 5 This is a flowchart of an intelligent question-answering method for collaborative office work provided by an embodiment of the present invention. This embodiment further refines the process after obtaining a multi-domain knowledge model based on the above optional embodiments. Figure 5 As shown, the method optimizes the learning parameters in a multi-domain knowledge model and the model in the linear layer of a language model, including:

[0093] S310. For multiple second sample data corresponding to different domains, with the first learning parameter, the second learning parameter in the multi-domain knowledge model and the parameters in the linear layer of the language model remaining unchanged, the fourth feature corresponding to the sample input data in the second sample data is extracted based on the first learning parameter in the feature extraction layer.

[0094] The first learning parameter is the weight parameter used by the feature extraction layer in the original language model to generate the key vector, located in the key projection matrix of the feature extraction layer. The second learning parameter is the weight parameter used by the feature extraction layer in the original language model to generate the value vector, located in the value projection matrix of the feature extraction layer. The fourth feature is the feature representation extracted from the sample input data in the second sample data through the first learning parameter. The second sample data is a set of sample data containing multiple domains, including sample input data and sample output data.

[0095] For example, the feature extraction layer in the language model generates the query projection matrix, the key projection matrix, and the value projection matrix, respectively:

[0096]

[0097] Where K is the first learning parameter, V is the second learning parameter, and Q is the learning parameter in the query projection matrix. The formula calculates the inner product of each row vector of matrix Q and K; to prevent the inner product from becoming too large, it is divided by . The square root of Q. Multiplying Q by the transpose of K yields a matrix with n rows and n columns, where n is the number of words in the sentence. This matrix represents the strength of the correlation between words.

[0098] Specifically, the second sample data is input into a multi-domain knowledge model for parameter learning. The sample input data in the second sample data passes through the feature extraction layer of the multi-domain knowledge model, outputting a key vector dependent on the first learning parameter and a value vector dependent on the second learning parameter. The first learning parameter, the second learning parameter, and the parameters in the linear layer of the language model remain fixed, obtaining the feature representation extracted from the sample input data through the first learning parameter.

[0099] S320. Input the fourth feature into a pre-built vector fusion architecture to determine the model fit attribute of the fourth feature relative to at least one expert model based on the evaluation route in the vector fusion architecture.

[0100] The vector hybrid architecture is a hybrid output vector architecture that introduces new vectors and performs soft merging of them. Figure 6 This is a schematic diagram of a vector hybrid architecture provided in an embodiment of the present invention. Figure 6As shown, the vector hybrid architecture includes an evaluation route, which is a trainable routing network consisting of a dense layer with trainable weights and a softmax function. Using the vector hybrid architecture to fine-tune multi-domain knowledge models, updating only a limited number of parameters, reduces the resource consumption of the model training process. The input to this vector hybrid architecture is the fourth feature. The model fit attribute, calculated by the evaluation route, is a weight used to quantify the matching degree between the fourth feature and each expert model. This attribute determines at least one expert model used for the final output.

[0101] For example, the evaluation route is determined by a set of trainable weights. It consists of dense layers and a softmax function. After the fourth feature is input into the vector mixing architecture, the probability distribution score is obtained through evaluation routing. :

[0102]

[0103] The probability distribution scores are combined with those of each expert to obtain the model fit attributes:

[0104]

[0105] Specifically, a fourth feature is obtained and input into a pre-constructed vector fusion architecture. The evaluation route within this architecture is used to determine at least one expert model most relevant to the fourth feature, thereby obtaining model fit attributes.

[0106] S330. Based on the model adaptation attributes and corresponding linear layer parameters of at least one expert model, determine the first adjustment parameter, the second adjustment parameter, and the third adjustment parameter for processing the input features.

[0107] This approach introduces multiple trainable vectors to represent different directions and patterns in the feature space. The first adjustment parameter corresponds to the transformation parameter of the key vector in the feature extraction layer of the multi-domain knowledge model. The second adjustment parameter corresponds to the transformation parameter of the value vector in the feature extraction layer of the multi-domain knowledge model. The third adjustment parameter corresponds to the transformation parameter of the original linear layer vector in the multi-domain knowledge model.

[0108] Specifically, the model adaptation attributes of the expert model are obtained, and the key projection matrix, value projection matrix, and transformation parameters of the original linear layer in the feature extraction layer of the multi-domain knowledge model are determined. The first adjustment parameter, the second adjustment parameter, and the third adjustment parameter are then determined. The model is then processed based on these parameters.

[0109] For example, the three new vectors obtained based on the model adaptation attributes and the corresponding linear layer parameters are as follows: , ,as well as The three adjustment parameters mentioned above re-adjust the feature extraction model for key and value activation, so that each feature extraction layer only requires vector and input. Each feature extraction layer requires key and value activation. Only vector and input are needed for the router to make a judgment, and the output features should be:

[0110]

[0111]

[0112] Where Q and V are the query, key, and value projection matrices of the feature extraction module. as well as These are the frozen weights for the feedforward layer in the pre-trained model. Besides parameter efficiency, the vector fusion architecture method also performs routing computation through soft merging. Since the vectors in the vector fusion architecture method are linear functions, a weighted average is first calculated, and the model adaptation attributes are output for fine-tuning transformation using the parameters of the model adaptation expert attributes.

[0113]

[0114]

[0115] in, For each expert vector in the vector hybrid architecture, and This refers to the weighted weights corresponding to each expert vector. Model adaptation attributes are calculated using soft merging to compute expert quantities.

[0116] S340. Adjust the learning parameters in the multi-domain knowledge model and the model parameters in the linear layer of the language model according to the first adjustment parameter, the second adjustment parameter and the third adjustment parameter.

[0117] The learning parameters are the first and second learning parameters of the feature extraction layer in the multi-domain knowledge model.

[0118] Specifically, based on the obtained first adjustment parameter, second adjustment parameter, and third adjustment parameter, the first learning parameter, second learning parameter, and parameters in the linear layer of the language model are adjusted.

[0119] It can be explained that the vector fusion framework is constructed to fine-tune for each expert, maintaining a fixed density layer and training weights during fine-tuning, while the evaluation routes and vectors are trained from scratch. This approach can adapt the pre-trained language model during fine-tuning.

[0120] The technical solution provided by this invention extracts a fourth feature corresponding to the input data of the sample data from multiple second sample data corresponding to different domains, while keeping the first learning parameters, second learning parameters, and parameters in the linear layer of the language model unchanged. This is done based on the first learning parameters in the feature extraction layer. A vector fusion architecture is pre-constructed, and the fourth feature is input into this architecture. Based on the evaluation route in the vector fusion architecture, the model fit attribute of the fourth feature relative to at least one expert model is determined. Further, based on the model fit attribute of at least one expert model and the corresponding linear layer parameters, a first adjustment parameter, a second adjustment parameter, and a third adjustment parameter are determined for processing the input feature. Based on these adjustment parameters, the learning parameters in the multi-domain knowledge model and the model parameters in the linear layer of the language model are adjusted. A vector fusion architecture is used to fine-tune the multi-domain knowledge model. Only a limited number of parameters are updated, reducing resource consumption during model training.

[0121] Figure 7 This is an overall architecture diagram of an intelligent question-answering method for collaborative office work provided by an embodiment of the present invention. Combined with... Figure 7 Understand the technical solutions of the embodiments of the present invention.

[0122] like Figure 7 As shown, this embodiment explains the overall implementation of the solution based on the above optional implementation methods. Specifically, it includes:

[0123] First, after the user inputs a question description, the system feeds this information into a multi-domain knowledge model. The model's feature extraction layer extracts the input features. These features are then processed by a gating network, which outputs a probability distribution for multiple expert models. All expert models are sorted in descending order of their probability scores, and the number of processable expert models is determined based on a pre-set threshold. The input features are then fed into the linear layer of the original language model, the target expert model, and the shared expert model, outputting the first, second, and third features. The target expert model includes a low-rank adaptation matrix. The weights corresponding to each feature are weighted and summed to obtain the final output features. Finally, the multi-domain knowledge model outputs the target question-answering result. For the training process of the multi-domain knowledge model, after obtaining the trained model, a vector hybrid architecture is constructed to optimize the model parameters. Sample input data from a second set of data samples containing different domains is input into the multi-domain knowledge model to be trained. The feature extraction layer outputs K (key projection matrix) dependent on the first learning parameter and V (value projection matrix) dependent on the second learning parameter. The key projection matrix is ​​input into the routing (evaluation routing) to output the probability distribution for each expert model, and the outputs of each expert are combined to obtain the model adaptation attribute. Multiple trainable vectors are multiplied with the model adaptation attribute, outputting three new vectors representing the first adjustment parameter, the second adjustment parameter, and the third adjustment parameter, which are then soft-merged. Based on these adjustment parameters, the first and second learning parameters of the feature extraction layer in the multi-domain knowledge model, as well as the model parameters in the linear layer of the language model, are adjusted.

[0124] The technical solution provided by this invention receives question description information input by a target user. Based on a pre-trained multi-domain knowledge model, the question description information is processed to obtain the target question-answering result corresponding to the question description. The multi-domain knowledge model is determined based on expert models, linear layers of language models, and expert-shared models corresponding to each domain. The expert models include low-rank adaptation matrices, which are used to adjust the parameters after processing the input data. The expert-shared model is used to process all the first samples output. This improves the collaborative working ability between expert models and increases the accuracy of question answering.

[0125] Figure 8 This is a schematic diagram of the structure of an intelligent question-and-answer device for collaborative office work provided by an embodiment of the present invention, as shown below. Figure 8 As shown, the device includes: a problem description information receiving module 410 and a target question and answer result acquisition module 420.

[0126] The problem description information receiving module 410 is used to receive the problem description information input by the target user; the target question answering result acquisition module 420 is used to process the problem description information based on the pre-trained multi-domain knowledge model to obtain the target question answering result corresponding to the problem description information; wherein, the multi-domain knowledge model is determined based on the expert model, the linear layer of the language model and the expert shared model corresponding to each domain, the expert model includes a low-rank adaptation matrix, which is used to adjust the parameter amount after processing the input data, and the expert shared model is used to process all the first sample data output.

[0127] The technical solution provided in this invention receives question description information input by a target user. Based on a pre-trained multi-domain knowledge model, the question description information is processed to obtain the target question-answering result corresponding to the question description. The multi-domain knowledge model is determined based on expert models, linear layers of language models, and expert-shared models corresponding to each domain. The expert models include low-rank adaptation matrices, which are used to adjust the parameters after processing the input data. The expert-shared model is used to process all the first samples output. This improves the collaborative working ability between expert models and increases the accuracy of question answering.

[0128] Based on the above technical solutions, the target question-answering result acquisition module includes a multi-domain knowledge model to be trained, comprising a feature extraction layer to be trained and a feature processing model connected to the output of the feature extraction layer. The feature processing model includes a gating network, multiple expert models to be trained communicating with the gating network, a linear layer of the language model to be trained, and a shared expert model to be trained. Training the multi-domain knowledge model to obtain the multi-domain knowledge model includes:

[0129] The input feature extraction unit is used to extract input features of the sample input data based on the feature extraction layer to be trained for multiple first sample data corresponding to different domains; wherein, the first sample data includes sample input data and sample output data.

[0130] The first feature output unit is used to determine at least one processable expert model corresponding to the input feature from multiple expert models to be trained based on a gating network, and output the first feature corresponding to the input feature based on at least one processable expert model.

[0131] The third feature determination unit is used to determine the second feature corresponding to the input feature based on the expert model to be trained; and to determine the third feature corresponding to the input feature based on the expert sharing model.

[0132] The model parameter correction unit is used to correct the model parameters of at least one of the following multi-domain knowledge models to be trained: the first feature, the second feature, the third feature, and the sample output data: the expert model, the linear layer of the language model to be trained, and the expert shared model to be trained.

[0133] The multi-domain knowledge model determination unit is used to take the model obtained when the loss function in the multi-domain knowledge model to be trained converges as the multi-domain knowledge model. The loss function includes the result loss function and the loss function corresponding to the labels of the sample input data.

[0134] Based on the above technical solutions, the first feature output unit includes: the expert model to be trained includes a low-rank adaptation matrix, and the adjustment of the model parameters of the expert model is reflected by adjusting the matrix weights of the low-rank adaptation matrix.

[0135] Based on the above technical solutions, the model parameter correction unit includes:

[0136] The prediction output data determination subunit is used to weight the first feature, the second feature, and the third feature according to the weight value corresponding to each feature, so as to output the prediction output data corresponding to the weighted features.

[0137] The model parameter correction subunit is used to determine the sample loss value based on the predicted output data and the sample output data, so as to correct the model parameters in at least one of the multi-domain knowledge models to be trained, including the expert model, the linear layer of the language model to be trained, and the expert shared model to be trained, based on the sample loss value.

[0138] Based on the above technical solutions, optionally, the multi-domain knowledge model determining unit further includes: optimizing the learning parameters in the multi-domain knowledge model and the model in the linear layer of the language model, including:

[0139] The fourth feature extraction subunit is used to extract the fourth feature corresponding to the sample input data in the second sample data based on the first learning parameter in the feature extraction layer, while keeping the first learning parameter, the second learning parameter in the multi-domain knowledge model and the parameter in the linear layer of the language model unchanged for multiple second sample data corresponding to different domains.

[0140] A model fit attribute determination subunit is used to input the fourth feature into a pre-built vector fusion architecture to determine the model fit attribute of the fourth feature relative to at least one expert model based on the evaluation route in the vector fusion architecture.

[0141] The parameter determination subunit is used to determine the first adjustment parameter, the second adjustment parameter, and the third adjustment parameter for processing the input features based on the model adaptation attributes and corresponding linear layer parameters of at least one expert model.

[0142] The model parameter adjustment subunit is used to adjust the learning parameters in the multi-domain knowledge model and the model parameters in the linear layer of the language model according to the first adjustment parameter, the second adjustment parameter, and the third adjustment parameter.

[0143] Based on the above technical solutions, the model parameter adjustment subunit includes:

[0144] The learning parameter determination subunit is used to determine the adjusted first learning parameter and second learning parameter based on the first adjustment parameter, the second adjustment parameter, the first learning parameter, the second learning parameter, and the parameter dimension corresponding to the first learning parameter.

[0145] The model parameter determination subunit is used to determine the optimized model parameters based on the third adjustment parameter, the first weight parameter and the second weight parameter in the language linear layer.

[0146] Based on the above technical solutions, the target question-answering result acquisition module includes:

[0147] The target feature acquisition unit is used to extract the input features corresponding to the problem description information based on the feature extraction layer in the multi-domain knowledge model, and to determine the target expert model that is adapted to the input features based on the gating network. The input features are then processed based on the target expert model, the linear layer of the language model, and the expert sharing model to obtain the target features. The input features are determined based on the optimized learning parameters, and the model parameters in the linear layer of the language model are the optimized parameters.

[0148] The target question-answering result determination unit is used to determine the target question-answering result based on the target features.

[0149] Based on the above technical solutions, optionally, the problem description information is a problem description about an office scenario, which includes at least one or more of the following: finance, human resources management, administration, business, and Party building.

[0150] The intelligent question-and-answer device for collaborative office work provided in this embodiment of the invention can execute the intelligent question-and-answer method for collaborative office work provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0151] Figure 9A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0152] like Figure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0153] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0154] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as an intelligent question-answering method for collaborative work.

[0155] In some embodiments, a smart question-answering method for collaborative work can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the smart question-answering method for collaborative work described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute a smart question-answering method for collaborative work by any other suitable means (e.g., by means of firmware).

[0156] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0158] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0159] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0160] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0161] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0162] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0163] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An intelligent question-answering method for collaborative office work, characterized in that, include: Receive problem description information input by the target user; The problem description information is processed based on a pre-trained multi-domain knowledge model to obtain the target question-answering result corresponding to the problem description information; The multi-domain knowledge model is determined based on the expert model, the linear layer of the language model, and the expert-shared model corresponding to each domain. The expert model includes a low-rank adaptation matrix, which is used to adjust the parameter quantity after processing the input data. The expert-shared model is used to process all the output first sample data.

2. The method according to claim 1, characterized in that, The multi-domain knowledge model to be trained includes a feature extraction layer to be trained, a feature processing model connected to the output of the feature extraction layer, and the feature processing model includes a gating network, multiple expert models to be trained that communicate with the gating network, a linear layer of the language model to be trained, and a shared expert model to be trained. Training the multi-domain knowledge model to obtain the multi-domain knowledge model includes: For multiple first sample data corresponding to different domains, the input features of the sample input data are extracted based on the feature extraction layer to be trained; wherein, the first sample data includes sample input data and sample output data; Based on the gating network, at least one processable expert model corresponding to the input feature is determined from multiple expert models to be trained, and a first feature corresponding to the input feature is output based on the at least one processable expert model. Based on the expert model to be trained, a second feature corresponding to the input feature is determined; and based on the expert sharing model, a third feature corresponding to the input feature is determined. Based on the first feature, the second feature, the third feature, and the sample output data, at least one of the expert models, the linear layer of the language model to be trained, and the model parameters in the expert shared model to be trained are modified. The model obtained when the loss function in the multi-domain knowledge model to be trained converges is taken as the multi-domain knowledge model. The loss function includes the result loss function and the loss function corresponding to the label of the sample input data.

3. The method according to claim 2, characterized in that, The expert model to be trained includes a low-rank adaptation matrix, and the model parameters of the expert model are adjusted by adjusting the matrix weights of the low-rank adaptation matrix.

4. The method according to claim 2, characterized in that, The step of correcting the model parameters in at least one usable expert model, the linear layer of the language model to be trained, and the expert shared model to be trained, based on the first feature, the second feature, the third feature, and the sample output data, includes: The first feature, the second feature, and the third feature are weighted according to the weight value corresponding to each feature, so as to output the predicted output data corresponding to the weighted features; Based on the predicted output data and the sample output data, a sample loss value is determined, and the model parameters in at least one of the expert models, the linear layer of the language model to be trained, and the expert shared model to be trained are corrected based on the sample loss value.

5. The method according to claim 2, characterized in that, After obtaining the multi-domain knowledge model, the method further includes: The learning parameters in the multi-domain knowledge model and the model in the linear layer of the language model are optimized. The optimization of the learning parameters in the multi-domain knowledge model and the model in the linear layer of the language model includes: For multiple second sample data corresponding to different domains, with the first learning parameter, the second learning parameter in the multi-domain knowledge model and the parameters in the linear layer of the language model remaining unchanged, the fourth feature corresponding to the sample input data in the second sample data is extracted based on the first learning parameter in the feature extraction layer. The fourth feature is input into a pre-built vector fusion architecture to determine the model fit attribute of the fourth feature relative to at least one expert model based on the evaluation route in the vector fusion architecture. Based on the model adaptation attributes of the at least one expert model and the corresponding linear layer parameters, determine the first adjustment parameter, the second adjustment parameter, and the third adjustment parameter for processing the input features; The learning parameters in the multi-domain knowledge model and the model parameters in the linear layer of the language model are adjusted according to the first adjustment parameter, the second adjustment parameter, and the third adjustment parameter.

6. The method according to claim 5, characterized in that, The step of adjusting the learning parameters in the multi-domain knowledge model and the model parameters in the linear layer of the language model according to the first adjustment parameter and the second adjustment parameter includes: Based on the first adjustment parameter, the second adjustment parameter, the first learning parameter, the second learning parameter, and the parameter dimension corresponding to the first learning parameter, the adjusted first learning parameter and the second learning parameter are determined. The optimized model parameters are determined based on the third adjustment parameter, the first weight parameter and the second weight parameter in the language linear layer.

7. The method according to claim 1, characterized in that, The process of processing the question description information based on a pre-trained multi-domain knowledge model to obtain the target question-answering result corresponding to the question description information includes: The input features corresponding to the problem description information are extracted based on the feature extraction layer in the multi-domain knowledge model, and a target expert model adapted to the input features is determined based on the gating network. The input features are then processed based on the target expert model, the linear layer of the language model, and the expert sharing model to obtain the target features. The input features are determined based on optimized learning parameters, and the model parameters in the linear layer of the language model are optimized parameters. Based on the target features, the target question-and-answer result is determined.

8. The method according to any one of claims 1-7, characterized in that, The problem description information is a description of a problem in an office setting, which includes at least one or more of the following: finance, human resources management, administration, and business operations.

9. An intelligent question-and-answer device for collaborative office work, characterized in that, include: The problem description information receiving module is used to receive problem description information input by the target user; The target question answering result acquisition module is used to process the question description information based on a pre-trained multi-domain knowledge model to obtain the target question answering result corresponding to the question description information; The multi-domain knowledge model is determined based on the expert model, the linear layer of the language model, and the expert-shared model corresponding to each domain. The expert model includes a low-rank adaptation matrix, which is used to adjust the parameter quantity after processing the input data. The expert-shared model is used to process all the output first sample data.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intelligent question-answering method for collaborative office work as described in any one of claims 1-8.

Images