Model training method, question-answering processing method, device and electronic equipment
By automatically selecting word participle feature vectors for adjustment, the problem of insufficient knowledge in specific fields and long fine-tuning time for pre-trained language models is solved, and more efficient model training and adaptability is achieved.
Patent Information
- Application Number
- CN202411289626.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-14
AI Technical Summary
The existing pre-trained language models have limited expertise in specific fields and industries, and the characterization fine-tuning method requires manual setting of intervention positions, resulting in long training time and large computational overhead, and there is redundant characterization intervention operations.
By obtaining multiple labeled training samples in the target field, fine-tuning the neural network model, automatically selecting which word segmentation feature vectors are adjusted, and adjusting them using the first model parameters and the second model parameters to avoid manual repeated debugging and Redundant intervention.
It shortens the time for fine-tuning of the model, reduces the computational complexity and overhead, and improves the adaptability and generalization capabilities of the model.
Smart Images

Figure CN118797022B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer technology and may involve fields such as artificial intelligence and natural language processing. Specifically, the present application relates to a model training method, a question and answer processing method, a device and an electronic device. Background Art
[0002] Pre-trained language models can learn general language representation and knowledge by training on large-scale corpora, but their expertise in specific fields and industries is very limited.
[0003] In order to make the pre-trained language model better suited to downstream tasks, it is usually necessary to further fine-tune the pre-trained language model using the data of downstream tasks. Common fine-tuning methods include Parameter-Efficient Fine-Tuning (PeFT) and Representation Finetuning (ReFT).
[0004] Among them, fine-tuning with ReFT requires manual setting of the intervention position, and debugging the parameters related to the intervention position requires a lot of experiments, which consumes a lot of training time and computing overhead. In addition, this manual setting method may have redundant representation intervention operations. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide a model training method that can effectively shorten the fine-tuning time of the model and reduce the computational overhead of the model, as well as a question-answering processing method, device, and electronic device based on the model training method. To achieve this purpose, the technical solutions provided by the embodiments of the present application are as follows:
[0006] On the one hand, an embodiment of the present application provides a model training method, the method comprising:
[0007] Acquire multiple labeled training samples in the target domain, each of the training samples includes a sample question, and the label of each training sample is a sample answer corresponding to the sample question in the training sample;
[0008] Based on the multiple training samples, a trained target question-answering model is obtained by continuously performing fine-tuning operations on the neural network model;
[0009] The fine-tuning operation includes:
[0010] For each sample question, extract features of the sample question through an encoder of a pre-trained first language model to obtain a first feature vector of each word segment in the sample question, determine each first word segment to be adjusted in each word segment of the sample question according to the first model parameter, and adjust the first feature vector of each first word segment according to the second model parameter to obtain a second feature vector of each adjusted first word segment;
[0011] For each sample question, decoding is performed by the decoder of the first language model based on the target feature vector of the sample question to obtain a predicted answer corresponding to the sample question, wherein the target feature vector of the sample question is obtained based on the second feature vector of each of the first participles in the sample question and the first feature vectors of other participles except each of the first participles;
[0012] According to the predicted answers and labels corresponding to each of the sample questions, a first training loss is determined, and based on the first training loss, the first model parameters and the second model parameters are adjusted to obtain the first model parameters and the second model parameters corresponding to the next fine-tuning operation.
[0013] On the other hand, an embodiment of the present application further provides a model training device, the device comprising:
[0014] A sample acquisition module, used to acquire a plurality of labeled training samples in a target field, each of the training samples including a sample question, and the label of each training sample is a sample answer corresponding to the sample question in the training sample;
[0015] A model fine-tuning module, configured to obtain a trained target question-answering model by continuously performing fine-tuning operations on the neural network model based on the multiple training samples;
[0016] The fine-tuning operation includes:
[0017] For each sample question, extract features of the sample question through an encoder of a pre-trained first language model to obtain a first feature vector of each word segment in the sample question, determine each first word segment to be adjusted in each word segment of the sample question according to the first model parameter, and adjust the first feature vector of each first word segment according to the second model parameter to obtain a second feature vector of each adjusted first word segment;
[0018] For each sample question, decoding is performed by the decoder of the first language model based on the target feature vector of the sample question to obtain a predicted answer corresponding to the sample question, wherein the target feature vector of the sample question is obtained based on the second feature vector of each of the first participles in the sample question and the first feature vectors of other participles except each of the first participles;
[0019] According to the predicted answers and labels corresponding to each of the sample questions, a first training loss is determined, and based on the first training loss, the first model parameters and the second model parameters are adjusted to obtain the first model parameters and the second model parameters corresponding to the next fine-tuning operation.
[0020] Optionally, the encoder includes a plurality of cascaded feature extraction layers; the first model parameters include layer parameters corresponding to at least one feature extraction layer;
[0021] For each sample question, the predicted answer corresponding to the sample question is obtained in the following way:
[0022] The sample question is input into the encoder, and the encoder is used to extract features of the sample question to obtain a first feature vector corresponding to each feature extraction layer for each word segment in the sample question;
[0023] For each of the at least one feature extraction layer, based on the layer parameters corresponding to the feature extraction layer, determine each first participle to be adjusted in each participle of the sample question, use the second model parameter to adjust the first feature vector of each first participle to obtain the second feature vector of each first participle, use the second feature vector of each first participle and the first feature vectors of other participles except the first participle as the target feature vector of the feature extraction layer, and use the target feature vector of the feature extraction layer as the input of the next feature extraction layer;
[0024] For each feature extraction layer except the at least one feature extraction layer, a first feature vector of each word in the sample question is obtained through the feature extraction layer as a target feature vector of the feature extraction layer, and the target feature vector of the feature extraction layer is used as an input of the next feature extraction layer;
[0025] Based on the target feature vector of the last feature extraction layer of the encoder, decoding is performed through the decoder to obtain a predicted answer corresponding to the sample question.
[0026] Optionally, the first model parameter includes a sub-parameter of each word segment in the sample question, and a sub-parameter of a word segment is used to determine whether to adjust the first feature vector of the word segment;
[0027] The model fine-tuning module can be used to:
[0028] Determining a second training loss according to the sum of the sub-parameters of each word segment in each of the sample questions;
[0029] Based on the first training loss and the second training loss, first model parameters and second model parameters are adjusted.
[0030] Optionally, the first model parameter includes a sub-parameter of each word segment in the sample question, and a sub-parameter of a word segment is used to determine whether to adjust the first feature vector of the word segment;
[0031] The model fine-tuning module can be used to:
[0032] Determine the random noise corresponding to each word in the sample question;
[0033] Adjusting the sub-parameters of each of the segmented words based on the random noise corresponding to each of the segmented words to obtain a target adjustment probability of each of the segmented words;
[0034] According to the target adjustment probability of each of the segmented words, each first segmented word to be adjusted is determined.
[0035] Optionally, the first model parameter includes a preset number of word segmentation sub-parameters;
[0036] The model fine-tuning module can be used to:
[0037] Determining a first number of participles in the sample question;
[0038] If the first number is equal to the preset number, the sample problem is used as a sample problem to be processed;
[0039] If the first number is less than the preset number, the sample question is padded with preset characters, and the padded sample question is used as the sample question to be processed, wherein the number of segmentations in the padded sample question is the preset number;
[0040] The encoder is used to extract features from the sample question to be processed, and obtains a first feature vector of each word in the sample question to be processed.
[0041] Optionally, the first model parameter is determined by:
[0042] According to at least one of the content and part of speech of each segmentation in the plurality of sample questions, counting the importance of the segmentation at each position;
[0043] Determining each first position according to the importance of the segmentation at each position in each of the sample questions;
[0044] A first model parameter is determined according to the sub-parameters of the word segments at each of the first positions.
[0045] Optionally, the model fine-tuning module can also be used for:
[0046] Determine the field type of the target field;
[0047] If the domain type is the first type, the at least one feature extraction layer includes each feature extraction layer in the plurality of cascaded feature extraction layers;
[0048] If the domain type is not the first type, the at least one feature extraction layer includes some feature extraction layers in the plurality of cascaded feature extraction layers.
[0049] Optionally, the model fine-tuning module can be used to:
[0050] Constructing a corresponding prompt word instruction based on the sample question, wherein the prompt word instruction is used to instruct the neural network model to output a corresponding answer based on the sample question;
[0051] The encoder of the first language model performs feature extraction on the prompt word instruction.
[0052] On the other hand, the embodiment of the present application also provides a question-answering processing method, the method comprising:
[0053] Get the questions to be answered in the target area;
[0054] The answer corresponding to the question to be answered is obtained through the trained target question-answering model; wherein the target question-answering model is trained using the model training method provided in any optional embodiment of the present application.
[0055] On the other hand, the embodiment of the present application further provides a question-answer processing device, the device comprising:
[0056] A question acquisition module is used to obtain questions to be answered in the target field;
[0057] A reply module is used to obtain a reply corresponding to the question to be answered through a trained target question-answering model; wherein the target question-answering model is trained using the model training method provided in any optional embodiment of the present application.
[0058] On the other hand, an embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the model training method provided in any optional embodiment of the present application, or to implement the question and answer processing method provided in any optional embodiment of the present application.
[0059] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the model training method provided in any optional embodiment of the present application, or implements the question and answer processing method provided in any optional embodiment of the present application.
[0060] On the other hand, an embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the model training method provided in any optional embodiment of the present application, or implements the question and answer processing method provided in any optional embodiment of the present application.
[0061] The beneficial effects of the technical solution provided by the embodiment of the present application are as follows:
[0062] The model training method provided in the embodiment of the present application, in the process of continuously fine-tuning the neural network model based on multiple labeled training samples in the target field, determines each first participle to be adjusted from each participle of the training sample based on the first model parameter, and adjusts the first feature vector of each first participle according to the second model parameter. By introducing the first model parameter and the second model parameter, the model can automatically select which first feature vectors of the participle to adjust without the need for repeated manual debugging, avoiding the introduction of redundant representation intervention operations, and at the same time can shorten the training time of model fine-tuning, effectively reduce the computational complexity of the model, and reduce the computational overhead of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.
[0064] Figure 1 A schematic diagram of the structure of the implementation environment of the model training method provided in the embodiment of the present application;
[0065] Figure 2 A flowchart of a model training method provided in an embodiment of the present application;
[0066] Figure 3 A schematic diagram of an interface for a medical consultation session in an online medical consultation platform provided in an embodiment of the present application;
[0067] Figure 4 A schematic diagram of the hierarchical structure of the encoder of the first language model provided in an embodiment of the present application;
[0068] Figure 5 A flowchart of a question-and-answer processing method provided in an embodiment of the present application;
[0069] Figure 6 A schematic diagram of the states of each intervention position in the target question-answering model provided in an embodiment of the present application;
[0070] Figure 7 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;
[0071] Figure 8A schematic diagram of the structure of a question-answer processing device provided in an embodiment of the present application;
[0072] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0073] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.
[0074] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B". When describing multiple (two or more) items, if the relationship between the multiple items is not clearly defined, the multiple items may refer to one, multiple or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A including A1 or A2 or A3, and can also be implemented as parameter A including at least two of the three items A1, A2, A3.
[0075] In order to better understand and illustrate the method provided in the embodiments of the present application, some technical terms involved in the embodiments of the present application are first explained and illustrated below.
[0076] Pre-trained language model: refers to a language model that does not require manual annotation and is trained on a large-scale corpus. It can learn common language representations and knowledge from a large-scale corpus and apply them to downstream tasks such as natural language understanding, generation, translation, and classification.
[0077] Transformer: A network structure based on the self-attention mechanism that can effectively process sequence data, such as text data. It consists of two parts: an encoder and a decoder. The encoder is used to encode the input sequence into a feature representation, and the decoder is used to decode the feature representation into an output sequence. Transformer is widely used in natural language processing tasks, such as machine translation, text classification, and text generation.
[0078] Self-attention mechanism: It can effectively learn the weight of each element in the input sequence. Specifically, the weight of each element is calculated by calculating the similarity between each element and other elements in the sequence, so that when processing sequence data, it can focus on the elements related to the current task.
[0079] Parameter-Efficient Fine-Tuning (PEFT) is a type of fine-tuning technology for pre-trained models. Full fine-tuning methods usually require updating all parameters of the pre-trained model, which consumes a lot of computing resources and storage space. However, the parameter-efficient fine-tuning method can achieve similar effects to full fine-tuning by only adjusting part of the model's parameters or introducing a small number of additional parameters, which can significantly reduce computing and storage overhead.
[0080] Low-Rank Adaptation (LoRA): A model adaptation method that introduces a low-rank matrix into the parameters of a large model and trains the parameters of the low-rank matrix so that the model can better adapt to specific tasks, thereby improving the performance and generalization ability of the model.
[0081] Model weights: The parameters of the model at each layer.
[0082] Model representation: The hidden feature representation of the input after calculating the weights of each layer of the model.
[0083] Representation fine-tuning: Representation fine-tuning refers to updating the representation of the model.
[0084] Representation intervention: Representation intervention refers to the intervention of the representation of the input prompt word at a specific layer of the model during the forward propagation process, and the intervention of the representation of a certain layer of the model will also affect the representation calculation of subsequent layers.
[0085] Intervention location: The intervention location refers to which tokens in the input prompt word are intervened and at which layers of the model the intervention is performed.
[0086] Adaptive representation intervention: During the representation intervention process, the model automatically selects the intervention location so that the intervention location can also participate in the model training process instead of being a pre-set hyperparameter. This can effectively reduce the computational complexity of the model and reduce redundant representation intervention operations.
[0087] Gumbel-Softmax: Used to introduce differentiable approximations in discrete choices. This is very useful for tasks that require discrete sampling in neural networks (such as classification tasks or generating discrete data) without disrupting gradient calculations. Specifically, in many machine learning tasks, it is necessary to sample from a discrete distribution (for example, in reinforcement learning or generative models, it may be necessary to select from a set of discrete actions or vocabularies), but sampling directly from a discrete distribution is discontinuous, which means that model parameters cannot be updated through backpropagation. Gumbel-Softmax provides a differentiable approximation, combined with Gumbel noise to approximate discrete one-hot vectors, so that discrete choices can be made while keeping backpropagation feasible.
[0088] In order to better adapt to downstream tasks, it is usually necessary to further fine-tune the pre-trained language model using the data of downstream tasks. Common fine-tuning methods include the parameter efficient fine-tuning method PeFT and the representation fine-tuning method ReFT.
[0089] When the representation fine-tuning method ReFT is used to fine-tune the pre-trained language model, ReFT does not directly update the model weights of the language model, but intervenes through representation (also called feature representation, feature vector) to adapt the model to downstream tasks. Representation intervention refers to the intervention of the representation of the input prompt word at a specific layer of the model during the forward propagation process. During the representation intervention process, it is necessary to set the intervention position, that is, which tokens in the input prompt word are intervened, and which layers of the model are intervened. Debugging these setting parameters requires a lot of experiments, which consumes a lot of training time and computing overhead. In addition, this manual setting method may have redundant representation intervention operations.
[0090] Based on the above-mentioned problems, the embodiments of the present application provide a model training method, device, electronic device and storage medium. In the process of fine-tuning the pre-trained language model, by introducing a first model parameter, the model automatically selects the intervention position, that is, selects which word segmentation feature vectors to adjust, so that the intervention position can also participate in the model training process without manually setting the intervention position and performing multiple debugging, which effectively reduces the complexity of model calculation and reduces redundant representation intervention operations.
[0091] It should be noted that in the optional embodiments of the present application, the data related to the object information involved (such as the user's medical consultation information, disease detection report, etc.), when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.
[0092] Figure 1 A structural schematic diagram of an implementation environment applicable to an embodiment of the present application is shown, and the implementation environment may include a terminal device 10, an application server 20 and a training server 30, wherein the application server 20 may be a background server that provides a question-and-answer service in a target field. The terminal device 10 may be an electronic device of the object asking questions, and the terminal device 10 may communicate with the application server 20 through a network, send questions to the application server 20, and receive replies returned by the application server 20 for display. Optionally, the target field may be the medical field, the financial field, etc., and the embodiment of the present application does not limit this. When the target field is the medical field, the application server 20 is a background server that provides medical question-and-answer services.
[0093] The training server 30 can fine-tune the neural network model by executing the model training method provided in the embodiment of the present application to obtain a trained target question-answering model. Optionally, the application server 20 and the training server 30 can communicate through a network. After obtaining the trained target question-answering model through the training server 30, the trained target question-answering model can be deployed on the application server 20, and the application server 20 executes the question-answering processing method provided in the embodiment of the present application, and responds to the questions sent by the terminal device 10 by calling the trained target question-answering model.
[0094] Among them, the above-mentioned servers (application server 20, training server 30) can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide cloud computing services. The above-mentioned terminal devices (also referred to as user terminals) can be smart phones, tablet computers, laptop computers, desktop computers, intelligent voice interaction devices (such as smart speakers), wearable electronic devices (such as smart watches), vehicle-mounted terminals, smart home appliances (such as smart TVs), AR / VR devices, etc., but are not limited to these. The terminals and servers can be connected to each other via wired or wireless networks, and this application does not limit this.
[0095] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including HyperText Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.
[0096] The following describes several embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.
[0097] Figure 2 A flow chart of a model training method provided in an embodiment of the present application is shown. The method can be executed by any electronic device, for example, Figure 1For the convenience of description, the following will take the training server as the execution subject as an example. Figure 2 As shown, the method includes the following steps S110 to S120.
[0098] Step S110: Acquire multiple labeled training samples in the target domain.
[0099] The target domain is the domain corresponding to the downstream task being performed, each training sample includes a sample question, and the label of each training sample is the sample answer corresponding to the sample question in the training sample. For example, when training a target question-answering model for consultation, the target domain is the medical field, the sample question can be the patient's consultation question, and the sample answer can be the doctor's answer.
[0100] As an optional implementation, the sample questions and sample answers corresponding to the sample questions in the training samples can be selected from the historical conversation tasks in the target field, and the questions asked by users in the historical conversation tasks are used as sample questions, and the answers to the questions in the historical conversation tasks are used as sample answers. For example, Figure 3 As shown, Figure 3 For a real historical consultation conversation on an online consultation platform, the patient’s question “Red round patches appeared on the inside of my elbow. They became hot when exposed to hot water and there were dandruff after scratching. What could be the problem?” can be used as a sample question, and the doctor’s answer “Contact dermatitis, eczema, and urticaria may all cause the above symptoms. If your symptoms persist or worsen, it is recommended that you seek medical attention offline and further determine the cause by examining skin samples” can be used as a sample answer corresponding to the sample question.
[0101] Step S120: Based on multiple training samples, the neural network model is continuously fine-tuned to obtain a trained target question-answering model.
[0102] Among them, the neural network model includes a pre-trained first language model and initialized first model parameters and second model parameters. The first language model is a large language model LLM. The first model parameters are used to indicate which word segmentation feature vectors in the sample question are to be intervened, and the second model parameters are used to indicate how to intervene in the word segmentation feature vectors. When fine-tuning the neural network model, the model parameters of the pre-trained first language model are kept unchanged, and only the first model parameters and the second model parameters to be trained are adjusted. This application does not make specific restrictions on the first language model, as long as it is a large language model based on the transformer architecture.
[0103] Optionally, the first model parameter includes sub-parameters of each word segmentation in the sample question, and the sub-parameter of a word segmentation is used to determine whether to adjust (intervene) the first feature vector of the word segmentation, the first feature vector is extracted by the first language model, and the second model parameter is used to indicate the adjustment method of the first feature vector of the first word segmentation, that is, how to adjust the first feature vector of the first word segmentation, and the first word segmentation is the word segmentation whose first feature vector needs to be adjusted among the word segmentations. Optionally, the sub-parameter of a word segmentation can be the probability of selecting to adjust the first feature vector of the word segmentation.
[0104] The fine-tuning operation includes the following steps S210 to S230:
[0105] Step S210: For each sample question, extract features of the sample question through the encoder of the pre-trained first language model to obtain the first feature vector of each word segment in the sample question, determine the first word segment to be adjusted in the word segment of the sample question according to the first model parameters, and adjust the first feature vector of each first word segment according to the second model parameters to obtain the adjusted second feature vector of each first word segment.
[0106] Since the feature representation learned by the pre-trained first language model encodes rich semantic information, the model training method provided in the embodiment of the present application intervenes in some feature representations extracted by the pre-trained language model to achieve more efficient model adaptation. Therefore, after the first feature vector of each word segment in the sample question is extracted by the encoder, each extracted first feature vector can be selectively fine-tuned.
[0107] Specifically, the first word segmentations to be adjusted in the word segmentations of the sample question are determined according to the first model parameters corresponding to the current fine-tuning operation, and then the first feature vectors of the first word segmentations are adjusted according to the second model parameters corresponding to the current fine-tuning operation to obtain the adjusted second feature vectors of the first word segmentations. If the current fine-tuning operation is the first fine-tuning operation, the first model parameters and the second model parameters corresponding to the current fine-tuning operation are the initialized first model parameters and the second model parameters, and if the current fine-tuning operation is not the first fine-tuning operation, the first model parameters and the second model parameters corresponding to the current fine-tuning operation are the first model parameters and the second model parameters adjusted based on the training loss after the last fine-tuning operation.
[0108] Optionally, in the process of characterization fine-tuning, the selection probability of the first feature vector of each segmentation can be expressed in a differentiable approximate manner by introducing random noise, so that the first model parameter in the neural network model can be updated by back propagation. Specifically, the random noise corresponding to each segmentation in the sample problem is determined, and the sub-parameters of each segmentation are adjusted based on the random noise corresponding to each segmentation to obtain the target adjustment probability of each segmentation, and each first segmentation is determined according to the target adjustment probability of each segmentation. As an optional implementation method, the first segmentations that need to be characterized can be selected from each segmentation by comparing the target adjustment probability with a preset adjustment probability threshold. For any segmentation, if the target adjustment probability of the segmentation is greater than the preset adjustment probability threshold, the segmentation is used as the first segmentation, that is, the first feature vector of the segmentation needs to be adjusted. If the target adjustment probability of the segmentation is less than or equal to the preset adjustment probability threshold, there is no need to adjust the first feature vector of the segmentation.
[0109] Optionally, for each sample question, a corresponding prompt word instruction is constructed based on the sample question, wherein the prompt word instruction corresponding to the sample question is used to instruct the neural network model to output a corresponding answer based on the sample question. The prompt word instruction is feature extracted by the encoder of the first language model to obtain the first feature vector of each word segment in the prompt word instruction. For example, assuming that the sample question is "A red circular patch appears on the inside of my elbow, which becomes hot when exposed to hot water and produces dandruff after scratching. What could be the problem?", the constructed prompt word instruction can be: I hope you can play the role of a doctor and provide a diagnosis and treatment suggestion based on the patient's description of the condition "A red circular patch appears on the inside of my elbow, which becomes hot when exposed to hot water and produces dandruff after scratching. What could be the problem?" When providing advice, you also need to consider the patient's age, lifestyle, and medical history to diagnose the most likely cause of their symptoms.
[0110] It should be noted that since the length of the sample questions is not fixed, the number of word segmentations contained may not be the same, and the model training needs to learn at which position the first feature vector of the word segmentation needs to be fine-tuned.
[0111] Therefore, in one embodiment of the present application, the length of the model input data can be constrained for consistency. Before the sample question is subjected to feature extraction by the encoder of the first language model, for each sample question, the first number of word segments in the sample question is determined. If the first number is equal to the preset number, the sample question is used as the sample question to be processed; if the first number is less than the preset number, the sample question is padded with preset characters, and the padded sample question is used as the sample question to be processed, wherein the number of word segments in the padded sample question is the preset number, and the preset number is the maximum number of word segments in the set sample question. Optionally, the preset character can be a pad_token (a padding marker that supplements the length of the input sequence).
[0112] In another embodiment provided by the present application, the position range that needs intervention in the input data can also be constrained, that is, the position that may need intervention in the sample problem can be specified. The importance of the segmentation at each position in the sample problem can be counted according to at least one of the content and part of speech of each segmentation in multiple sample problems, and each first position can be determined according to the importance of the segmentation at each position in each sample problem. The first model parameter is determined according to the sub-parameters of the segmentation at each first position, so as to learn through model training which positions of the segmentation first feature vectors in each first position need to be adjusted. For example, when the statistics show that the importance of the first p segmentations and the last q segmentations in the sample problem is high, the first p segmentation positions and the last q segmentation positions in the sample problem are set as target positions, and the corresponding sub-parameters are set for the segmentations at the target positions.
[0113] Step S220: For each sample question, based on the target feature vector of the sample question, decoding is performed through the decoder of the first language model to obtain a predicted answer corresponding to the sample question.
[0114] The target feature vector of the sample question is obtained based on the adjusted second feature vectors of each first participle in the sample question and the first feature vectors of other participles except the first participles.
[0115] Step S230: Determine the first training loss according to the predicted answers and labels corresponding to each sample question, and adjust the first model parameters and the second model parameters based on the first training loss to obtain the first model parameters and the second model parameters corresponding to the next fine-tuning operation.
[0116] Optionally, when the first model parameters and the second model parameters are adjusted based on the training loss of the model, in order to further control the sparsity of the intervention position (i.e., the first feature vector extracted by which feature extraction layer of which word in the model is intervened), the second training loss can be determined according to the sum of the sub-parameters of each word in each sample problem, and the first model parameters and the second model parameters can be adjusted based on the first training loss and the second training loss.
[0117] It can be understood that in actual implementation, after each fine-tuning operation is completed, it can be determined whether the model at this time meets the preset training end conditions. If the model at this time meets the preset training end conditions, the model can be used as the trained target question and answer model. If it does not meet the preset training end conditions, the first model parameters and the second model parameters continue to be adjusted.
[0118] Among them, the training end conditions of the above-mentioned model fine-tuning operation and the model's loss function can be configured according to needs. For example, the training end conditions may include but are not limited to the number of training times reaching a preset number, the loss function convergence (such as the model's training loss is less than a preset value, or the training losses for multiple consecutive times are less than a preset value, etc.), the model's test indicators meet the preset indicators, etc. The first training loss of the model represents the deviation between the actual sample answer (label) of each sample question and the predicted answer predicted by the model. Through the labeled training samples and the model's training loss function, the model is supervised by using the gradient descent algorithm, so that the sample answers predicted by the model can continuously approach the actual sample answers to the sample questions, thereby obtaining a trained target question-answering model that meets the needs of actual applications.
[0119] based on Figure 2 The model training method shown freezes the model parameters of the pre-trained first language model during the fine-tuning stage of model training. By introducing the first model parameters, the intervention positions (i.e., which word first feature vectors are adjusted) can participate in the model training process. The model automatically selects the intervention positions, thereby effectively reducing the computational complexity of the model while maintaining the model performance, avoiding redundant representation intervention operations introduced by manual settings, and improving the generalization ability of the model.
[0120] In one or more embodiments of the present application, the encoder of the pre-trained first language model includes multiple cascaded feature extraction layers, and each feature extraction layer is used to extract features of each word in the sample question in turn to obtain the first feature vector of each word in the sample question corresponding to the feature extraction layer. Therefore, during the model training process, it is also necessary to learn at which feature extraction layer the first feature vector of each word is extracted to intervene.
[0121] Among them, the first model parameters include layer parameters corresponding to at least one feature extraction layer, the layer parameters corresponding to a feature extraction layer include sub-parameters of each participle corresponding to the feature extraction layer, and the sub-parameters of a participle corresponding to a feature extraction layer indicate whether to adjust the first feature vector extracted by the first feature extraction layer for the participle.
[0122] For each sample question, when the predicted answer corresponding to the sample question is generated by the neural network model, the sample question is input into the encoder of the first language model, and the encoder performs feature extraction on the sample question to obtain the first feature vector of each feature extraction layer corresponding to each word in the sample question, and based on the target feature vector of the last feature extraction layer of the encoder, decoding is performed through the decoder of the first language model to obtain the predicted answer corresponding to the sample question.
[0123] Among them, for each feature extraction layer in at least one feature extraction layer, based on the layer parameters corresponding to the feature extraction layer, the first participles to be adjusted in the participles of the sample question (corresponding to the feature extraction layer) are determined, the first feature vectors of the first participles of the feature extraction layer are adjusted using the second model parameters to obtain the second feature vectors of the first participles of the feature extraction layer, the second feature vectors of the first participles of the feature extraction layer and the first feature vectors of the other participles of the feature extraction layer except the first participles are used as the target feature vector of the feature extraction layer, and the target feature vector of the feature extraction layer is used as the input of the next feature extraction layer.
[0124] For each feature extraction layer except at least one feature extraction layer, the first feature vector of each word in the sample question is extracted through the feature extraction layer as the target feature vector of the feature extraction layer, and the target feature vector of the feature extraction layer is used as the input of the next feature extraction layer.
[0125] Optionally, for each feature extraction layer in at least one feature extraction layer, when determining each first participle corresponding to the feature extraction layer based on the layer parameters corresponding to the feature extraction layer, the sub-parameters of each participle corresponding to the feature extraction layer can be adjusted by introducing random noise, so as to determine each first participle corresponding to the feature extraction layer based on the target adjustment probability corresponding to each participle after adjustment. Specifically, for each feature extraction layer and each participle in at least one feature extraction layer, the random noise corresponding to the feature extraction layer of the participle is determined, and based on the random noise corresponding to the feature extraction layer of the participle, the sub-parameter corresponding to the feature extraction layer of the participle is adjusted to obtain the target adjustment probability of the participle corresponding to the feature extraction layer, and the first participle corresponding to the feature extraction layer is determined according to the target adjustment probability of each participle corresponding to the feature extraction layer.
[0126] For example, Figure 4 A schematic diagram of the hierarchical structure of the encoder of the first language model provided in an embodiment of the present application, assuming that the encoder of the pre-trained first language model includes L feature extraction layers, and the figure represents multiple stacked feature extraction layers from top to bottom The first model parameter includes the layer parameters corresponding to each feature extraction layer. The layer parameters corresponding to a feature extraction layer include the sub-parameters of each word segmentation in the feature extraction layer. The number of tokens in the prompt word instruction constructed based on the sample question is T. The figure shows the input prompt word sequence from left to right. , Represents the intervention position, which is used to locate the first feature vector extracted by the i-th feature extraction layer for the j-th token in the prompt word (not shown in the figure), the jth token corresponds to the sub-parameter of the i-th feature extraction layer, which is the intervention position Among them, , .
[0127] For any intervention position , according to the intervention location Sub-parameters and the intervention location The corresponding random noise , the intervention position is obtained by normalization The corresponding target adjustment probability (target selection probability), according to the intervention position The corresponding target adjustment probability , determine the intervention location Among them, each intervention position There are two optional states, selected or not selected. The state set of each intervention position can be expressed as , Indicates being selected. Indicates that it is not selected. Correspondingly, It can be expressed as or .
[0128] Exemplarily, the introduced random noise may be Gumbel noise, and the Gumbel-Softmax function may be used to calculate the target adjustment probability of each word segmentation corresponding to each feature extraction layer:
[0129]
[0130]
[0131]
[0132] in, Indicates intervention location The corresponding target adjustment probability, that is, the probability that the first feature vector of the jth word segmentation extracted by the i-th feature extraction layer is adjusted, reflects the intervention position The importance of the corresponding first eigenvector, Uniform distribution . Intervention location Corresponding Gumbel noise, T represents the maximum length of the sample problem, Indicates intervention location The sub-parameters of , that is, the j-th word segmentation corresponds to the sub-parameters of the i-th feature extraction layer, represents the temperature hyperparameter, and ,when The smaller ( ), the smoother the softmax function is, the closer it is to the argmax function. The value of will become 1 or 0, where Indicates that the jth word extracted by the i-th feature extraction layer (intervention position ) Make adjustments; Indicates that there is no need to extract the jth word (intervention position) through the i-th feature extraction layer ) Make adjustments.
[0133] For each feature extraction layer i, according to each intervention position in the feature extraction layer i ( - ), determine the target adjustment probability corresponding to each intervention position in the feature extraction layer i, determine each first position whose state is "selected", use the second model parameter to fine-tune the first feature vector corresponding to each first position, and obtain the adjusted second feature vector corresponding to each first position. Among them, the word segmentation corresponding to the first position is the first word segmentation. The adjusted second feature vectors corresponding to each first position in the feature extraction layer i and the first feature vectors corresponding to each other position (non-first position) in the feature extraction layer i are used to determine the target feature vector of the feature extraction layer i, and the target feature vector of the feature extraction layer i is used as the input of the next feature extraction layer i+1.
[0134] Exemplarily, when the second model parameters are used to fine-tune the first eigenvectors corresponding to the first positions, the corresponding calculation formula is as follows:
[0135]
[0136] Among them, the trainable second model parameters are , Represents the jth word of the sample question Through the model The first feature vector extracted by the feature extraction layer, Represents the dimension of the feature vector. represents the second eigenvector of the output after intervention (fine-tuning), and . Representation is a low-rank projection matrix with orthogonal rows, and , represents the size of the rank, that is, the dimension of the subspace to be intervened, and , that is, the dimension of the subspace to be intervened is less than or equal to the dimension of the eigenvector. represents the projection matrix, and , represents the bias vector, and Among them, the rank of R in the second model parameter is and the dimension of the feature vector Can be set as needed.
[0137] Optionally, when determining the training loss of the model, a second training loss may be determined based on the sum of the target adjustment probabilities corresponding to each intervention position, and the first model parameters and the second model parameters may be adjusted based on the first training loss and the second training loss.
[0138] For example, the training loss of the model is as follows:
[0139]
[0140]
[0141] in, Loss function representing the first training loss. represents the loss function of the second training loss, used to control the number of interventions, is the second training loss The weight hyperparameter can be set as needed. middle, Represents a second trainable model parameter. Represents the training dataset. Indicates that in the known input sample problem , and the output prediction answer In the case of tokens, the predicted answer The token is The probability of t is y, and t represents the length of the sequence of predicted responses y. During the training process, the weights of the pre-trained first language model are kept unchanged (i.e., the model parameters are unchanged), and only the second model parameters are trained. and the first model parameter The loss function of the second training loss is middle, represents the L1 norm, which can control the sparsity of the intervention position (i.e., the first eigenvector for fine-tuning).
[0142] In the embodiment of the present application, by introducing the second training loss, the number / coefficient degree of the intervention positions can be constrained, thereby controlling the complexity of the model calculation.
[0143] In the embodiment of the present application, it is considered that different fields have different requirements for the accuracy of replies. For example, the replies provided in the medical consultation conversation need to be professional, reliable, and rigorous, and the accuracy requirements for the question-answering model are relatively high, while the accuracy requirements for the question-answering model in the general chat scenario are relatively low. Therefore, in order to further reduce the computational overhead of model training, when training the target question-answering model applied to the target field, the field type of the target field can be determined first. If the field type of the target field is the first type, at least one feature extraction layer is set to include each feature extraction layer in a plurality of cascaded feature extraction layers, that is, the first model parameter includes the layer parameters corresponding to each feature extraction layer, the parameter amount of the model parameters to be trained is relatively large, and the accuracy of the trained target question-answering model is higher. If the field type of the target field is not the first type, at least one feature extraction layer is set to include some feature extraction layers in a plurality of cascaded feature extraction layers, that is, the first model parameter includes the layer parameters corresponding to some feature extraction layers, the parameters of the model parameters to be trained are relatively small, and the accuracy of the trained target question-answering model is reduced. Among them, the first type is the type of field with high accuracy requirements for the question-answering model.
[0144] Based on the model training method provided in the embodiment of the present application, while ensuring that the model parameters of the pre-trained first language model remain unchanged, by introducing a first model parameter for indicating which word segmentations of the sample question are extracted at which feature extraction layers to adjust the first feature vectors, during the model fine-tuning process, by learning the importance of the first feature vector corresponding to each intervention position, the first feature vectors with lower importance can be filtered out, so that the model can adaptively select the intervention position.
[0145] It can be understood that the model training method provided in the embodiment of the present application can be used to fine-tune the large language model as long as it is a downstream task performed based on the large language model, and is not limited to fine-tuning the model in question-answering tasks.
[0146] The present application also provides a question-answer processing method, which can be executed by any electronic device, for example, Figure 1 The application server or terminal device in the embodiment is executed for the purpose of description. The following description will be made with the application server as the execution subject. Figure 5 As shown, the method includes the following steps S310 to S320.
[0147] Step S310: Obtain unanswered questions in the target field.
[0148] Step S320: Obtain the answer corresponding to the question to be answered through the trained target question-answering model.
[0149] The target question-answering model can be obtained by training the training server using the method of steps S110 to S120. For details, please refer to the above content, and this application will not repeat them here. After obtaining the target question-answering model trained in the target field, the training server can deploy the target question-answering model to the application server.
[0150] In the embodiment of the present application, after receiving the unanswered question in the target field sent by the user terminal, the application server obtains the answer corresponding to the unanswered question through the trained target question-answering model, and sends the answer to the user terminal for display. The trained target question-answering model includes a pre-trained first language model, and trained first model parameters and second model parameters.
[0151] Optionally, the first model parameter includes sub-parameters of each word segment corresponding to each feature extraction layer, that is, sub-parameters of each intervention position, and the state of each intervention position is determined according to the sub-parameters of each intervention position and a preset adjustment probability threshold. For example, assuming that the sub-parameter of a certain intervention position is 0.8 and the preset adjustment probability threshold is 0.7, since the sub-parameter of the intervention position is greater than the preset adjustment probability threshold, the state of the intervention position is 1, indicating "selected", and the first feature vector corresponding to the intervention position needs to be adjusted. If the sub-parameter of the intervention position is 0.6 and the sub-parameter of the intervention position is less than the preset adjustment probability threshold, the state of the intervention position is 0, indicating "not selected", and there is no need to adjust the first feature vector corresponding to the intervention position.
[0152] For example, Figure 6 As shown in the figure, the encoder of the trained target question answering model includes L feature extraction layers, which are , the length of the input question to be answered is T, that is, the number of tokens contained is T, which are The values in the boxes in the figure indicate the status of the intervention position. The 1 in the orange box indicates that the first eigenvector of the right intervention position will be adjusted, and the 0 in the gray box indicates that the first eigenvector of the right intervention position will not be adjusted.
[0153] For the intervention position corresponding to the orange box, the trained second model parameters are used to adjust the first eigenvector corresponding to the intervention position to obtain the adjusted second eigenvector corresponding to the intervention position, and the second eigenvector corresponding to the intervention position is used as the input of the next feature extraction layer; for the intervention position corresponding to the gray box, the first eigenvector corresponding to the intervention position is directly used as the input of the next feature extraction layer.
[0154] After learning the selection state of each intervention position, the formula characterizing the intervention can be expressed as:
[0155]
[0156] in, Indicates intervention location is selected, then the intervention position The corresponding first eigenvector Conduct interventions; Indicates intervention location Not selected, no intervention will be made to the position The corresponding first eigenvector to intervene.
[0157] based on Figure 5 In the question-answering processing method shown, the target question-answering model used keeps the model parameters of the pre-trained first language model unchanged during the training process, introduces the first model parameters, so that the model can automatically select which word segmentation first feature vectors to adjust, avoiding the introduction of redundant representation intervention operations. The target question-answering model trained by this training method can obtain more accurate answers.
[0158] The question and answer processing method provided in the embodiment of the present application can be applied to question and answer scenarios in various application fields, including but not limited to medical consultation scenarios in the medical field, customer service question and answer scenarios in the e-commerce field, and the like.
[0159] Taking the online medical consultation scenario as an example, when users buy medicines online, they usually need to obtain a medical condition detection report through the online medical consultation platform. Users can send problem description information (including the patient's condition description information and medication information) to the corresponding application server through the online medical consultation platform of the user terminal, such as "Hello, a red circular patch appeared on the inside of my elbow, which becomes hot when exposed to hot water and has dandruff after scratching. I was prescribed XX ointment during an offline visit before. Please issue a medical condition detection report for me." The application server can generate the patient's medical condition detection report based on the received problem description information and the trained target question-answering model and send it to the user terminal so that the user can use the medical condition detection report to purchase the corresponding medicine.
[0160] Taking the customer service question-and-answer scenario in the e-commerce field as an example, users can consult online customer service for product-related information when shopping online. Users can enter question description information in the customer service chat interface of the e-commerce platform, such as "I ordered product A, what is the size of the product?", and send it to the e-commerce server through the terminal. The e-commerce server can generate the corresponding answer "The size of product A is 50×40×70" based on the received question description information through the trained target question-and-answer model, and send the generated answer to the user terminal for display in the customer service chat interface.
[0161] Based on the same principle as the model training method provided in the embodiment of the present application, the embodiment of the present application provides a model training device, such as Figure 7 As shown, the model training device 400 may include: a sample acquisition module 410 and a model fine-tuning module 420.
[0162] The sample acquisition module 410 is used to acquire a plurality of labeled training samples in the target domain, each of the training samples includes a sample question, and the label of each training sample is a sample answer corresponding to the sample question in the training sample;
[0163] A model fine-tuning module 420, configured to obtain a trained target question-answering model by continuously performing fine-tuning operations on the neural network model based on the multiple training samples;
[0164] The fine-tuning operation includes:
[0165] For each sample question, extract features of the sample question through an encoder of a pre-trained first language model to obtain a first feature vector of each word segment in the sample question, determine each first word segment to be adjusted in each word segment of the sample question according to the first model parameter, and adjust the first feature vector of each first word segment according to the second model parameter to obtain a second feature vector of each adjusted first word segment;
[0166] For each sample question, decoding is performed by the decoder of the first language model based on the target feature vector of the sample question to obtain a predicted answer corresponding to the sample question, wherein the target feature vector of the sample question is obtained based on the second feature vector of each of the first participles in the sample question and the first feature vectors of other participles except each of the first participles;
[0167] According to the predicted answers and labels corresponding to each of the sample questions, a first training loss is determined, and based on the first training loss, the first model parameters and the second model parameters are adjusted to obtain the first model parameters and the second model parameters corresponding to the next fine-tuning operation.
[0168] Optionally, the encoder includes a plurality of cascaded feature extraction layers; the first model parameters include layer parameters corresponding to at least one feature extraction layer;
[0169] For each sample question, the predicted answer corresponding to the sample question is obtained in the following way:
[0170] The sample question is input into the encoder, and the encoder is used to extract features of the sample question to obtain a first feature vector corresponding to each feature extraction layer for each word segment in the sample question;
[0171] For each of the at least one feature extraction layer, based on the layer parameters corresponding to the feature extraction layer, determine each first participle to be adjusted in each participle of the sample question, use the second model parameter to adjust the first feature vector of each first participle to obtain the second feature vector of each first participle, use the second feature vector of each first participle and the first feature vectors of other participles except the first participle as the target feature vector of the feature extraction layer, and use the target feature vector of the feature extraction layer as the input of the next feature extraction layer;
[0172] For each feature extraction layer except the at least one feature extraction layer, a first feature vector of each word in the sample question is obtained through the feature extraction layer as a target feature vector of the feature extraction layer, and the target feature vector of the feature extraction layer is used as an input of the next feature extraction layer;
[0173] Based on the target feature vector of the last feature extraction layer of the encoder, decoding is performed through the decoder to obtain a predicted answer corresponding to the sample question.
[0174] Optionally, the first model parameter includes a sub-parameter of each word segment in the sample question, and a sub-parameter of a word segment is used to determine whether to adjust the first feature vector of the word segment;
[0175] The model fine-tuning module 420 can be used to:
[0176] Determining a second training loss according to the sum of the sub-parameters of each word segment in each of the sample questions;
[0177] Based on the first training loss and the second training loss, first model parameters and second model parameters are adjusted.
[0178] Optionally, the first model parameter includes a sub-parameter of each word segment in the sample question, and a sub-parameter of a word segment is used to determine whether to adjust the first feature vector of the word segment;
[0179] The model fine-tuning module 420 can be used to:
[0180] Determine the random noise corresponding to each word in the sample question;
[0181] Adjusting the sub-parameters of each of the segmented words based on the random noise corresponding to each of the segmented words to obtain a target adjustment probability of each of the segmented words;
[0182] According to the target adjustment probability of each of the segmented words, each first segmented word to be adjusted is determined.
[0183] Optionally, the model fine-tuning module 420 may also be used to:
[0184] Determining a first number of participles in the sample question;
[0185] If the first number is equal to the preset number, the sample problem is used as a sample problem to be processed;
[0186] If the first number is less than the preset number, the sample question is filled with preset characters, and the filled sample question is used as the sample question to be processed, wherein the number of segmentations in the filled sample question is the preset number.
[0187] Optionally, the first model parameter is determined by:
[0188] According to at least one of the content and part of speech of each segmentation in the plurality of sample questions, counting the importance of the segmentation at each position;
[0189] Determining each first position according to the importance of the segmentation at each position in each of the sample questions;
[0190] A first model parameter is determined according to the sub-parameters of the word segments at each of the first positions.
[0191] Optionally, the model fine-tuning module 420 may also be used to:
[0192] Determine the field type of the target field;
[0193] If the domain type is the first type, the at least one feature extraction layer includes each feature extraction layer in the plurality of cascaded feature extraction layers;
[0194] If the domain type is not the first type, the at least one feature extraction layer includes some feature extraction layers in the plurality of cascaded feature extraction layers.
[0195] Optionally, the model fine-tuning module 420 may be used to:
[0196] Constructing a corresponding prompt word instruction based on the sample question, wherein the prompt word instruction is used to instruct the neural network model to output a corresponding answer based on the sample question;
[0197] The encoder of the first language model performs feature extraction on the prompt word instruction.
[0198] based on Figure 7 The model training device shown, in the process of continuously fine-tuning the neural network model based on multiple labeled training samples in the target field, determines the first participles to be adjusted from the participles of the training samples based on the first model parameters, and adjusts the first feature vectors of each first participle according to the second model parameters. By introducing the first model parameters and the second model parameters, the model can automatically select which first feature vectors of the participles to adjust without the need for repeated manual debugging, avoiding the introduction of redundant representation intervention operations, and at the same time shortening the training time of model fine-tuning, effectively reducing the computational complexity of the model, and reducing the computational overhead of the model.
[0199] Based on the same principle as the question-answer processing method provided in the embodiment of the present application, the embodiment of the present application provides a question-answer processing device, such as Figure 8 As shown, the question-answer processing device 500 may include: a question acquisition module 510 and a reply module 520, wherein:
[0200] A question acquisition module 510 is used to acquire questions to be answered in the target field;
[0201] The answer module 520 is used to obtain the answer corresponding to the question to be answered through the trained target question-answering model; wherein the target question-answering model is trained using the model training method provided in any optional embodiment of the present application.
[0202] based on Figure 8The question-answering processing device shown, in which the target question-answering model is adopted, keeps the model parameters of the pre-trained first language model unchanged during the training process, and introduces the first model parameters to select the first participle to be adjusted, so that the model can automatically select which participles' first feature vectors to adjust, thereby avoiding the introduction of redundant representation intervention operations. The target question-answering model trained by this training method can obtain more accurate answers.
[0203] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.
[0204] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program stored in the memory, the method in any optional embodiment of the present application can be implemented.
[0205] Fig. 9 FIG. 1 is a schematic diagram showing a structure of an electronic device to which an embodiment of the present invention is applicable. Fig. 9 As shown, the electronic device may be a server or a user terminal, and the electronic device may be used to implement the method provided in any embodiment of the present invention.
[0206] like Fig. 9 As shown in FIG. , the electronic device 2000 may mainly include at least one processor 2001 ( Fig. 9 ), memory 2002, communication module 2003 and input / output interface 2004 and other components. Optionally, the components can be connected and communicated through bus 2005. It should be noted that Fig. 9 The structure of the electronic device 2000 shown in the figure is merely illustrative and does not constitute a limitation on the electronic device to which the method provided in the embodiment of the present application is applicable.
[0207] The memory 2002 may be used to store an operating system and an application program, etc. The application program may include a computer program that implements the method shown in the embodiment of the present invention when called by the processor 2001, and may also include a program for implementing other functions or services. The memory 2002 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
[0208] The processor 2001 is connected to the memory 2002 via the bus 2005, and implements corresponding functions by calling the application program stored in the memory 2002. The processor 2001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0209] The electronic device 2000 can be connected to the network through the communication module 2003 (which may include but is not limited to components such as a network interface) to communicate with other devices (such as a user terminal or a server, etc.) through the network to achieve data interaction, such as sending data to other devices or receiving data from other devices. Among them, the communication module 2003 may include a wired network interface and / or a wireless network interface, etc., that is, the communication module may include at least one of a wired communication module or a wireless communication module.
[0210] The electronic device 2000 can be connected to the required input / output devices, such as a keyboard, a display device, etc., through the input / output interface 2004. The electronic device 2000 itself can have a display device, and can also be connected to other display devices through the interface 2004. Optionally, a storage device, such as a hard disk, can also be connected through the interface 2004, so that data in the electronic device 2000 can be stored in the storage device, or data in the storage device can be read, and data in the storage device can also be stored in the memory 2002. It can be understood that the input / output interface 2004 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 2004 can be a component of the electronic device 2000, or it can be an external device connected to the electronic device 2000 when needed.
[0211] The bus 2005 for connecting the components may include a path for transmitting information between the above components. The bus 2005 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. According to different functions, the bus 2005 may be divided into an address bus, a data bus, a control bus, etc.
[0212] Optionally, for the solution provided by the embodiment of the present invention, the memory 2002 can be used to store a computer program for executing the solution of the present invention, and run by the processor 2001. When the processor 2001 runs the computer program, the action of the method or device provided by the embodiment of the present invention is implemented.
[0213] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the corresponding content of the aforementioned method embodiment can be implemented.
[0214] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the corresponding content of the aforementioned method embodiment can be implemented.
[0215] It should be noted that the terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the application described herein can be implemented in an order other than that shown in the figure or described in the text.
[0216] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.
[0217] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.
Claims
1. A model training method, characterized in that: include: Acquire multiple labeled training samples in the target domain, each of the training samples includes a sample question, and the label of each training sample is a sample answer corresponding to the sample question in the training sample; Based on the multiple labeled training samples, a trained target question-answering model is obtained by continuously performing fine-tuning operations on the neural network model; wherein the neural network model includes a pre-trained first language model and initialized first model parameters and second model parameters, the first model parameters are used to indicate the word segmentation that needs to be intervened in the sample question, and the second model parameters are used to indicate the way to intervene in the feature vector of the word segmentation; wherein the first model parameters include sub-parameters of each word segmentation in the sample question, and the sub-parameters of a word segmentation are used to determine whether to adjust the first feature vector of the word segmentation; The fine-tuning operation includes: For each sample question, extract features of the sample question through the encoder of the first language model to obtain a first feature vector of each word segment in the sample question; determine the random noise corresponding to each word segment in the sample question; adjust the sub-parameters of each word segment in the first model parameter based on the random noise corresponding to each word segment to obtain a target adjustment probability of each word segment; determine each first word segment to be adjusted according to the target adjustment probability of each word segment; and adjust the first feature vector of each first word segment according to the second model parameter to obtain an adjusted second feature vector of each first word segment; For each sample question, decoding is performed by the decoder of the first language model based on the target feature vector of the sample question to obtain a predicted answer corresponding to the sample question, wherein the target feature vector of the sample question is obtained based on the second feature vector of each of the first participles in the sample question and the first feature vectors of other participles except each of the first participles; According to the predicted answers and labels corresponding to each of the sample questions, a first training loss is determined, and based on the first training loss, the first model parameters and the second model parameters are adjusted to obtain the first model parameters and the second model parameters corresponding to the next fine-tuning operation.
2. The method according to claim 1, characterized in that The encoder includes a plurality of cascaded feature extraction layers; the first model parameters include layer parameters corresponding to at least one feature extraction layer; For each sample question, the predicted answer corresponding to the sample question is obtained in the following way: The sample question is input into the encoder, and the encoder is used to extract features of the sample question to obtain a first feature vector corresponding to each feature extraction layer for each word segment in the sample question; For each of the at least one feature extraction layer, based on the layer parameters corresponding to the feature extraction layer, determine each first participle to be adjusted in each participle of the sample question, use the second model parameter to adjust the first feature vector of each first participle to obtain the second feature vector of each first participle, use the second feature vector of each first participle and the first feature vectors of other participles except the first participle as the target feature vector of the feature extraction layer, and use the target feature vector of the feature extraction layer as the input of the next feature extraction layer; For each feature extraction layer except the at least one feature extraction layer, a first feature vector of each word in the sample question is obtained through the feature extraction layer as a target feature vector of the feature extraction layer, and the target feature vector of the feature extraction layer is used as an input of the next feature extraction layer; Based on the target feature vector of the last feature extraction layer of the encoder, decoding is performed through the decoder to obtain a predicted answer corresponding to the sample question.
3. The method according to claim 1, characterized in that The adjusting the first model parameter and the second model parameter based on the first training loss includes: Determining a second training loss according to the sum of the sub-parameters of each word segment in each of the sample questions; Based on the first training loss and the second training loss, first model parameters and second model parameters are adjusted.
4. The method according to claim 2, characterized in that: The layer parameters corresponding to a feature extraction layer include sub-parameters of each word segment corresponding to the feature extraction layer. The sub-parameter of a word segment corresponding to a feature extraction layer indicates whether to adjust the first feature vector extracted by the feature extraction layer for the word segment.
5. The method according to claim 1, characterized in that Before extracting features from the sample question by the encoder of the first language model, the method further includes: Determining a first number of participles in the sample question; If the first number is equal to the preset number, the sample problem is used as a sample problem to be processed; If the first number is less than the preset number, the sample question is filled with preset characters, and the filled sample question is used as the sample question to be processed, wherein the number of word segments in the filled sample question is the preset number.
6. The method according to claim 1, characterized in that The first model parameters are determined in the following manner: According to at least one of the content and part of speech of each segmentation in multiple sample questions, the importance of the segmentation at each position is counted; Determining each first position according to the importance of the segmentation at each position in each of the sample questions; A first model parameter is determined according to the sub-parameters of the word segments at each of the first positions.
7. The method according to claim 2, characterized in that The method further comprises: Determine the field type of the target field; If the domain type is the first type, the at least one feature extraction layer includes each feature extraction layer in the plurality of cascaded feature extraction layers; If the domain type is not the first type, the at least one feature extraction layer includes some feature extraction layers in the plurality of cascaded feature extraction layers.
8. The method according to claim 1, characterized in that The extracting features of the sample question by using the encoder of the first language model includes: Constructing a corresponding prompt word instruction based on the sample question, wherein the prompt word instruction is used to instruct the neural network model to output a corresponding answer based on the sample question; The encoder of the first language model performs feature extraction on the prompt word instruction.
9. A question-answering processing method, characterized in that: include: Get the questions to be answered in the target area; The answer corresponding to the question to be answered is obtained by using the trained target question-answering model; wherein the target question-answering model is trained using the method described in any one of claims 1 to 8.
10. A model training device, characterized in that: include: A sample acquisition module, used to acquire a plurality of labeled training samples in a target field, each of the training samples including a sample question, and the label of each training sample is a sample answer corresponding to the sample question in the training sample; A model fine-tuning module, configured to obtain a trained target question-answering model by continuously performing fine-tuning operations on the neural network model based on the multiple labeled training samples; wherein the neural network model includes a pre-trained first language model and initialized first model parameters and second model parameters, wherein the first model parameters are used to indicate the word segmentation that needs to be intervened in the sample question, and the second model parameters are used to indicate the way to intervene in the feature vector of the word segmentation; wherein the first model parameters include sub-parameters of each word segmentation in the sample question, and the sub-parameters of a word segmentation are used to determine whether to adjust the first feature vector of the word segmentation; The fine-tuning operation includes: For each sample question, extract features of the sample question through the encoder of the first language model to obtain a first feature vector of each word segment in the sample question; determine the random noise corresponding to each word segment in the sample question; adjust the sub-parameters of each word segment in the first model parameter based on the random noise corresponding to each word segment to obtain a target adjustment probability of each word segment; determine each first word segment to be adjusted according to the target adjustment probability of each word segment; and adjust the first feature vector of each first word segment according to the second model parameter to obtain an adjusted second feature vector of each first word segment; For each sample question, decoding is performed by the decoder of the first language model based on the target feature vector of the sample question to obtain a predicted answer corresponding to the sample question, wherein the target feature vector of the sample question is obtained based on the second feature vector of each of the first participles in the sample question and the first feature vectors of other participles except each of the first participles; According to the predicted answers and labels corresponding to each of the sample questions, a first training loss is determined, and based on the first training loss, the first model parameters and the second model parameters are adjusted to obtain the first model parameters and the second model parameters corresponding to the next fine-tuning operation.
11. The device according to claim 10, characterized in that The encoder includes a plurality of cascaded feature extraction layers; the first model parameters include layer parameters corresponding to at least one feature extraction layer; For each sample question, the predicted answer corresponding to the sample question is obtained in the following way: The sample question is input into the encoder, and the encoder is used to extract features of the sample question to obtain a first feature vector corresponding to each feature extraction layer for each word segment in the sample question; For each of the at least one feature extraction layer, based on the layer parameters corresponding to the feature extraction layer, determine each first participle to be adjusted in each participle of the sample question, use the second model parameter to adjust the first feature vector of each first participle to obtain the second feature vector of each first participle, use the second feature vector of each first participle and the first feature vectors of other participles except the first participle as the target feature vector of the feature extraction layer, and use the target feature vector of the feature extraction layer as the input of the next feature extraction layer; For each feature extraction layer except the at least one feature extraction layer, a first feature vector of each word in the sample question is obtained through the feature extraction layer as a target feature vector of the feature extraction layer, and the target feature vector of the feature extraction layer is used as an input of the next feature extraction layer; Based on the target feature vector of the last feature extraction layer of the encoder, decoding is performed through the decoder to obtain a predicted answer corresponding to the sample question.
12. The device according to claim 10, characterized in that The model fine-tuning module is used to: Determining a second training loss according to the sum of the sub-parameters of each word segment in each of the sample questions; Based on the first training loss and the second training loss, first model parameters and second model parameters are adjusted.
13. The device according to claim 11, characterized in that The layer parameters corresponding to a feature extraction layer include sub-parameters of each word segment corresponding to the feature extraction layer. The sub-parameter of a word segment corresponding to a feature extraction layer indicates whether to adjust the first feature vector extracted by the feature extraction layer for the word segment.
14. The device according to claim 10, characterized in that The model fine-tuning module is also used to: Determining a first number of participles in the sample question; If the first number is equal to the preset number, the sample problem is used as a sample problem to be processed; If the first number is less than the preset number, the sample question is filled with preset characters, and the filled sample question is used as the sample question to be processed, wherein the number of word segments in the filled sample question is the preset number.
15. The device according to claim 10, characterized in that The first model parameters are determined in the following manner: According to at least one of the content and part of speech of each segmentation in multiple sample questions, the importance of the segmentation at each position is counted; Determining each first position according to the importance of the segmentation at each position in each of the sample questions; A first model parameter is determined according to the sub-parameters of the word segments at each of the first positions.
16. The device according to claim 11, characterized in that The model fine-tuning module is also used to: Determine the field type of the target field; If the domain type is the first type, the at least one feature extraction layer includes each feature extraction layer in the plurality of cascaded feature extraction layers; If the domain type is not the first type, the at least one feature extraction layer includes some feature extraction layers in the plurality of cascaded feature extraction layers.
17. The device according to claim 10, characterized in that The model fine-tuning module is used to: Constructing a corresponding prompt word instruction based on the sample question, wherein the prompt word instruction is used to instruct the neural network model to output a corresponding answer based on the sample question; The encoder of the first language model performs feature extraction on the prompt word instruction.
18. A question-answer processing device, characterized in that: include: A question acquisition module is used to obtain questions to be answered in the target field; A reply module is used to obtain a reply corresponding to the question to be answered through a trained target question-answering model; wherein the target question-answering model is trained using the method described in any one of claims 1 to 8.
19. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 8 or claim 9.
20. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 or claim 9 is implemented.
21. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 or claim 9 is implemented.
