Preference prediction method and device, electronic equipment and readable storage medium
By acquiring and splicing the initial text and response text, and using the text prediction model to determine the preference, it solves the problem of difficulty in accurately predicting user preferences in the prior art, and achieves a higher user experience and the matching degree of the model to user needs.
Patent Information
- Application Number
- CN202411618389.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-13
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art is difficult to accurately predict users' preferences for the generation of different large-scale language models, and is easily affected by position deviations and verbose deviations, resulting in the generation results of the model that do not match the actual needs of users and reduce user experience.
By obtaining the initial text and at least two response texts, splicing them into the target dialogue text, and entering the text prediction model, the text prediction model trains the initial text prediction model through the training set to determine the preferences of the initial text and each response text to determine the target response text.
By objectively evaluating the dialogue performance of each response generation model, determine the best response generation model and the response text it generates, improve user experience, and avoid the problem that the model generation results do not match the user's real needs.
Smart Images
Figure CN119537572A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a preference prediction method, device, electronic device and readable storage medium. Background Art
[0002] With the development of deep learning technology, large-scale language models have made significant progress in the field of natural language processing. These models have performed well in tasks such as dialogue generation and text classification, and are widely used in dialogue systems, virtual assistants, generative AI products and other fields.
[0003] However, in the interaction between users and large-scale language models, there will be certain differences in the text responses generated by different large-scale language models for the same text, which makes it difficult to accurately predict the user's preferences for the results generated by different large-scale language models. When processing user preference text responses, the existing technology is easily affected by position bias (preferentially selecting the first result displayed) and length bias (longer responses are considered better), making it difficult to achieve accurate preference prediction, resulting in the model generation results not meeting the user's actual needs. Summary of the invention
[0004] In view of this, the embodiments of the present application provide a preference prediction method, device, electronic device and readable storage medium to solve the problem in the prior art that it is difficult to predict the user's true preference response, resulting in the model generation results not being consistent with the user's real needs, thereby reducing the user's usage experience.
[0005] In a first aspect of an embodiment of the present application, a preference prediction method is provided, the method comprising:
[0006] An initial text and at least two response texts corresponding to the initial text are obtained, wherein each response text is generated by a different response generation model according to the initial text; the initial text and the at least two response texts are spliced to obtain a target dialogue text; the target dialogue text is input into a text prediction model, the text prediction model is used to determine the preference between the initial text and each response text in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes a plurality of sample initial texts, each sample initial text corresponds to at least two sample response texts, the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model; the target response text is determined according to the preference between the initial text and each response text.
[0007] According to a second aspect of an embodiment of the present application, a preference prediction device is provided, the device comprising:
[0008] An acquisition module is configured to acquire an initial text and at least two response texts corresponding to the initial text, wherein each response text is generated according to the initial text by a different response generation model;
[0009] The splicing module is configured to splice the initial text and at least two response texts to obtain the target dialogue text; the prediction module is configured to input the target dialogue text into the text prediction model, the text prediction model is used to determine the preference between the initial text and each response text in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes multiple sample initial texts, each sample initial text corresponds to at least two sample response texts, and the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model; the determination module is configured to determine the target response text according to the preference between the initial text and each response text.
[0010] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0011] According to a fourth aspect of an embodiment of the present application, a readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0012] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0013] According to the technical solution provided by the present application, an initial text and at least two response texts corresponding to the initial text are obtained; the initial text and at least two response texts are spliced to obtain a target dialogue text; the target dialogue text is input into a text prediction model, the text prediction model is used to determine the preference between the initial text and each response text in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes multiple sample initial texts, each sample initial text corresponds to at least two sample response texts, the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model; the target response text is determined according to the preference between the initial text and each response text. The loss value of the text prediction model is determined by the preference between the sample initial text and each sample response text and the matching degree between the sample response text and each response generation model. The dialogue performance of each response generation model can be objectively evaluated to determine the best response generation model and the response text generated by it. The text prediction model is further used to determine the preference between the initial text and each response text in the target dialogue text. The target response text that meets the user's preference can be determined from at least two response texts, thereby improving the user experience. This avoids the problem in the prior art that it is difficult to predict the user's real preference response, resulting in the model generation result not being consistent with the user's real needs, and reducing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0015] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present application;
[0016] Figure 2 It is a flowchart of a preference prediction method provided in an embodiment of the present application;
[0017] Figure 3 is a schematic diagram of the structure of a preference prediction device provided in an embodiment of the present application;
[0018] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0020] A preference prediction method, device, electronic device, and readable storage medium according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.
[0021] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario may include terminal devices 1, 2 and 3, a server 4 and a network 5.
[0022] Terminal devices 1, 2 and 3 can be hardware or software. When terminal devices 1, 2 and 3 are hardware, they can be various electronic devices with display screens and supporting communication with server 4, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc.; when terminal devices 1, 2 and 3 are software, they can be installed in the above electronic devices. Terminal devices 1, 2 and 3 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present application are not limited to this. Furthermore, various applications can be installed on terminal devices 1, 2 and 3, such as data processing applications, instant messaging tools, social platform software, preference prediction applications, shopping applications, etc.
[0023] The server 4 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 4 may be a single server, a server cluster composed of several servers, or a cloud computing service center, which is not limited in the present application.
[0024] It should be noted that the server 4 can be hardware or software. When the server 4 is hardware, it can be various electronic devices that provide various services for the terminal devices 1, 2, and 3. When the server 4 is software, it can be multiple software or software modules that provide various services for the terminal devices 1, 2, and 3, or it can be a single software or software module that provides various services for the terminal devices 1, 2, and 3, and the embodiments of the present application are not limited to this.
[0025] The network 5 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present application.
[0026] It should be noted that the specific types, quantities and combinations of the terminal devices 1, 2 and 3, the server 4 and the network 5 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present application do not limit this.
[0027] Figure 2 is a flow chart of a preference prediction method provided in an embodiment of the present application. Figure 2 As shown, the method includes:
[0028] S201, obtaining an initial text and at least two response texts corresponding to the initial text, wherein each response text is generated by a different response generation model according to the initial text;
[0029] S202, concatenating the initial text and at least two response texts to obtain a target dialogue text;
[0030] S203, inputting the target dialogue text into a text prediction model, the text prediction model is used to determine the preference between the initial text and each response text in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes a plurality of sample initial texts, each sample initial text corresponds to at least two sample response texts, and the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model;
[0031] S204, determining a target response text according to the preference between the initial text and each response text.
[0032] Specifically, the initial text is the text to be answered or the text to be responded to, which can be input by the user or by the dialogue model. Secondly, at least two response texts are response texts to the initial text, each of which is generated by a different response generation model. Exemplarily, the initial text can be "How are you doing recently?" The response generation model can be a natural language generation model based on deep learning, such as a Transformer-based model, such as BERT, GPT, etc. The response text can be "I'm fine, thank you." output by BERT or "I'm a little busy recently" output by GPT.
[0033] It is understandable that in the process of intelligent dialogue, multiple rounds of dialogue are generally included. Therefore, before inputting the target dialogue text into the text prediction model, each round of dialogue needs to be spliced into a continuous text stream (target dialogue text). In addition, special markers can be inserted at the beginning and end of each round of dialogue, such as<start_of_turn> and<end_of_turn> , helps the text prediction model identify the boundaries of each round of text, so as to better understand the structure and order of the conversation, thereby providing the text prediction model with more comprehensive and accurate conversation information, thereby improving the prediction quality and accuracy of the model.
[0034] In addition, in order to ensure that the target dialogue text length input to the text prediction model does not exceed the processing capacity of the text prediction model, a dynamic truncation strategy can be used to truncate the target dialogue text length. Exemplarily, when the target dialogue text length exceeds the maximum limit (such as 3072 tokens), the model can delete it from the back to the front to ensure that the target dialogue text length remains within the maximum limit (such as 3072 tokens). At the same time, the strategy of deleting from the back to the front can also ensure that important context information is retained, ensuring the accuracy of the text prediction model during prediction.
[0035] It can be understood that after the target dialogue text is input into the text prediction model, the text prediction model can determine the preference of the initial text and each response text in the target dialogue text, wherein the preference represents the user's preference for each response text, which can be reflected by a specific value. Continuing with the above example, if the text prediction model outputs a preference of 70% for "I'm fine, thank you" and a preference of 30% for "I'm a little busy recently", it can be determined that the user may be more inclined to the response text "I'm fine, thank you". That is, the response text "I'm fine, thank you" is determined as the target response text based on the preference of the initial text and each response text.
[0036] It should be noted that the above-mentioned text prediction model is obtained by training the initial text prediction model with a preset training set, which includes multiple sample initial texts, each sample initial text corresponds to at least two sample response texts, and the loss value of the initial text prediction model is determined by the preference of the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model, wherein the matching degree is used to indicate the probability that the sample response text is generated by which specific response generation model. During the training process, by determining the matching degree between the sample response text and each response generation model, the response generation model corresponding to the sample response text can be determined, thereby guiding the training of the text prediction model and improving the prediction accuracy of the text prediction model.
[0037] According to the technical solution provided in the embodiment of the present application, an initial text and at least two response texts corresponding to the initial text are obtained; the initial text and the at least two response texts are spliced to obtain a target dialogue text; the target dialogue text is input into a text prediction model, the text prediction model is used to determine the preference between the initial text and each response text in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes multiple sample initial texts, each sample initial text corresponds to at least two sample response texts, the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model; the target response text is determined according to the preference between the initial text and each response text. The loss value of the text prediction model is determined by the preference between the sample initial text and each sample response text and the matching degree between the sample response text and each response generation model. The dialogue performance of each response generation model can be objectively evaluated to determine the best response generation model and the response text generated by it. The text prediction model is further used to determine the preference between the initial text and each response text in the target dialogue text. The target response text that meets the user's preference can be determined from at least two response texts, thereby improving the user experience. This avoids the problem in the prior art that it is difficult to predict the user's real preference response, resulting in the model generation result not being consistent with the user's real needs, and reducing the user experience.
[0038] In some embodiments, the text prediction model includes a multi-head classifier, and before the target dialogue text is input into the text prediction model, it includes: inputting the sample initial text and at least two sample response texts into the multi-head classifier for prediction, obtaining the preference between the sample initial text and each sample response text, and obtaining the matching degree between the sample response text and each response generation model; calculating a first loss value based on the preference between the sample initial text and each sample response text and the true preference label; calculating a second loss value based on the matching degree between the sample response text and each response generation model and the true matching label; obtaining the loss value output by the multi-head classifier based on the first loss value and the second loss value; and obtaining a trained text prediction model when the loss value is less than or equal to a preset value.
[0039] Specifically, a multi-head classifier is a machine learning model that can handle multiple classification tasks at the same time, each with its own output layer. In a multi-head classifier, the sample initial text and at least two sample response texts are fed into a shared feature extractor, and then the data is fed into different output layers for prediction based on the different tasks. Each output layer is optimized specifically for a specific prediction task, allowing the model to learn multiple prediction targets at the same time.
[0040] It can be understood that in this embodiment, the multi-head classifier can be used to determine the preference of the sample initial text and each sample response text, and determine the matching degree of the sample response text and each response generation model. However, the preference and matching degree determined by the multi-head classifier may be different from the true preference and true matching degree. Therefore, it is necessary to calculate the first loss value of the preference of the sample initial text and each sample response text and the true preference label, and calculate the second loss value of the matching degree of the sample response text and each response generation model and the true matching label. Then, based on these two loss values, the multi-head classifier is optimized, that is, the initial text prediction model is optimized to reduce the difference between the prediction result and the true label. When the loss value of the initial text prediction model is less than or equal to the preset value, a trained text prediction model is obtained.
[0041] According to the technical solution provided in the embodiment of the present application, the preference between the sample initial text and each sample response text is obtained through a multi-head classifier, and the matching degree between the sample response text and each response generation model is obtained, which can effectively improve the accuracy and efficiency of the text prediction model. Since the multi-head classifier can handle multiple classification tasks at the same time, it can optimize multiple classification targets at the same time, thereby improving the generalization ability of the text prediction model. In addition, by calculating the loss value and optimizing the model based on the loss value, it can be ensured that the model continuously adjusts parameters during the training process to better fit the real data, thereby improving the reliability of the prediction results.
[0042] In some embodiments, based on the first loss value and the second loss value, a loss value output by the multi-head classifier is obtained, including: determining a first weighting coefficient corresponding to the first loss value, and determining a second weighting coefficient corresponding to the second loss value; according to the first weighting coefficient and the second weighting coefficient, weighted summing the first loss value and the second loss value is performed to obtain the loss value.
[0043] Specifically, during the training process of the model, it is often necessary to face scenarios where multiple loss values need to be optimized. To this end, a multi-objective optimization or weighted loss function method can be used to assign a specific weighting coefficient to each loss function (or objective) to balance the importance between them. It can be understood that the weighting coefficients (first weighting coefficient and second weighting coefficient) corresponding to each of the two loss values (i.e., the first loss value and the second loss value) need to be clarified. The setting basis of this coefficient can be based on specific task requirements, the importance of loss values, and the experience of parameter adjustment in the experiment. For example, if the preference prediction has a greater impact on the performance of the overall text prediction model, a higher weighting coefficient can be assigned to the first loss value. On the contrary, if the matching degree of the response generation model is more critical to the accuracy of the final prediction result, a higher weighting coefficient can be assigned to the second loss value. In addition, the first weighting coefficient and the second weighting coefficient can be set according to user settings or models to ensure that the model can balance the importance of different classification tasks during training.
[0044] According to the technical solution provided in the embodiment of the present application, by reasonably determining the first weighting coefficient and the second weighting coefficient, and performing weighted summation on the loss value based on these weighting coefficients, the importance of different prediction tasks can be effectively balanced, thereby improving the accuracy and generalization ability of the text prediction model. At the same time, it is ensured that the model continuously adjusts parameters during the training process to better fit the real data, thereby improving the reliability of the prediction results.
[0045] In some embodiments, before inputting the target dialogue text into the text prediction model, it also includes: using a low-rank matrix method to fine-tune the initial text prediction model to reduce the number of parameters of the initial text prediction model; quantizing the fine-tuned initial text prediction model to compress the weight of the initial text prediction model, and using a preset optimizer to configure the parameters of the quantized initial text prediction model to obtain a text prediction model.
[0046] Specifically, the low-rank matrix method is used to fine-tune the initial text prediction model. The basic idea is to reduce the number of parameters by approximating the parameter matrix of the original model and expressing it with a lower rank. During the fine-tuning process, the parameter matrix of the initial text prediction model is first decomposed to obtain a low-rank approximation matrix. Then, the parameters of the low-rank matrix are adjusted through the optimization algorithm to minimize the difference with the original model parameter matrix. In this way, not only the number of parameters of the model can be reduced, but also the prediction performance of the model can be maintained. Specifically, low-rank matrix methods such as Adapter, Prefix Tuning and LoRA can be used to fine-tune the initial text prediction model.
[0047] Furthermore, after fine-tuning is completed, the initial text prediction model is quantized to compress the model weights and reduce the storage and computing requirements of the model. During the quantization process, the model weights are converted from floating point numbers to low-precision numerical representations, such as fixed-point numbers or binary representations. By selecting appropriate quantization strategies and precision, the storage and computing costs of the model can be significantly reduced while ensuring model performance. Specifically, the initial text prediction model after fine-tuning can be quantized with QLoRA's 4-bit and 8-bit precisions.
[0048] Furthermore, the model after quantization needs to be parameterized to ensure the performance of the model in practical applications. The preset optimizer is used to configure the parameters of the model after quantization, and adjust the parameters of the model through the optimization algorithm to achieve the best prediction effect. For example, the adamw_8bit optimizer can be used, and the batch size of the optimizer is set to 2, the number of gradient accumulation steps is 4, and the total global batch size is 8. In this way, the model can complete the training task of large-scale models with limited GPU resources.
[0049] Finally, after fine-tuning, quantizing, and parameterizing the initial text prediction model, an optimized text prediction model is obtained. This model not only has a lower number of parameters and smaller storage requirements, but also can provide accurate prediction results in practical applications. In this way, the text prediction model can be effectively deployed on resource-constrained devices, such as mobile devices or embedded systems, thereby expanding the application scope of text prediction technology.
[0050] In some embodiments, the method further includes: dividing the training set into K subsets, wherein each subset includes multiple sample initial texts, and each sample initial text corresponds to at least two sample response texts; taking K-1 subsets as training data and the remaining subset as verification data in turn, performing K times of training and verification to ensure that each subset is verified as a verification set.
[0051] Specifically, by dividing the training set into K subsets and performing K-fold cross validation, the overfitting phenomenon of the model during the training process can be effectively avoided. In each iteration, K-1 subsets are used to train the model, and the remaining subset is used to verify the performance of the model. In this way, it can be ensured that the model is fully trained and verified on different data subsets, thereby improving the generalization ability of the model. This embodiment preferably uses 5-fold cross validation to train the model.
[0052] In addition, during the training process, the model can be trained according to the set number of epochs and the log of each epoch can be recorded. After each training, the system evaluates the performance of the model and optimizes the model based on the loss. Finally, through K-fold cross-validation, a model that performs well on multiple data subsets can be obtained. This makes the model not only have high accuracy and generalization ability, but also can provide stable and reliable prediction results in practical applications, thereby improving the robustness and applicability of the text prediction model, so as to better meet the prediction needs in different scenarios.
[0053] In some embodiments, after the target dialogue text is input into the text prediction model, it also includes: predicting the target dialogue text through a multi-head classifier in the text prediction model to obtain a first preference for the initial text and each response text in the target dialogue text; exchanging the response order of at least two response texts in the target dialogue text, predicting the target dialogue text after the response order is exchanged through the multi-head classifier in the text prediction model to obtain a second preference for the initial text and each response text in the target dialogue text; and determining the target response text based on the first preference and the second preference.
[0054] Specifically, in machine learning and deep learning, the prediction results of the model may be affected by the response order. For example, taking two text responses as an example, the response text generated by the existing A response generation model and the response text generated by the B response generation model, if the A response text appears before the B response text in the target dialogue text, the model may tend to believe that the user prefers the A response text. In order to reduce the impact of this order deviation on the prediction results, this embodiment proposes a method of exchanging the order of the response texts and re-predicting.
[0055] It can be understood that the target dialogue text is first predicted by the multi-head classifier in the text prediction model to obtain the first preference between the initial text and each response text. Then, the response order of at least two response texts in the target dialogue text is exchanged, and the target dialogue text after the exchanged order is predicted again by the multi-head classifier in the text prediction model to obtain the second preference between the initial text and each response text. By comparing the first preference and the second preference, the target response text can be determined.
[0056] For example, if the first preference between the initial text and the A response text is higher than the first preference between the initial text and the B response text, for example, the first preference between the initial text and the A response text is 70%, and the first preference between the initial text and the B response text is 60%. However, after the order is swapped, the second preference between the initial text and the B response text is higher than the second preference between the initial text and the A response text, for example, the first preference between the initial text and the A response text is 55%, and the first preference between the initial text and the B response text is 65%. At this time, the final target response text can be comprehensively determined based on the first preference and the second preference, so as to reduce the influence of the response order on the text prediction model.
[0057] According to the technical solution provided in the embodiment of the present application, the target dialogue text is predicted by the multi-head classifier in the text prediction model to obtain the first preference of the initial text and each response text in the target dialogue text; the response order of at least two response texts in the target dialogue text is exchanged, and the target dialogue text after the response order is exchanged is predicted by the multi-head classifier in the text prediction model to obtain the second preference of the initial text and each response text in the target dialogue text; the target response text is determined according to the first preference and the second preference, which can effectively reduce the impact of the response order on the prediction result and improve the accuracy and reliability of the text prediction model. In addition, this method can also be applied to other scenarios that need to consider the impact of text order, such as sentiment analysis, text classification, etc., so as to improve the robustness and applicability of the model in practical applications.
[0058] In some embodiments, a target response text is determined based on a first preference and a second preference, including: taking a weighted average of the first preference and the second preference to calculate a comprehensive preference for each response text; sorting at least two response texts based on the comprehensive preference to determine a preference ranking of the at least two response texts; and selecting the response text with the highest comprehensive preference from the preference ranking as the target response text of the initial text.
[0059] Specifically, the first preference and the second preference are weighted averaged to synthesize the preferences of different response texts under the response order, so as to obtain a more comprehensive and accurate comprehensive preference. After calculating the comprehensive preference of each response text, at least two response texts can be sorted according to these comprehensive preferences. The basis for sorting is based on the level of comprehensive preference, that is, the higher the comprehensive preference, the higher the position of the response text in the sorting. Finally, the response text with the highest comprehensive preference is selected from the sorted response texts as the target response text of the initial text.
[0060] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the process of the embodiments of the present application.
[0061] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.
[0062] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0063] Figure 3 Schematic diagram of a preference prediction device provided in an embodiment of the present application. Figure 3 As shown, the device comprises:
[0064] An acquisition module 301 is configured to acquire an initial text and at least two response texts corresponding to the initial text, wherein each response text is generated by a different response generation model according to the initial text;
[0065] A splicing module 302 is configured to splice the initial text and at least two response texts to obtain a target dialogue text;
[0066] The prediction module 303 is configured to input the target dialogue text into a text prediction model, the text prediction model is used to determine the preference between the initial text and each response text in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes a plurality of sample initial texts, each sample initial text corresponds to at least two sample response texts, and the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model;
[0067] The determination module 304 is configured to determine the target response text according to the preference between the initial text and each response text.
[0068] In some embodiments, the preference prediction device of the present application further includes a training module 305, which is configured to input the sample initial text and at least two sample response texts into a multi-head classifier for prediction, obtain the preference of the sample initial text and each sample response text, and obtain the matching degree of the sample response text and each response generation model; calculate a first loss value based on the preference and true preference label of the sample initial text and each sample response text; calculate a second loss value based on the matching degree and true matching label of the sample response text and each response generation model; obtain the loss value output by the multi-head classifier based on the first loss value and the second loss value;
[0069] In some embodiments, the training module 305 is further configured to determine a first weighting coefficient corresponding to the first loss value, and determine a second weighting coefficient corresponding to the second loss value; and perform weighted summation on the first loss value and the second loss value according to the first weighting coefficient and the second weighting coefficient to obtain a loss value. When the loss value is less than or equal to a preset value, a trained text prediction model is obtained.
[0070] In some embodiments, the prediction module 303 is also configured to use a low-rank matrix method to fine-tune the initial text prediction model to reduce the number of parameters of the initial text prediction model; quantize the fine-tuned initial text prediction model to compress the weight of the initial text prediction model, and use a preset optimizer to configure parameters of the quantized initial text prediction model to obtain a text prediction model.
[0071] In some embodiments, the prediction module 303 is further configured to divide the training set into K subsets, each subset including multiple sample initial texts, and each sample initial text corresponds to at least two sample response texts; K-1 subsets are used as training data in turn, and the remaining subset is used as verification data, and K training and verification are performed to ensure that each subset is verified as a verification set.
[0072] In some embodiments, the prediction module 303 is further configured to predict the target dialogue text through a multi-head classifier in a text prediction model to obtain a first preference for the initial text and each response text in the target dialogue text; exchange the response order of at least two response texts in the target dialogue text, and predict the target dialogue text after the response order is exchanged through a multi-head classifier in the text prediction model to obtain a second preference for the initial text and each response text in the target dialogue text; and determine the target response text based on the first preference and the second preference.
[0073] In some embodiments, the prediction module 303 is also configured to perform a weighted average of the first preference and the second preference to calculate a comprehensive preference for each response text; sort at least two response texts according to the comprehensive preference to determine a preference ranking of at least two response texts; and select the response text with the highest comprehensive preference from the preference ranking as the target response text of the initial text.
[0074] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0075] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.
[0076] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0077] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.
[0078] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.
[0079] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The readable storage medium may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0080] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A preference prediction method, characterized in that: include: Acquire an initial text and at least two response texts corresponding to the initial text, wherein each of the response texts is generated according to the initial text by a different response generation model; splicing the initial text and the at least two response texts to obtain a target dialogue text; Inputting the target dialogue text into a text prediction model, the text prediction model is used to determine the preference between the initial text and each of the response texts in the target dialogue text, the text prediction model is obtained by training the initial text prediction model with a preset training set, the training set includes a plurality of sample initial texts, each of the sample initial texts corresponds to at least two sample response texts, and the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model; The target response text is determined according to the preference between the initial text and each of the response texts.
2. The method according to claim 1, characterized in that The text prediction model includes a multi-head classifier, and the step of inputting the target dialogue text into the text prediction model includes: Inputting the sample initial text and at least two sample response texts into the multi-head classifier for prediction, obtaining the preference of the sample initial text and each sample response text, and obtaining the matching degree of the sample response text and each response generation model; Calculate a first loss value according to the preference and true preference label of the sample initial text and each sample response text; Calculate a second loss value according to the matching degree between the sample response text and each response generation model and the true matching label; Based on the first loss value and the second loss value, obtaining a loss value output by the multi-head classifier; When the loss value is less than or equal to the preset value, a trained text prediction model is obtained.
3. The method according to claim 2, characterized in that The step of obtaining the loss value output by the multi-head classifier based on the first loss value and the second loss value includes: Determine a first weighting coefficient corresponding to the first loss value, and determine a second weighting coefficient corresponding to the second loss value; The first loss value and the second loss value are weighted and summed according to the first weighting coefficient and the second weighting coefficient to obtain the loss value.
4. The method according to claim 1, characterized in that: Before inputting the target dialogue text into the text prediction model, the method further includes: Fine-tuning the initial text prediction model using a low-rank matrix method to reduce the number of parameters of the initial text prediction model; The fine-tuned initial text prediction model is quantized to compress the weight of the initial text prediction model, and a preset optimizer is used to configure parameters of the quantized initial text prediction model to obtain the text prediction model.
5. The method according to claim 4, characterized in that The method further comprises: Dividing the training set into K subsets, wherein each subset includes a plurality of the sample initial texts, and each of the sample initial texts corresponds to at least two sample response texts; K-1 subsets are used as training data and the remaining subset is used as validation data. K training and validation are performed to ensure that each subset is validated as a validation set.
6. The method according to claim 1, characterized in that After inputting the target dialogue text into the text prediction model, the method further includes: Predicting the target dialogue text by using the multi-head classifier in the text prediction model to obtain a first preference degree between the initial text and each of the response texts in the target dialogue text; The response order of at least two response texts in the target dialogue text is exchanged, and the target dialogue text after the response order is exchanged is predicted by the multi-head classifier in the text prediction model to obtain a second preference degree between the initial text and each response text in the target dialogue text; The target response text is determined according to the first preference and the second preference.
7. The method according to claim 6, characterized in that The step of determining the target response text according to the first preference and the second preference includes: Taking a weighted average of the first preference and the second preference to calculate a comprehensive preference of each response text; Sorting the at least two response texts according to the comprehensive preference to determine a preference ranking of the at least two response texts; The response text with the highest comprehensive preference is selected from the preference ranking as the target response text of the initial text.
8. A preference prediction device, characterized in that: include: An acquisition module is configured to acquire an initial text and at least two response texts corresponding to the initial text, wherein each of the response texts is generated according to the initial text by a different response generation model; A splicing module is configured to splice the initial text and the at least two response texts to obtain a target dialogue text; A prediction module is configured to input the target dialogue text into a text prediction model, wherein the text prediction model is used to determine the preference between the initial text and each of the response texts in the target dialogue text, wherein the text prediction model is obtained by training the initial text prediction model with a preset training set, wherein the training set includes a plurality of sample initial texts, each of the sample initial texts corresponds to at least two sample response texts, and the loss value of the initial text prediction model is jointly determined by the preference between the sample initial text and each sample response text, and the matching degree between the sample response text and each response generation model; The determination module is configured to determine the target response text according to the preference between the initial text and each of the response texts.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.