Model automatic optimization method and device, medium, electronic equipment and program product

Abstract

The invention relates to an automatic optimization method and device of a model, a medium, electronic equipment and a program product, and relates to the technical field of computers.The method comprises the steps that in response to feedback operation of a user on an answer text output by a target model, a question text corresponding to the answer text is obtained; a model optimization step is automatically executed, and the model optimization step comprises the steps that the quality type corresponding to the answer text is determined according to the feedback operation, the answer text and the question text; determining a target optimization strategy corresponding to the target model according to the quality type; and optimizing the target model according to the target optimization strategy. Therefore, on one hand, the dialogue data of the user is automatically collected and analyzed in an actual application scene to serve as a basis for optimizing the target model, and the sample number and sample diversity of model optimization are increased. On the other hand, human intervention is not needed in the whole process, the data analysis efficiency is greatly improved, and then the model optimization efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a method, device, medium, electronic device and program product for automatic optimization of a model. Background Art

[0002] With the rapid development of computer technology, the application scope of large models has rapidly expanded to all walks of life. However, despite the increasing application of large models, they still face great challenges in the application process. The effect of large models is highly dependent on the richness and accuracy of training data. However, in real scenarios, on the one hand, the amount of training data is very limited, and on the other hand, the analysis and screening of training data currently rely mainly on manual work, which makes the iterative optimization efficiency of large models low. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides a method, device, medium, electronic device and program product for automatic optimization of a model.

[0004] In a first aspect, the present disclosure provides an automatic optimization method of a model, applied to an electronic device, the method comprising: in response to a user's feedback operation on an answer text output by a target model, obtaining a question text corresponding to the answer text; the answer text is the answer text output by the target model after the question text input by the user is input into the target model; Automatically perform a model optimization step, wherein the model optimization step includes: Determine, according to the feedback operation, the answer text and the question text, a quality type corresponding to the answer text; the quality type includes a high-quality answer and a low-quality answer; Determining a target optimization strategy corresponding to the target model according to the quality type; The target model is optimized according to the target optimization strategy.

[0005] Optionally, determining the quality type corresponding to the answer text according to the feedback operation, the answer text and the question text includes: Determining the user's evaluation intention for the answer text according to the feedback operation; According to the question text, a target knowledge text corresponding to the question text is retrieved from a preset knowledge database; the target knowledge text is key information for answering the question text; According to the evaluation intention, the target knowledge text, the question text and the answer text, the quality type corresponding to the answer text is determined.

[0006] Optionally, the retrieving a target knowledge text corresponding to the question text from a preset knowledge database according to the question text comprises: According to the question text, obtaining knowledge text related to the question text from the preset knowledge database; Determine a first weight corresponding to each of the knowledge texts; The knowledge text with the largest first weight is used as the target knowledge text.

[0007] Optionally, the determining, according to the evaluation intention, the target knowledge text, the question text and the answer text, the quality type corresponding to the answer text comprises: Determining whether the answer text meets a preset evaluation condition according to the target knowledge text, the question text and the answer text; In the case where the evaluation intention includes satisfaction, if the answer text satisfies the preset evaluation condition, determining that the quality type corresponding to the answer text includes a high-quality answer; or, In the case where the evaluation intention includes dissatisfaction, if the answer text does not meet the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0008] Optionally, the preset evaluation conditions include: The matching degree between the answer text and the question text is greater than or equal to a preset matching degree threshold; The answer text does not have any semantic ambiguity; There are no logical problems in the answer text; The answer text is consistent with the target knowledge text in terms of content.

[0009] Optionally, the method further comprises: Obtain the number of satisfied users corresponding to the answer text, where the number of satisfied users is the number of users whose evaluation intention is satisfied; In the case where the evaluation intention includes satisfaction, if the answer text meets the preset evaluation condition, determining that the quality type corresponding to the answer text includes a high-quality answer includes: In the case where the evaluation intention includes satisfaction and the number of satisfied users is greater than or equal to a first preset number threshold, if the answer text meets the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a high-quality answer.

[0010] Optionally, the method further comprises: determining whether the question text is valid; In the case where the evaluation intention includes dissatisfaction, if the answer text does not meet the preset evaluation condition, determining that the quality type corresponding to the answer text includes a low-quality answer includes: In the case where the evaluation intention includes dissatisfaction and the question text is determined to be valid, if the answer text does not meet the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0011] Optionally, the answer text is also obtained by: Combining the question text and the target knowledge text to generate a corresponding target text; The target text is input into the target model to obtain the answer text output by the target text.

[0012] Optionally, in the case where the quality type includes high-quality answers, the target optimization strategy includes taking the question text and the answer text as positive samples to train the target model.

[0013] Optionally, the method further comprises: In a case where the quality type includes a high-quality answer, a second weight corresponding to the target knowledge text in the preset knowledge database is increased.

[0014] Optionally, when the quality type includes low-quality answers, the target optimization strategy includes taking the question text and the answer text as negative samples to train the target model.

[0015] Optionally, the method further comprises: In a case where the quality type includes a low-quality answer, a second weight corresponding to the target knowledge text in the preset knowledge database is reduced.

[0016] In a second aspect, the present disclosure provides an automatic optimization device for a model, which is applied to an electronic device, and the device includes: An acquisition module, configured to acquire a question text corresponding to the answer text in response to a user's feedback operation on the answer text output by the target model; the answer text is the answer text output by the target model after the question text input by the user is input into the target model; A first determination module is used to determine the quality type corresponding to the answer text according to the feedback operation, the answer text and the question text; the quality type includes a high-quality answer and a low-quality answer; A second determination module is used to determine a target optimization strategy corresponding to the target model according to the quality type; The optimization module is used to optimize the target model according to the target optimization strategy.

[0017] Optionally, the first determination module is used to determine the user's evaluation intention for the answer text based on the feedback operation; retrieve a target knowledge text corresponding to the question text from a preset knowledge database based on the question text; the target knowledge text is key information used to answer the question text; and determine the quality type corresponding to the answer text based on the evaluation intention, the target knowledge text, the question text and the answer text.

[0018] Optionally, the first determination module is used to obtain knowledge texts related to the question text from the preset knowledge database based on the question text; determine the first weight corresponding to each of the knowledge texts; and use the knowledge text with the largest first weight as the target knowledge text.

[0019] Optionally, the first determination module is used to determine whether the answer text satisfies preset evaluation conditions based on the target knowledge text, the question text and the answer text; when the evaluation intention includes satisfaction, if the answer text satisfies the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a high-quality answer; or, when the evaluation intention includes dissatisfaction, if the answer text does not satisfy the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0020] Optionally, the preset evaluation conditions include: The matching degree between the answer text and the question text is greater than or equal to a preset matching degree threshold; The answer text does not have any semantic ambiguity; There are no logical problems in the answer text; The answer text is consistent with the target knowledge text in terms of content.

[0021] Optionally, the acquisition module is further used to acquire the number of satisfied users corresponding to the answer text, where the number of satisfied users is the number of users whose evaluation intention is satisfied; The first determination module is used to determine that the quality type corresponding to the answer text includes a high-quality answer if the evaluation intention includes satisfaction and the number of satisfied users is greater than or equal to a first preset number threshold and if the answer text meets the preset evaluation condition.

[0022] Optionally, the first determination module is used to determine whether the question text is valid; when the evaluation intention includes dissatisfaction and the question text is determined to be valid, if the answer text does not meet the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0023] Optionally, the answer text is also obtained by: Combining the question text and the target knowledge text to generate a corresponding target text; The target text is input into the target model to obtain the answer text output by the target text.

[0024] Optionally, in the case where the quality type includes high-quality answers, the target optimization strategy includes taking the question text and the answer text as positive samples to train the target model.

[0025] Optionally, the device further comprises: An adjustment module is used to increase the second weight corresponding to the target knowledge text in the preset knowledge database when the quality type includes a high-quality answer.

[0026] Optionally, when the quality type includes low-quality answers, the target optimization strategy includes taking the question text and the answer text as negative samples to train the target model.

[0027] Optionally, the adjustment module is further configured to reduce a second weight corresponding to the target knowledge text in the preset knowledge database when the quality type includes a low-quality answer.

[0028] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the automatic optimization method of the model provided in the first aspect of the present disclosure.

[0029] In a fourth aspect, the present disclosure provides an electronic device, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of the automatic optimization method of the model provided in the first aspect of the present disclosure.

[0030] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the automatic optimization method of the model provided in the first aspect of the present disclosure.

[0031] The present disclosure provides a method for automatically optimizing a model. First, in response to a user's feedback operation on an answer text output by a target model, a question text corresponding to the answer text is obtained; the answer text is the answer text output by the target model after the question text input by the user is input into the target model; secondly, according to the feedback operation, the answer text and the question text, the quality type corresponding to the answer text is determined; the quality type includes high-quality answers and low-quality answers; then, according to the quality type, the target optimization strategy corresponding to the target model is determined; finally, according to the target optimization strategy, the target model is optimized. Through the above technical solution, it is possible to determine whether the quality type of the current answer text is a high-quality answer or a low-quality answer based on the question text input by the user and the answer text output by the target model, combined with the user's feedback operation on the answer text output by the target model, so as to realize automatic review and marking of the answer text. Furthermore, according to the determined quality type, a corresponding target optimization strategy is generated, and the target model is controlled to be optimized according to the target optimization strategy, thereby guiding the model to achieve automatic optimization. On the one hand, by automatically collecting and analyzing user conversation data in actual application scenarios as the basis for optimizing the target model, the number and diversity of samples for model optimization are increased. On the other hand, the entire process does not require human intervention, which greatly improves the efficiency of data analysis and thus the efficiency of model optimization.

[0032] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 It is a flowchart of a method for automatic optimization of a model according to an exemplary embodiment.

[0034] Figure 2 The invention is a flowchart of a method for automatically optimizing a model according to an exemplary embodiment.

[0035] Figure 3 It is a block diagram of an automatic optimization device for a model according to an exemplary embodiment.

[0036] Figure 4 It is a block diagram of an automatic optimization device of another model according to an exemplary embodiment.

[0037] Figure 5 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0039] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0040] The terms "first", "second", etc. in the specification and claims of this application and the above drawings are used to distinguish similar objects and do not have to be understood as a specific order or sequence. In addition, in the description with reference to the drawings, the same symbols in different drawings represent the same elements.

[0041] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0042] In the description of the present disclosure, unless otherwise specified, "multiple" means two or more than two, and other quantifiers are similar thereto; "at least one item", "one item or multiple items" or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one item a can represent any number of a; for another example, one item or multiple items among a, b and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple; "and / or" is a kind of description of the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " indicates that the associated objects before and after are in an "or" relationship.

[0043] Although operations or steps are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood that it is required to perform these operations or steps in the specific order shown or in a serial order, or to perform all the operations or steps shown to obtain the desired results. In the embodiments of the present disclosure, these operations or steps can be performed in series; these operations or steps can also be performed in parallel; or some of these operations or steps can be performed.

[0044] Before introducing the automatic optimization method, device, medium, electronic device and program product of the model provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. The present disclosure can be applied in the scenario of training a large model, wherein the large model may include, for example but not limited to, BERT (Bidirectional Encoder Representations from Transformers), LLM (Large Language Model), GPT (Generative Pre-Trained Transformer) series, etc. The effect of the large model is highly dependent on the richness and accuracy of the training data, but in real scenarios, on the one hand, the amount of training data is very limited, and on the other hand, the current analysis and screening of training data still mainly rely on manual work, which makes the iterative optimization efficiency of the large model low.

[0045] In order to solve the above technical problems, the present invention provides a method, device, medium, electronic device and program product for automatic optimization of a model, which can determine whether the quality type of the current answer text is a high-quality answer or a low-quality answer based on the question text input by the user and the answer text output by the target model, combined with the user's feedback operation on the answer text output by the target model, so as to automatically review and mark the answer text. Then, according to the determined quality type, a corresponding target optimization strategy is generated, and the target model is controlled to be optimized according to the target optimization strategy, thereby guiding the model to achieve automatic optimization. On the one hand, by automatically collecting and analyzing the user's dialogue data in actual application scenarios as the basis for optimizing the target model, the number of samples and sample diversity of model optimization are increased. On the other hand, the whole process does not require human intervention, which greatly improves the efficiency of data analysis, thereby improving the efficiency of model optimization.

[0046] The following is a brief introduction to the automatic optimization method of the model provided by the present disclosure. Figure 1As shown, the present disclosure adds a feedback collection module to the front end on the basis of the existing user-large model dialogue, which is used to collect feedback actions triggered by the user on whether the answer output by the current model is satisfactory. For example, a feedback button that is convenient to operate can be provided in the user interface, such as like, step, copy, paste, share, export, text input, etc. The user can trigger the corresponding feedback operation through the interface displayed by the front end to interact, which serves as the basis for the subsequent back-end analysis of the user's intention. After obtaining the user feedback, the user feedback can be analyzed to determine whether the user is satisfied with the answer output by the current large model, and then further determine whether the answer text output by the large model is a high-quality answer or a low-quality answer according to different user intentions. Secondly, in order to more accurately determine whether the answer text is a high-quality answer or a low-quality answer in the present disclosure, a preset knowledge database is introduced, in which a large amount of knowledge information is stored, and the knowledge answer corresponding to the user's question can be obtained by searching the user's question text in the preset database. Then, according to the knowledge answer and the user's question, it is determined whether the answer text output by the current large model is a high-quality answer or a low-quality answer. Afterwards, based on the judgment results, the large model is reinforced and annotated. For high-quality answers, the large model can further strengthen learning to enhance the model effect. For low-quality answers, the large model can make feedback adjustments to reduce the occurrence of such problems. At the same time, the judgment results and the knowledge information in the preset knowledge database can be iteratively updated to further improve the accuracy and reliability of the knowledge information in the preset knowledge database.

[0047] In this embodiment, the above analysis and judgment can be achieved by building an intelligent agent in the electronic device. Specifically, LangChain can be introduced as the core scheduling tool in the intelligent agent to integrate the large model, vector database, interactive layer prompt, external knowledge, and external agent tools. Among them, the LangChain is mainly composed of the following 6 core components: a) Model I / O: The interface for interacting with the large model. The large model can be called and information can be extracted from the output of the large model through the general interface; b) Data connection: an interface for interacting with data of a specific application, through which data can be loaded, converted, stored and queried; c) Chains: combine components to achieve end-to-end applications; d) Memory: used to record the input and output of the large model; e) Agents: extend the reasoning capability of the model and, as external modules, provide computing, logic, retrieval and other functional support; f) Callbacks: Extend the reasoning capabilities of the model, used for complex application call sequences, and provide support for functions such as logging, monitoring, and streaming processing.

[0048] The specific implementation modes of the present invention are described in detail below with reference to the accompanying drawings.

[0049] Figure 2 is a flow chart of a method for automatic optimization of a model according to an exemplary embodiment, which is applied to electronic devices such as Figure 2 As shown, the method may include the following steps: In step S101, in response to a user's feedback operation on an answer text output by a target model, a question text corresponding to the answer text is obtained.

[0050] The answer text is the answer text output by the target model after the question text input by the user is input into the target model.

[0051] In some embodiments, the feedback operation is, for example, but not limited to, like, dislike, copy, paste, share, export, and text input. For example, all user operation behaviors can be captured by using front-end hooks and event listeners. After obtaining the feedback operation triggered by the user, the captured operation behavior can be converted into a unified processable format to facilitate further analysis of the user's intention. In this embodiment, it can be determined whether the user is satisfied with the answer text output by the current target model based on the feedback operation triggered by the user.

[0052] After obtaining the question text corresponding to the answer text, the following model optimization steps from step S102 to step S104 can be automatically executed: In step S102, the quality type corresponding to the answer text is determined according to the feedback operation, the answer text and the question text.

[0053] The quality type includes a first quality answer and a second quality answer, wherein the quality of the first quality answer is better than the quality of the second quality answer. The first quality answer can also be called a high quality answer, and the second quality answer can also be called a low quality answer.

[0054] In some embodiments, the user's evaluation intention for the answer text may be determined based on the feedback operation, and the evaluation intention may include satisfaction or dissatisfaction. Then, different analyses may be performed based on the answer text and the question text according to different evaluation intentions to determine the quality type corresponding to the answer text.

[0055] In step S103, a target optimization strategy corresponding to the target model is determined according to the quality type.

[0056] In step S104, the target model is optimized according to the target optimization strategy.

[0057] In some embodiments, when the quality type includes high-quality answers, it can be determined that the answer text output by the current target model is reliable and accurate. Therefore, it can be determined that the target optimization strategy includes using the question text and the answer text as positive samples to train the target model, so as to inform the model that it can continue to refer to such questions to output answers, thereby achieving the effect of strengthening the target model.

[0058] In other embodiments, when the quality type includes low-quality answers, it can be determined that the answer text output by the current target model is unreliable and inaccurate. Therefore, it can be determined that the target optimization strategy includes using the question text and the answer text as negative samples to train the target model to adjust relevant parameters in the target model to avoid the same error in the model later.

[0059] By adopting the above method, it is possible to determine whether the quality type of the current answer text is a high-quality answer or a low-quality answer based on the question text input by the user and the answer text output by the target model, combined with the user's feedback operation on the answer text output by the target model, so as to automatically review and mark the answer text. Then, according to the determined quality type, a corresponding target optimization strategy is generated, and the target model is controlled to be optimized according to the target optimization strategy, thereby guiding the model to achieve automatic optimization. On the one hand, by automatically collecting and analyzing the user's conversation data in actual application scenarios as the basis for optimizing the target model, the number and diversity of samples for model optimization are increased. On the other hand, the entire process does not require human intervention, which greatly improves the efficiency of data analysis, thereby improving the efficiency of model optimization.

[0060] The above-mentioned determination of the user's evaluation intention for the answer text is described in detail below.

[0061] In a possible implementation, preset operations corresponding to different evaluation intentions can be pre-set, so that when the feedback operation includes a first preset operation, it can be determined that the evaluation intention includes satisfaction; or when the feedback operation includes a second preset operation, it can be determined that the evaluation intention includes dissatisfaction.

[0062] For example, if the user's feedback operation includes a like, since the intention of satisfaction expressed by the like is very clear, it can be determined that the user is satisfied with the answer text output by the current target model. Therefore, it can be determined that the current evaluation intention includes satisfaction. If the user's feedback operation includes a dislike, since the intention of dissatisfaction expressed by the dislike is very clear, it can be determined that the user is dissatisfied with the answer text output by the current target model. Therefore, it can be determined that the current evaluation intention includes dissatisfaction.

[0063] In another possible implementation, different weight values ​​may be set in advance for different feedback operations. For example, the weight value of an operation behavior with a clear negative intention may be set to a negative value, and the weight value of an operation behavior with a clear positive intention may be set to a positive value. In addition, different weight values ​​may be set for different operation behaviors under the same intention. For example, the clearer the positive intention is, the greater the weight value is, and vice versa. Then, after obtaining the user's feedback operation, the weight value corresponding to the feedback operation may be determined. And when the weight value is greater than or equal to the first preset weight value, it is determined that the evaluation intention includes satisfaction; or, when the weight value is less than or equal to the second preset weight value, it is determined that the evaluation intention includes dissatisfaction; wherein the second preset weight value is less than the first preset weight value. If the feedback operation includes multiple, the weight sum corresponding to the multiple feedback operations may be determined, and according to the weight sum, the user's evaluation intention may be further judged by the above method.

[0064] It should be noted that if the weight value is between the second preset weight value and the first preset weight value, it indicates that the user's evaluation intention is unclear, and the feedback operation can be ignored, that is, no subsequent analysis and judgment will be performed. In this way, some answer texts with unclear intentions can be excluded to a certain extent, reducing the amount of data analysis and improving the accuracy of data analysis.

[0065] After determining the user's evaluation intention, the evaluation intention can also be stored in the back-end database, and a behavior log can be created for each user to facilitate subsequent information retrieval.

[0066] At the same time, based on each user's behavior log, the user portrait and user preferences corresponding to each user can be analyzed, so that in the subsequent retrieval process, the user portrait can be combined to output target knowledge text that better meets user needs.

[0067] The following is a further detailed description of how to determine the quality type corresponding to the answer text under the above two evaluation intentions.

[0068] In the above step S102, determining the quality type corresponding to the answer text according to the feedback operation, the answer text and the question text may include: Step A: Determine the user's evaluation intention for the answer text based on the feedback operation.

[0069] Step B: according to the question text, retrieve the target knowledge text corresponding to the question text from the preset knowledge database.

[0070] The target knowledge text is the key information used to answer the question text.

[0071] In order to more accurately analyze the quality type of the answer text output by the target model, it is also necessary to provide a basis for evaluation. In this embodiment, the target knowledge text corresponding to the question text can also be retrieved from the preset knowledge database according to the question text, as an important basis for subsequent analysis of the quality type of the answer text.

[0072] In some embodiments, the preset knowledge database may be a RAG database. The RAG database uses the question text raised by the user as input through the Retrieval-Augmented Generation (RAG) technology to retrieve relevant background knowledge to enhance the understanding of the question and the answer, and uses machine learning algorithms such as collaborative filtering or cluster analysis to explore the correlation between the question text and various domain knowledge.

[0073] Specifically, the implementation of RAG technology can be divided into the following steps: (1) Document loading: Parse the document (such as PDF) into a string and process it using a document loader.

[0074] (2) Text segmentation: Split the parsed string into text chunks that fit the target model’s dialog window size. This size depends on the target model’s window size.

[0075] (3) Vectorization: Use the embedding model to convert text blocks into vectors and save them in a vector database.

[0076] (4) Index construction: Designing information such as databases, collections, fields, and indexes to retrieve data from the database.

[0077] (5) Retrieval: Based on the user query (i.e., the question text entered by the user), the same embedding model is used for vectorization and a search is performed in the index to find the most relevant text block.

[0078] (6) Generation: The retrieved text block is used as the knowledge corresponding to the question text.

[0079] In other words, the target knowledge text corresponding to the question text can be obtained through RAG retrieval.

[0080] Furthermore, considering that a question text may correspond to multiple possible knowledge, when constructing a database, a higher weight can be set for knowledge with high reliability, so that when searching, knowledge with high data weight will be prioritized. Specifically, according to the question text, knowledge texts related to the question text can be obtained from the preset knowledge database; the first weight corresponding to each of the knowledge texts can be determined; and the knowledge text with the largest first weight can be used as the target knowledge text.

[0081] In addition, on the basis of obtaining the target knowledge text, in order to further improve the reliability of the target model output and avoid model hallucinations, the question text and the target knowledge text can also be combined to generate a corresponding target text, and the target text can be input into the target model to obtain the answer text output by the target text. Among them, the question text and the target knowledge text can be combined through the corresponding prompt template to generate the corresponding target text. In other words, in the subsequent model generation process, it is no longer just the question text input by the user that is answered, but a more reliable and accurate target knowledge text is provided as input to improve the accuracy of the model output and reduce the risk of the model generating misleading information.

[0082] Step C, determining the quality type corresponding to the answer text according to the evaluation intention, the target knowledge text, the question text and the answer text.

[0083] The following explains how to determine the quality type corresponding to the answer text according to different evaluation intentions.

[0084] 1. Evaluation intention includes satisfaction When the evaluation intention includes satisfaction, it indicates that the user is satisfied with the answer text, but it does not mean that the answer text is necessarily reliable and accurate. Therefore, it is possible to further determine whether the answer text meets the preset evaluation conditions.

[0085] The preset evaluation conditions may include, for example: a. The matching degree between the answer text and the question text is greater than or equal to the preset matching degree threshold.

[0086] That is, by judging whether the answer text matches the question text, it is determined whether the answer output by the target model is related to the question.

[0087] b. The answer text does not have the problem of unclear semantics.

[0088] That is, determine whether the answer text is clearly organized.

[0089] c. There is no logical problem in the answer text.

[0090] For example, the Prompt Engineering technology in LangChain can be used to detect whether the answer text is logically coherent.

[0091] d. The answer text is consistent with the target knowledge text in terms of content.

[0092] By ensuring that the answer text is consistent with the target knowledge text in terms of content, it is possible to verify whether the content in the answer text is consistent with objective facts.

[0093] It should be noted that determining the degree of match between the answer text and the question text, detecting whether the answer text has unclear semantics, whether the answer text has logical problems, and whether the answer text is consistent with the target knowledge text in content can all be detected through models or algorithms in related technologies, which will not be elaborated here.

[0094] In some embodiments, when it is determined that the evaluation intention includes satisfaction and the answer text meets the above-mentioned preset evaluation conditions, it can be determined that the quality type corresponding to the answer text includes a high-quality answer.

[0095] In other embodiments, in order to further improve the accuracy of the judgment and avoid the unreliable evaluation intention of a certain user affecting the judgment of the quality type, the number of satisfied users corresponding to the answer text can also be obtained, wherein the number of satisfied users is the number of users whose evaluation intention is satisfied. Specifically, the number of satisfied users can be obtained by obtaining the user's behavior log, and then, if the evaluation intention includes satisfaction and the number of satisfied users is greater than or equal to the first preset number threshold, if the answer text meets the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a high-quality answer.

[0096] In other words, it is possible to further detect whether multiple people have given satisfactory evaluations to the same question (the content of the question is consistent, but the form may be inconsistent). At the same time, in order to ensure that the determined number of satisfied users is reliable and accurate. It is also possible to analyze each user who gives a satisfactory evaluation. For example, it is possible to determine whether each user who gives a satisfactory evaluation has abnormal behavior. The abnormal behavior may include the number of consecutive satisfactory evaluations (not limited to the current question text) exceeding the second preset number threshold. In other words, if a user gives a satisfactory evaluation to all questions, it can be considered that the credibility of the user's evaluation is low. Therefore, the number of users who give satisfactory evaluations to the same question and do not have abnormal behavior can be used as the number of satisfied users.

[0097] In the case where the evaluation intention includes satisfaction, if the number of satisfied users is less than a first preset number threshold and / or the answer text does not meet the preset evaluation condition, the feedback operation may be discarded without any processing.

[0098] Furthermore, if the answer text is determined to be a high-quality answer, the question text and the answer text can be used as positive samples to perform SFT (Supervised Finetuning) training on the target model. SFT training can stimulate the target model's ability to understand various questions in the field and answer them. Preferably, SFT training can be performed for 3-5 rounds to enhance the effect of the model.

[0099] In addition, in the case where the quality type includes high-quality answers, the second weight corresponding to the target knowledge text in the preset knowledge database can also be increased. Through the above analysis, it can be determined that the answer text is a high-quality answer, and the answer text is generated based on the target knowledge text, which also shows that the target knowledge text is reliable and accurate. Therefore, the second weight corresponding to the target knowledge text in the preset knowledge database can be increased to increase the possibility that the subsequent RGA will output the target knowledge text during retrieval. Specifically, the quality of the target knowledge text and the accuracy and comprehensiveness when it is retrieved can be improved through methods such as data cleaning, recall optimization, global processing and multi-level indexing.

[0100] It is also possible to record the prompt word list of the target model generated by the information of the frequently retrieved knowledge, and synchronously update the prompt word template in the prompt template library to improve the accuracy of the corresponding target text generated by the model in subsequent applications.

[0101] 2. Evaluation intention includes dissatisfaction When the evaluation intention includes dissatisfaction, it indicates that the user is dissatisfied with the answer text, but it does not mean that the answer text is necessarily unreliable and inaccurate. Therefore, it is possible to further determine whether the answer text meets the preset evaluation conditions.

[0102] In some embodiments, when the evaluation intention includes dissatisfaction and the answer text does not meet the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0103] As long as the answer text does not satisfy any one of the above-mentioned preset evaluation conditions, it can be considered that the answer text does not satisfy the preset evaluation condition.

[0104] In other embodiments, it is considered that in some scenarios, the question text input by the user is unreasonable, for example, why is XX a table tennis champion, but in fact XX is a basketball player. Therefore, in order to avoid the unreasonable question text affecting the subsequent model training, the question text raised by the user can be first analyzed in depth, that is, to determine whether the question text is valid. Specifically, the validity of the question text can be determined by the detection method in the relevant technology, which will not be elaborated here. If it is determined that the question text is valid, it means that the question text raised by the user is reasonable and clear. Therefore, if the evaluation intention includes dissatisfaction and the question text is determined to be valid, if the answer text does not meet the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0105] If it is determined that the question text is invalid, it means that the question text raised by the user is unreasonable, and the feedback operation can be abandoned without any processing.

[0106] Furthermore, when the quality type includes low-quality answers, the question text and the answer text can be used as negative samples to perform RLHF (Reinforcement Learning from Human Feedback) training on the target model. Through RLHF training, the training process is accelerated, the model performance is improved, and the model is made more in line with human expectations, so as to reduce the occurrence of similar problems.

[0107] In some embodiments, when it is determined that the answer text does not meet the preset evaluation condition, the answer text can also be scored according to the evaluation condition that the answer text specifically does not meet, and the more evaluation conditions that are not met, the lower the score. The answer text corresponding question type can also be further given, for example, if the answer text does not meet the matching degree of the answer text and the question text is greater than or equal to the preset matching degree threshold, then its question type is determined to be that the answer fails to accurately respond to the question. If the answer text does not meet the problem that the answer text does not have unclear semantics, then its question type is determined to be that the answer content is chaotic and lacks a clear organizational structure. If the answer text does not meet the logical problem that the answer text does not have, then its question type is determined to be that the discussion in the answer lacks logic and is difficult for the user to understand. If the answer text does not meet the answer text and the target knowledge text are consistent in content, then its question type is determined to be that the case contains wrong information or is contrary to known facts. At the same time, a question feedback module can also be provided for the user, allowing the user to further point out the problem of the answer text, and then the feedback problem output by the user can be used as the question type of the answer text.

[0108] Therefore, the question type and score of the question text can also be fed back to the target model, and the target model can be optimized through RLHF training.

[0109] In addition, when the quality type includes low-quality answers, the second weight corresponding to the target knowledge text in the preset knowledge database may be reduced to reduce the possibility that the subsequent RGA outputs the target knowledge text during retrieval.

[0110] In actual application scenarios, low-quality answers that have been tuned by RLHF can continue to be produced and evaluated, and user feedback can be observed until there is no user feedback that the question text is of low quality, and RLHF training is no longer performed.

[0111] In some embodiments, corresponding personalized options can also be set for the user. For example, if the user hopes that the answer text output by the target model is concise, the target model can refine and integrate the text before outputting the answer text to obtain a more concise text for output, so as to improve the user experience.

[0112] In other embodiments, when it is determined that the answer text is a low-quality answer, the answer text related to the question text can be further recommended to the user. For example, the knowledge text with the largest weight value other than the target knowledge text retrieved by RGA can be used as the new target knowledge text, and a new answer text is output based on the new target knowledge text. At the same time, the user's feedback operation can be continuously monitored, and corresponding analysis and judgment can be performed based on the user's feedback operation, and the user experience and model effectiveness can be improved through continuous iteration and optimization of the model.

[0113] By adopting the above method, it is possible to determine whether the quality type of the current answer text is a high-quality answer or a low-quality answer based on the question text input by the user and the answer text output by the target model, combined with the user's feedback operation on the answer text output by the target model, so as to automatically review and mark the answer text. Then, according to the determined quality type, a corresponding target optimization strategy is generated, and the target model is controlled to be optimized according to the target optimization strategy, thereby guiding the model to achieve automatic optimization. On the one hand, by automatically collecting and analyzing the user's conversation data in actual application scenarios as the basis for optimizing the target model, the number and diversity of samples for model optimization are increased. On the other hand, the entire process does not require human intervention, which greatly improves the efficiency of data analysis, thereby improving the efficiency of model optimization.

[0114] Figure 3 is a block diagram of an automatic optimization device for a model according to an exemplary embodiment, which is applied to electronic devices such as Figure 3 As shown, the device 200 includes: The acquisition module 201 is used to obtain the question text corresponding to the answer text in response to the user's feedback operation on the answer text output by the target model; the answer text is the answer text output by the target model after the question text input by the user is input into the target model; A first determination module 202 is used to determine the quality type corresponding to the answer text according to the feedback operation, the answer text and the question text; the quality type includes a high-quality answer and a low-quality answer; A second determination module 203 is used to determine a target optimization strategy corresponding to the target model according to the quality type; The optimization module 204 is used to optimize the target model according to the target optimization strategy.

[0115] Optionally, the first determination module 202 is used to determine the user's evaluation intention for the answer text based on the feedback operation; retrieve a target knowledge text corresponding to the question text from a preset knowledge database based on the question text; the target knowledge text is key information for answering the question text; and determine the quality type corresponding to the answer text based on the evaluation intention, the target knowledge text, the question text and the answer text.

[0116] Optionally, the first determination module 202 is used to obtain knowledge texts related to the question text from the preset knowledge database according to the question text; determine the first weight corresponding to each of the knowledge texts; and use the knowledge text with the largest first weight as the target knowledge text.

[0117] Optionally, the first determination module 202 is used to determine whether the answer text satisfies preset evaluation conditions based on the target knowledge text, the question text and the answer text; when the evaluation intention includes satisfaction, if the answer text satisfies the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a high-quality answer; or, when the evaluation intention includes dissatisfaction, if the answer text does not meet the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0118] Optionally, the preset evaluation condition includes: The matching degree between the answer text and the question text is greater than or equal to a preset matching degree threshold; The answer text does not have the problem of unclear semantics; There are no logical problems with this answer text; The answer text is consistent with the target knowledge text in terms of content.

[0119] Optionally, the acquisition module 201 is further used to acquire the number of satisfied users corresponding to the answer text, where the number of satisfied users is the number of users whose evaluation intention is satisfied; The first determination module 202 is used to determine that the quality type corresponding to the answer text includes a high-quality answer if the evaluation intention includes satisfaction and the number of satisfied users is greater than or equal to a first preset number threshold, and if the answer text meets the preset evaluation condition.

[0120] Optionally, the first determination module 202 is used to determine whether the question text is valid; when the evaluation intention includes dissatisfaction and the question text is determined to be valid, if the answer text does not meet the preset evaluation conditions, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

[0121] Optionally, the answer text is also obtained by: The question text and the target knowledge text are combined to generate a corresponding target text; The target text is input into the target model to obtain the answer text output by the target text.

[0122] Optionally, when the quality type includes high-quality answers, the target optimization strategy includes taking the question text and the answer text as positive samples to train the target model.

[0123] Alternatively, if Figure 4 As shown, the device 200 also includes: The adjustment module 205 is used to increase the second weight corresponding to the target knowledge text in the preset knowledge database when the quality type includes a high-quality answer.

[0124] Optionally, when the quality type includes low-quality answers, the target optimization strategy includes taking the question text and the answer text as negative samples to train the target model.

[0125] Optionally, the adjustment module 205 is further configured to reduce the second weight corresponding to the target knowledge text in the preset knowledge database when the quality type includes a low-quality answer.

[0126] By adopting the above device, it is possible to determine whether the quality type of the current answer text is a high-quality answer or a low-quality answer based on the question text input by the user and the answer text output by the target model, combined with the user's feedback operation on the answer text output by the target model, so as to automatically review and mark the answer text. Then, according to the determined quality type, a corresponding target optimization strategy is generated, and the target model is controlled to be optimized according to the target optimization strategy, thereby guiding the model to achieve automatic optimization. On the one hand, by automatically collecting and analyzing the user's conversation data in actual application scenarios as the basis for optimizing the target model, the number and diversity of samples for model optimization are increased. On the other hand, the entire process does not require human intervention, which greatly improves the efficiency of data analysis, thereby improving the efficiency of model optimization.

[0127] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0128] Figure 5 is a block diagram of an electronic device 300 according to an exemplary embodiment. For example, the electronic device 300 may be provided as a server. Figure 5 The electronic device 300 includes a processor 322, which may be one or more, and a memory 332 for storing a computer program executable by the processor 322. The computer program stored in the memory 332 may include one or more modules each corresponding to a set of instructions. In addition, the processor 322 may be configured to execute the computer program to perform the above-mentioned automatic optimization method of the model.

[0129] In addition, the electronic device 300 may further include a power supply component 326 and a communication component 350, wherein the power supply component 326 may be configured to perform power management of the electronic device 300, and the communication component 350 may be configured to implement communication of the electronic device 300, for example, wired or wireless communication. In addition, the electronic device 300 may further include an input / output (I / O) interface 358. The electronic device 300 may operate based on an operating system stored in the memory 332, such as Windows ServerTM, Mac OSXTM, UnixTM, LinuxTM, etc.

[0130] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by the processor, the steps of the automatic optimization method of the model described above are implemented. For example, the computer-readable storage medium can be the memory 332 including the program instructions, and the program instructions can be executed by the processor 322 of the electronic device 300 to complete the automatic optimization method of the model described above.

[0131] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a processor. When the computer program is executed by the processor, the steps of the above-mentioned automatic optimization method of the model are implemented.

[0132] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0133] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0134] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for automatic optimization of a model, characterized in that: Applied to electronic equipment, the method comprises: In response to a user's feedback operation on an answer text output by a target model, a question text corresponding to the answer text is obtained; the answer text is the answer text output by the target model after the question text input by the user is input into the target model; Automatically perform a model optimization step, wherein the model optimization step includes: Determine, according to the feedback operation, the answer text and the question text, a quality type corresponding to the answer text; the quality type includes a high-quality answer and a low-quality answer; Determining a target optimization strategy corresponding to the target model according to the quality type; The target model is optimized according to the target optimization strategy.

2. The method according to claim 1, characterized in that Determining the quality type corresponding to the answer text according to the feedback operation, the answer text and the question text includes: Determining the user's evaluation intention for the answer text according to the feedback operation; According to the question text, a target knowledge text corresponding to the question text is retrieved from a preset knowledge database; the target knowledge text is key information for answering the question text; According to the evaluation intention, the target knowledge text, the question text and the answer text, the quality type corresponding to the answer text is determined.

3. The method according to claim 2, characterized in that The retrieving a target knowledge text corresponding to the question text from a preset knowledge database according to the question text comprises: According to the question text, obtaining knowledge text related to the question text from the preset knowledge database; Determine a first weight corresponding to each of the knowledge texts; The knowledge text with the largest first weight is used as the target knowledge text.

4. The method according to claim 2, characterized in that: The determining, according to the evaluation intention, the target knowledge text, the question text and the answer text, of the quality type corresponding to the answer text comprises: Determining whether the answer text meets a preset evaluation condition according to the target knowledge text, the question text and the answer text; In the case where the evaluation intention includes satisfaction, if the answer text satisfies the preset evaluation condition, determining that the quality type corresponding to the answer text includes a high-quality answer; or, In the case where the evaluation intention includes dissatisfaction, if the answer text does not meet the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

5. The method according to claim 4, characterized in that The preset evaluation conditions include: The matching degree between the answer text and the question text is greater than or equal to a preset matching degree threshold; The answer text does not have any semantic ambiguity; There are no logical problems in the answer text; The answer text is consistent with the target knowledge text in content.

6. The method according to claim 4, characterized in that The method further comprises: Obtain the number of satisfied users corresponding to the answer text, where the number of satisfied users is the number of users whose evaluation intention is satisfied; In the case where the evaluation intention includes satisfaction, if the answer text meets the preset evaluation condition, determining that the quality type corresponding to the answer text includes a high-quality answer includes: In the case where the evaluation intention includes satisfaction and the number of satisfied users is greater than or equal to a first preset number threshold, if the answer text meets the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a high-quality answer.

7. The method according to claim 4, characterized in that The method further comprises: determining whether the question text is valid; In the case where the evaluation intention includes dissatisfaction, if the answer text does not meet the preset evaluation condition, determining that the quality type corresponding to the answer text includes a low-quality answer includes: In the case where the evaluation intention includes dissatisfaction and the question text is determined to be valid, if the answer text does not meet the preset evaluation condition, it is determined that the quality type corresponding to the answer text includes a low-quality answer.

8. The method according to claim 2, characterized in that: The answer text is also obtained by: Combining the question text and the target knowledge text to generate a corresponding target text; The target text is input into the target model to obtain the answer text output by the target text.

9. The method according to any one of claims 2 to 8, characterized in that In the case where the quality type includes high-quality answers, the target optimization strategy includes taking the question text and the answer text as positive samples to train the target model.

10. The method according to claim 9, characterized in that The method further comprises: In a case where the quality type includes a high-quality answer, a second weight corresponding to the target knowledge text in the preset knowledge database is increased.

11. The method according to any one of claims 2 to 8, characterized in that In the case where the quality type includes low-quality answers, the target optimization strategy includes taking the question text and the answer text as negative samples to train the target model.

12. The method according to claim 11, characterized in that The method further comprises: In a case where the quality type includes a low-quality answer, a second weight corresponding to the target knowledge text in the preset knowledge database is reduced.

13. An automatic optimization device for a model, characterized in that: Applied to electronic equipment, the device comprises: An acquisition module, configured to acquire a question text corresponding to the answer text in response to a user's feedback operation on the answer text output by the target model; the answer text is the answer text output by the target model after the question text input by the user is input into the target model; A first determination module is used to determine the quality type corresponding to the answer text according to the feedback operation, the answer text and the question text; the quality type includes a high-quality answer and a low-quality answer; A second determination module is used to determine a target optimization strategy corresponding to the target model according to the quality type; The optimization module is used to optimize the target model according to the target optimization strategy.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images