A model hallucination mitigation method, program product, device, and medium

Abstract

The embodiment of the application provides a model illusion alleviation method, a program product, equipment and a medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: obtaining a question set and an initial answer set of a target model; inputting the question set and the initial answer set into a scoring model to obtain average scores of all class answers of the target model and an average score of the initial answer set as a whole; wherein the average score of each class answer is obtained according to the scores of all answers corresponding to each question in the question set, and the average score of the initial answer set as a whole is obtained according to the scores of all answers in the initial answer set; and based on an optimization rule of gradient descent, the target model is optimized according to the average scores of all class answers and the average score of the initial answer set as a whole. The application can improve the illusion alleviation effect of the target model, while reducing the cost of model calculation resources and human input.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method, program product, device, and medium for alleviating model hallucinations. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), various fields are actively using AI-related technologies based on LLMs to improve the automation and intelligence of their operations. A Large Language Model refers to a neural network model with a large number of parameters, which can learn rich knowledge and expressions through training on large text language datasets. Therefore, AI assistants based on Large Language Models and interacting through natural language are currently the most widespread applications. For ease of description, Large Language Models will be abbreviated as "LLM" below.

[0003] While large models can handle vast amounts of data and tasks, their outputs are sometimes not entirely accurate. This phenomenon, known as "model illusion," refers to the phenomenon where the output appears reasonable and logical, but actually contains errors or fabricated facts. Since large models rely on semantic information for text inference, model illusion cannot be completely eliminated; it can only be mitigated through fine-tuning and other techniques.

[0004] If fine-tuning is used to alleviate model illusions, a high-quality dataset containing both correct facts and typical illusions needs to be constructed, relying on data labeled by domain experts, which incurs extremely high costs for data collection and cleaning. In terms of model computation, high-performance GPU clusters are also required, along with hardware rental and electricity costs. Furthermore, fine-tuning has limited generalization capabilities, requiring repeated investment for new scenarios or knowledge updates. While this approach ultimately achieves the best illusion mitigation results, it also incurs significantly higher costs and expenses. Summary of the Invention

[0005] The purpose of this application is to provide a model hallucination relief method, program product, device and medium to solve the problems of high cost and poor effect of current model hallucination relief methods.

[0006] In a first aspect, embodiments of this application provide a method for alleviating model hallucinations, the method comprising: Obtain the question set and initial answer set for the target model; Input the question set and the initial answer set into the scoring model to obtain the average score of all answer classes in the target model and the average score of the initial answer set as a whole; where the average score of each answer class is obtained based on the scores of all answers corresponding to each question in the question set, and the average score of the initial answer set as a whole is obtained based on the scores of all answers in the initial answer set; The optimization rule based on gradient descent optimizes the target model based on the average score of all answer classes and the overall average score of the initial answer set.

[0007] In the above implementation process, by inputting the question set and initial answer set of the target model into the scoring model, the average score of all answer classes of the target model and the overall average score of the initial answer set are obtained. Based on the gradient descent optimization rule, the target model is optimized according to the average score of all answer classes and the overall average score of the initial answer set. The scoring model can efficiently obtain the scoring results of each answer class and the overall scoring results of the initial answer set. Based on the gradient descent optimization rule, the target model is comprehensively optimized by combining these scoring results, thereby alleviating the illusion of the target model and reducing the cost of computational resources and human resources invested in the target model.

[0008] Furthermore, the problem set for obtaining the target model includes: Obtain the questions corresponding to the application scenarios and business domains of the target model to obtain a question set.

[0009] In the above implementation process, by obtaining the problem set corresponding to the application scenarios and business domains of the target model, it is possible to ensure that the problem set covers the common application scenarios of the target model and the typical problem types that are prone to illusion, which is conducive to the subsequent targeted optimization of the target model.

[0010] Furthermore, obtaining the initial answer set of the target model includes: Input a set of questions into the target model, obtain multiple answers for each question in the set of questions, and classify all answers for each question into one type of answer; Summarize all the answers to obtain the initial answer set.

[0011] In the above implementation process, by using the target model to answer each question in the question set, all answers corresponding to the question are determined as one type of answer, and all types of answers are aggregated to obtain the initial answer set. This can take into account the case that each question has multiple answers, and ensure that answers covering different perspectives are obtained.

[0012] Furthermore, before inputting the input question set and initial answer set into the scoring model to obtain the average score of all answer classes in the target model and the overall average score of the initial answer set, the following steps are also included: Set the scoring rules for the scoring model; The scoring rules for setting the scoring model include: The question and answer are broken down into their constituent elements. The constituent elements of the question include the question background, question requirements, and question object. The constituent elements of the answer include the answer object, answer context, and answer conclusion. Each splitting element is vectorized to obtain the vector corresponding to each splitting element; Determine the cosine similarity between the vector of the question object and the vector of the answer object; Determine the Euclidean distance between the vectors representing the problem context and the vectors representing the answer context; Determine the Pearson correlation coefficient between the vector required by the question and the vector of the answer conclusion; The score for the answer is determined based on cosine similarity, Euclidean distance, and Pearson correlation coefficient.

[0013] In the above implementation process, by setting scoring rules based on the splitting elements of the question and the splitting elements of the answer, and using the scoring model to score all answers corresponding to each question output by the target model according to the scoring rules, it can be ensured that the scoring model reasonably scores all answers corresponding to each question output by the target model.

[0014] Furthermore, the input question set and initial answer set are fed into the scoring model to obtain the average score of all answer classes in the target model and the overall average score of the initial answer set, including: Input each question in the question set and all the answers corresponding to the current question into the scoring model, and obtain the scores of all answers output by the scoring model after scoring each answer according to the scoring rules; Based on the scores of all answers to the current question, determine the average score of all answers to the current question to obtain the average score of a category of answers; Determine the overall average score of the initial answer set based on the scores of all answers in the initial answer set.

[0015] In the above implementation process, the average score of all types of answers and the overall average score of the initial answer set can be obtained quickly, which facilitates the subsequent optimization of the target model based on the gradient descent optimization rule and the scoring results.

[0016] Furthermore, the gradient descent-based optimization rule optimizes the target model based on the average score of all answer classes and the overall average score of the initial answer set, including: The optimization space is defined by all types of answers, and the current optimization direction is the type of answer with the lowest average score. Sort the answers with the lowest average score from lowest to highest, and select the first set quantity answer with the lowest score in the current category. Input the first set-value answer and the corresponding scoring criteria into the target model, and obtain the second set-value answer output by the target model after learning and re-answering the question corresponding to the current class of answers; wherein, the scoring criteria are generated by the scoring model, and the scoring criteria include scoring dimensions and scoring details; Input the second set of answers into the scoring model to obtain the average score of the second set of answers; Compare the average score of the first setpoint answer with the average score of the second setpoint answer to determine whether the target model has been optimized; If the target model is optimized, the second set quantity answer and the corresponding scoring criteria are used to optimize the target model, and the optimized target model replaces the unoptimized target model. Continue to optimize the target model using the current type of answer until the average score of the subsequent set of answers generated by the target model is less than or equal to the overall average score of the previous answer set.

[0017] In the above implementation process, by using gradient descent-based optimization rules to optimize the target model according to the scoring results, the illusion of the target model can be effectively alleviated.

[0018] Further, the step of comparing the average score of the first set quantity answer and the average score of the second set quantity answer to determine whether the target model has been optimized includes: If the average score of the first set quantity answers is less than the average score of the second set quantity answers, then the target model is optimized; If the average score of the first set quantity answer is greater than or equal to the average score of the second set quantity answer, then record the number of boundary touches in one optimization direction, and continue to use the current type of answer to optimize the target model.

[0019] The above implementation process ensures the optimization effect of the target model.

[0020] Furthermore, the method also includes: Input the question set into the optimized target model to obtain a new answer set; Input a new set of answers into the scoring model for scoring and obtain the scoring criteria; The lowest average score of the answer type is selected as the current optimization direction, and the target model is optimized until the termination condition for optimization is met. Save the structure and parameters of the current target model to obtain the final optimized target model.

[0021] In the above implementation process, the optimal target model can be obtained by repeatedly optimizing the target model in a loop.

[0022] Furthermore, the termination conditions for reaching the end of optimization include: When the number of boundary touches reaches the preset maximum value, the optimization ends. When the number of iterations reaches the preset maximum limit, the optimization ends.

[0023] In the above implementation process, once the conditions are met, the structure and parameters of the current target model are saved, thus obtaining the optimized model.

[0024] Secondly, embodiments of this application provide a model hallucination relief device, the device comprising: The data acquisition module is used to acquire the question set and initial answer set of the target model; The scoring generation module is used to input the question set and the initial answer set into the scoring model to obtain the average score of all answer classes in the target model and the overall average score of the initial answer set. The average score of each answer class is obtained based on the scores of all answers corresponding to each question in the question set, and the overall average score of the initial answer set is obtained based on the scores of all answers in the initial answer set. The model optimization module is used to optimize the target model based on gradient descent optimization rules, according to the average score of all answer classes and the overall average score of the initial answer set.

[0025] Thirdly, embodiments of this application provide a computer program product, the computer program product including instructions, which, when executed by a computer, cause the computer to perform the method described above.

[0026] Fourthly, embodiments of this application provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the processor executes the computer program to implement the method described above.

[0027] Fifthly, embodiments of this application provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the method described above. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating a method for alleviating model hallucinations provided in the first embodiment of this application; Figure 2 A schematic diagram of a model hallucination relief device provided in the second embodiment of this application; Figure 3This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of this application. Detailed Implementation

[0030] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0031] It should be noted that in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. Furthermore, the step numbers in the text are only for the convenience of explaining the embodiments of this application and are not intended to limit the order in which the steps are performed.

[0032] With the rapid development of Large Language Models (LLMs), various fields are actively using AI-related technologies based on LLMs to improve the automation and intelligence of their operations. A Large Language Model refers to a neural network model with a large number of parameters, which can learn rich knowledge and expressions through training on large text language datasets. Therefore, AI assistants based on Large Language Models and interacting through natural language are currently the most widespread applications. For ease of description, Large Language Models will be abbreviated as "LLM" below.

[0033] While large models can handle large amounts of data and tasks, the results are not always perfect. A common flaw in large models is the "model illusion," where the output appears reasonable and logical, but actually contains errors or fabricated facts. Since large models perform text inference based on semantic information, the model illusion cannot be completely eliminated at present; it can only be mitigated through fine-tuning and other techniques.

[0034] If fine-tuning is used to alleviate model illusions, a high-quality dataset containing both correct facts and typical illusions needs to be constructed, relying on data labeled by domain experts, which incurs extremely high costs for data collection and cleaning. In terms of model computation, high-performance GPU clusters are also required, along with hardware rental and electricity costs. Furthermore, fine-tuning has limited generalization capabilities, requiring repeated investment for new scenarios or knowledge updates. While this approach ultimately achieves the best illusion mitigation results, it also incurs significantly higher costs and expenses.

[0035] To address this, this application proposes a model illusion mitigation method. By inputting the question set and initial answer set of the target model into a scoring model, the average score of all answer classes in the target model and the overall average score of the initial answer set are obtained. Based on the gradient descent optimization rule, the target model is optimized according to the average scores of all answer classes and the overall average score of the initial answer set. This method can efficiently obtain the scoring results of each answer class and the overall scoring results of the initial answer set using the scoring model. Based on the gradient descent optimization rule, the target model is comprehensively optimized by combining these scoring results, thereby mitigating the model illusion and reducing the cost of computational resources and human resources invested in the target model.

[0036] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0037] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for alleviating model hallucinations according to a first embodiment of this application. The method includes steps S101-S103: S101. Obtain the question set and initial answer set of the target model.

[0038] Specifically, select several questions corresponding to the target model to obtain the question set of the target model, and answer all the questions in the question set through the target model to obtain the initial answer set.

[0039] As an example, based on the application scenarios and business domains of the target model, several questions corresponding to the application scenarios and business domains of the target model are selected to obtain the question set of the target model. These questions can be collected from multiple channels, such as professional literature and materials, online Q&A platforms, historical data mining, etc. Clear screening criteria are established to ensure the quality of the question set. Screening criteria may include relevance, clarity, and diversity of the questions. The screened questions are then categorized and organized according to certain rules for easy use and management. Classification can be based on factors such as the topic, type, and difficulty of the questions.

[0040] As an example, a target model is used to answer all questions in the question set, with each question corresponding to multiple answers, and all answers to all questions forming an initial answer set. Specifically, after receiving a question, the target model uses its internal neural network structure and algorithms to analyze and process the question, ultimately generating the corresponding answer.

[0041] S102. Input the question set and the initial answer set into the scoring model to obtain the average score of all answer classes in the target model and the average score of the initial answer set as a whole. The average score of each answer class is obtained based on the scores of all answers corresponding to each question in the question set, and the average score of the initial answer set as a whole is obtained based on the scores of all answers in the initial answer set.

[0042] It should be noted that the core function of the scoring model is to evaluate the quality of the target model's answer through the correlation analysis between the question and the answer, including whether it meets the question requirements and whether there are logical or factual errors in the content. Therefore, the scoring model needs to have strong natural language understanding capabilities, long text processing capabilities, and the ability to access multi-domain knowledge bases. Optionally, this embodiment of the application selects Qwen3 as the scoring model based on this.

[0043] It should be noted that the target model provides multiple answers for each question in the question set, forming a category of answers (i.e., one question corresponds to multiple answers).

[0044] As an example, if the question set contains Z questions, then Z types of answers will be formed, constituting the initial answer set; the number of times each question is answered needs to be set according to the scenario requirements in order to cover answer performance from different perspectives.

[0045] Each question in the question set and all its corresponding answers are input into the Qwen3 scoring model. The scoring model scores each answer according to the scoring rules. Then, based on the scores of all answers for each question, the average score of that type of answer is calculated and recorded as the average score of a type of answer. At the same time, based on the scores of all answers for all questions, the overall average score of the answer dataset is calculated as the initial optimization baseline.

[0046] S103. Based on gradient descent optimization rules, the target model is optimized according to the average score of all answer classes and the overall average score of the initial answer set.

[0047] It should be noted that the optimization rule based on gradient descent specifically selects the optimization direction based on the average score of all answer classes (analogous to selecting the direction with the steepest gradient in gradient descent), and combines it with the overall average score of the initial answer set to optimize the target model.

[0048] This application embodiment inputs the question set and initial answer set of the target model into the scoring model to obtain the average score of all answer classes in the target model and the overall average score of the initial answer set. Based on the gradient descent optimization rule, the target model is optimized according to the average score of all answer classes and the overall average score of the initial answer set. It can efficiently obtain the scoring results of each answer class and the overall scoring results of the initial answer set using the scoring model. Based on the gradient descent optimization rule, the target model is comprehensively optimized by combining these scoring results, thereby alleviating the illusion of the target model and reducing the cost of computational resources and human resources invested in the target model.

[0049] In an optional embodiment, obtaining the problem set of the target model includes: obtaining problems corresponding to the application scenarios and business domains of the target model to obtain the problem set.

[0050] As an example, based on the application scenarios and business domains of the target model, a set of questions highly relevant to the business is randomly selected and numbered; the set of questions must cover common application scenarios of the target model and typical question types that are prone to causing illusions, so as to ensure the targeted nature of subsequent optimization.

[0051] Optionally, each question should be described in detail and accurately to ensure clarity and unambiguity. The description may include background, purpose, and specific content. The source of each initial question should be recorded, including the basis for selection and reference cases. This helps in subsequent tracing and analysis of the questions, understanding their rationality and scientific validity. To facilitate querying and management of the question set, a question index should be created, which can be set based on question number, question type, keywords, etc. The question set should be updated and improved regularly.

[0052] This application embodiment obtains a problem set by acquiring questions corresponding to the application scenarios and business domains of the target model. This ensures that the problem set covers common application scenarios of the target model and typical problem types that are prone to causing illusions, which is beneficial for subsequent targeted optimization of the target model.

[0053] In an optional embodiment, obtaining the initial answer set of the target model includes: inputting a question set into the target model, obtaining multiple answers corresponding to each question in the question set, and determining all answers corresponding to each question as one type of answer; summarizing all types of answers to obtain the initial answer set.

[0054] As an example, the target model answers each question in the initial question set multiple times, forming one type of answer (i.e., one question corresponds to multiple answers). If the question set contains Z questions, then Z types of answers are ultimately formed, constituting the initial answer set. The number of times each question is answered needs to be set according to the scenario requirements to cover answer performance from different perspectives.

[0055] This application embodiment uses a target model to answer each question in the question set, determines all answers corresponding to the question as one type of answer, and summarizes all types of answers to obtain an initial answer set. This can take into account the case that each question has multiple answers, and ensure that answers covering different perspectives are obtained.

[0056] In an optional embodiment, before inputting the input question set and initial answer set to the scoring model to obtain the average score of all types of answers in the target model and the overall average score of the initial answer set, the method further includes: setting the scoring rules of the scoring model; the setting of the scoring rules of the scoring model includes: splitting the question and answer to obtain split elements; wherein, the split elements of the question include the question background, question requirements, and question object, and the split elements of the answer include the answer object, answer context, and answer conclusion; vectorizing each split element to obtain the vector corresponding to each split element; determining the cosine similarity between the vector of the question object and the vector of the answer object; determining the Euclidean distance between the vector of the question background and the vector of the answer context; determining the Pearson correlation coefficient between the vector of the question requirements and the vector of the answer conclusion; and determining the score of the answer based on the cosine similarity, Euclidean distance, and Pearson correlation coefficient.

[0057] As an example, the dimensions for breaking down a problem are: background (the scenario or premise in which the problem exists), requirements (the core demands of the problem), and object (the subject that the problem focuses on); the dimensions for breaking down an answer are: object (the subject around which the answer revolves), context (the content that supports the conclusion), and conclusion (the response to the problem).

[0058] The scoring model scores according to the following steps: 1. Vectorize each part after splitting to obtain the corresponding vector; 2. Calculate the cosine similarity between the question object vector and the answer object vector. The higher the similarity, the higher the matching degree between the answer object and the question object, and the higher the score for this dimension; 3. Calculate the Euclidean distance between the question background vector and the answer context vector. The smaller the distance, the stronger the logical connection between the answer context and the question background, and the higher the score for this dimension; 4. Calculate the Pearson correlation coefficient between the question requirement vector and the answer conclusion vector. The larger the coefficient, the higher the fit between the conclusion and the question requirement, and the higher the score for this dimension; 5. Based on the scores of the above three dimensions, calculate the score of the answer by averaging. Optional, the maximum score is 100 points.

[0059] This application embodiment sets scoring rules based on the splitting elements of the question and the splitting elements of the answer. The scoring model scores all answers corresponding to each question output by the target model according to the scoring rules, which can ensure that the scoring model scores all answers corresponding to each question output by the target model reasonably.

[0060] In an optional embodiment, the step of inputting the question set and the initial answer set into the scoring model to obtain the average score of all types of answers in the target model and the overall average score of the initial answer set includes: inputting each question in the question set and all answers corresponding to the current question into the scoring model to obtain the scores of all answers output by the scoring model after scoring each answer according to the scoring rules; determining the average score of all answers corresponding to the current question based on the scores of all answers corresponding to the current question to obtain the average score of one type of answer; and determining the overall average score of the initial answer set based on the scores of all answers in the initial answer set.

[0061] Understandably, a question set is created by collecting questions that might cause the model to output illusory answers. Each question prompts the target optimization model to respond multiple times, resulting in multiple answers. Such multiple answers for a single question are called a category of answers. Scoring is done based on the question, but ultimately, we primarily focus on the average score of the entire answer dataset and the average score of each answer category.

[0062] The embodiments of this application can quickly obtain the average score of all types of answers and the overall average score of the initial answer set, which facilitates subsequent optimization of the target model based on the gradient descent optimization rule and the scoring results.

[0063] In an optional embodiment, the gradient descent-based optimization rule optimizes the target model based on the average score of all answer classes and the overall average score of the initial answer set. This includes: using all answer classes as the optimization space and selecting the class with the lowest average score as the current optimization direction; sorting the class with the lowest average score from lowest to highest score, and selecting the lowest-scoring first predetermined answer from the current class; inputting the first predetermined answer and its corresponding scoring criteria into the target model to obtain a second predetermined answer output by the target model after learning and re-answering the question corresponding to the current class; wherein the scoring criteria are generated by a scoring model, including scoring dimensions and scoring details; inputting the second predetermined answer into the scoring model to obtain the average score of the second predetermined answer; comparing the average score of the first predetermined answer and the average score of the second predetermined answer to determine whether the target model has been optimized; if the target model has been optimized, then using the second predetermined answer and its corresponding scoring criteria to optimize the target model, and replacing the unoptimized target model with the optimized target model; continuing to optimize the target model using the current class of answers until the average score of subsequent predetermined answers generated by the target model is less than or equal to the overall average score of the previous answer set. The preceding answer set includes all the answers output by the target model in the previous iteration, which can be understood as the preceding answers in this iteration. The overall average score of the preceding answer set is based on the scores of each answer output by the target model in the previous iteration.

[0064] In an optional embodiment, said comparing the average score of the answers of the first set quantity and the average score of the answers of the second set quantity to determine whether the target model is optimized comprises: if the average score of the answers of the first set quantity is less than the average score of the answers of the second set quantity, the target model is optimized; if the average score of the answers of the first set quantity is greater than or equal to the average score of the answers of the second set quantity, recording the number of boundary touches in the current optimization direction once, and continuing to optimize the target model by using the current type of answers.

[0065] By way of example, for the gradient descent-based optimization method, the optimization direction is selected first: taking all types of answers as the optimization space, taking the type of answers with the lowest average score as the current optimization direction, which is analogous to selecting the steepest gradient direction in gradient descent; specifically, sorting the answers of this type from the lowest score to the highest score, and selecting K answers with the lowest scores, wherein 0 < K ≤ the total number N of answers of this type.

[0066] Inputting the selected K low-score answers and their corresponding scoring basis into the target model, after the target model performs targeted learning based on the information, it re-answers the original question and generates new K' answers; wherein the scoring basis is generated by a scoring model and includes scoring details of each dimension. The scoring model scores the newly generated K' answers, calculates the average score thereof and compares it with the average score of the original K answers; if the new average score is higher than the old average score, it indicates that the hallucination of the target model is alleviated and the model is optimized; if the new average score is not improved, recording the number of boundary touches in the optimization direction once, and repeatedly selecting L answers with the lowest scores in the type of answers until the average score of the new answers is higher than the average score of the initial answers.

[0067] The embodiments of the present application can optimize the target model according to the scoring result through the gradient descent-based optimization rule, and can effectively alleviate the hallucination of the target model.

[0068] In an optional embodiment, the method further comprises step S104: S104, inputting a question set into the optimized target model to obtain a new answer set; inputting the new answer set into the scoring model for scoring and obtaining scoring basis; reselecting the type of answers with the lowest average score as the current optimization direction to optimize the target model until the termination condition for ending optimization is reached; saving the structure and parameters of the current target model to obtain the final optimized target model.

[0069] By way of example, after optimizing the target model, a new model with updated parameters of the target model is obtained, then the new model is used to generate a new answer set corresponding to the initial question set, and the scoring model is used for scoring to continue optimizing the target model.

[0070] This application embodiment treats all one type of answer as an optimization space, and takes the type of answer with the lowest average score as the direction of optimization. The number of answers that can be modified in this direction (among the type of answer with the lowest average score) is regarded as the step size of each step. This process is abstracted as a gradient descent process of the average score of each type of answer towards the optimal point. This process is repeated until the overall average score does not improve after a certain number of consecutive loops or reaches the upper limit of the specified number of loops, thus obtaining the optimal target model.

[0071] In an optional embodiment, the termination condition for reaching the end of optimization includes: reaching the termination condition for reaching the end of optimization when the number of boundary touches reaches a preset maximum value; and reaching the termination condition for reaching the end of optimization when the number of iterations reaches a preset maximum number of rounds.

[0072] Specifically, the overall average score of the answer dataset reaches the optimal boundary, that is, the number of times the boundary is touched in the overall optimization process of the model reaches the preset maximum value, indicating that there is no significant improvement if the optimization continues; or the number of iterations reaches the preset maximum limit, reaching the termination condition for the end of optimization.

[0073] For example, let the number of major loops be x (i.e., the number of times a new answer set is regenerated), and the maximum number of loops be X. Before starting the loop, i.e., when x=0, the target model initially answers z questions, obtaining an initial answer set containing z types of answers. Let the scoring model initially score the initial answer set. Let the overall average score of the initial answer set in the x-th loop be score_ans_sx, and the average score of the z-th answer type be score_ans_sx_z. Then the initial average score for each answer type is score_ans_s0_1=Y1, score_ans_s0_2=Y2, ..., score_ans_s0_z=Yz, thus obtaining the initial average score of the answer set as score_ans_s0=Y. The step size k for each optimization is selected, which is the k lowest-scoring questions in the category of answers with the lowest average score (selected based on model performance); the number of boundary touches in the optimization direction t_s (the number of consecutive times when the average score does not improve when optimizing the category of questions with the lowest score); the number of boundary touches in the overall optimization process t_m (the number of consecutive times when the average score of the newly generated answer dataset does not improve when optimizing the model); the maximum number of boundary touches in the optimization direction T_s; and the maximum number of boundary touches in the overall optimization process T_m.

[0074] Start the first cycle of optimization. That is, when x=0, t_s=0 and t_m=0. Select the class-z answer with the lowest average score from the class-z answers, select k answers with the lowest scores therefrom, and make the target model learn based on the scoring basis of these k answers, re-answer and score these k problems to obtain a new average score for class-z problems, that is, score_ans_s0_z =Yz'. Compare the scores of the new and old answers. If Yz>Yz', it indicates that the target model has not been optimized, the number of boundary touches in the optimization direction is increased by one, t_s=t_s+1, and the target model is made to re-answer these k problems again based on the scoring basis. Repeat the above cycle optimization process. Until Yz<Yz', which represents that the target model has been optimized, and the current model parameters are saved.

[0075] Make the optimized target model answer the problem set, at this time one cycle is finished, the number of cycles is increased by one, x=x+1, a new answer set and scores are generated, score_ans_s1=Y', if Y>Y', it indicates that the overall model has not been optimized, the number of boundary touches for overall optimization is increased by one, t_m= t_m+1; if Y<Y', it indicates that the model has been optimized. Continue to start the next optimization cycle process until t_m=T_m or x>X, then the cycle ends automatically, and the model optimization has reached the optimum.

[0076] By setting scientific and reasonable scoring rules, reasonable and effective scoring is performed on the answers of the target model, and an indicator that can measure the quality of model answers and detect the degree of hallucination of the model is obtained. The score of the model answer is based on the scoring model, and the scoring model will provide a scoring basis, so that the optimized target model can combine the scoring basis when optimizing answers, which improves the interpretability of the model hallucination mitigation process. It has significant advantages in terms of calculation and labor costs. In terms of calculation costs, it does not require large-scale fine-tuning of the model or reliance on high-performance computing clusters, and core relies on the scoring model to score existing answers. Each time optimization is only performed on the type of answer with the lowest average score, and only k low-score answers (small batch) are selected to drive the target model to learn, avoiding additional computing power consumption for model training; in terms of labor costs, it does not require constructing massive labeled training data, and relies on the scoring basis自带 by the scoring model to guide optimization, which reduces the demand for large-scale data annotation. The optimization process automatically iterates according to the gradient descent logic, and only the initial rules and termination conditions need to be set manually, which greatly reduces the input of professional labor.

[0077] By way of example, step 1: There is a large model Test Model with model hallucination, that is, the target model that needs hallucination mitigation, and the evaluation large model Qwen3. Prepare a question set containing 1000 questions; traverse these 1000 questions, make the Test Model answer each question 10 times, and obtain an answer data set containing 10000 answers, that is, a data set of 1000 classes of answers.

[0078] Step 2: Set the current number of cycles x=0, the maximum number of cycles X=100, the optimization step size k=1, the number of boundary touches t_s=0, t_m=0, and the maximum number of boundary touches T_s=5, T_m=5. Input the problem set and the initial answer set into the evaluation large language model Qwen3 in one-to-one correspondence. The evaluation large language model Qwen3 scores according to the scoring rules described above and provides a scoring basis. Calculate and obtain the overall average score of the initial answer dataset score_ans_s0=65, as well as the average score of each type of answers, which is score_ans_s0_z herein. At this time, the termination condition is judged: the number of boundary touches of the overall optimization process t_m<T_m, and the number of cycles x<X; the termination condition is not satisfied.

[0079] Step 3: Select the 63rd type of answers with the lowest average score (e.g., the score is 51) (63 refers to 10 answers corresponding to question No. 63) as the optimization direction. Sort the answers in the 63rd type of answers by score, and select k answers with the lowest score (k is set to 1 herein), as shown below: Question No. 63 is "How many dynasties are there in China?"; among the 63rd type of answers, 1 answer with the lowest score (score 47) is "There have been 25 major dynasties in Chinese history. Among them, the Xia Dynasty, the Shang Dynasty, the Western Zhou Dynasty, and the Eastern Zhou Dynasty laid the foundation for early civilization. After that, it successively experienced the Qin Dynasty, the Western Han Dynasty, the Eastern Han Dynasty, the Wei, Shu and Wu during the Three Kingdoms period, the Western Jin Dynasty, the Eastern Jin Dynasty, the Song, Qi, Liang and Chen of the Northern and Southern Dynasties, the Northern Wei, the Eastern Wei, the Western Wei, the Northern Qi and the Northern Zhou. Then came the Sui Dynasty, the Tang Dynasty, the Liang, Tang, Jin, Han and Zhou of the Five Dynasties and Ten Kingdoms, after which it directly entered the Yuan Dynasty, followed by the Ming Dynasty and the Qing Dynasty. Therefore, there are 25 dynasties in China".

[0080] The scoring process given by the evaluation large language model Qwen3 is as follows: Question splitting: Background: "Background of counting the number of dynasties within the scope of Chinese history", requirement: "Count and answer the number of Chinese dynasties", object: "Dynasties of China". Answer splitting: Object: "Dynasties in Chinese history", context: "The Xia Dynasty, the Shang Dynasty, the Western Zhou Dynasty, and the Eastern Zhou Dynasty laid the foundation for early civilization. After that, it successively experienced the Qin Dynasty, the Western Han Dynasty, the Eastern Han Dynasty, the Wei, Shu and Wu during the Three Kingdoms period, the Western Jin Dynasty, the Eastern Jin Dynasty, the Song, Qi, Liang and Chen of the Northern and Southern Dynasties, the Northern Wei, the Eastern Wei, the Western Wei, the Northern Qi and the Northern Zhou. Then came the Sui Dynasty, the Tang Dynasty, the Liang, Tang, Jin, Han and Zhou of the Five Dynasties and Ten Kingdoms, after which it directly entered the Yuan Dynasty, followed by the Ming Dynasty and the Qing Dynasty"; conclusion: "There are 25 dynasties in China"; Vectorization processing is performed respectively on "background, requirement, object" obtained from question splitting and "object, context, conclusion" obtained from answer splitting.

[0081] Calculation and scoring analysis across various dimensions: Cosine similarity calculation between the question and answer objects. Analysis: The answer object clearly points to "dynasties in Chinese history," consistent with the question object, but there is a chronological error in the context (skipping the Song Dynasty, etc.). Calculation result: Cosine similarity = 0.75 (minimum 0, maximum 1), score: 75 points. Euclidean distance calculation between background and context. Analysis: The question background is a dynastic count. Although the context lists dynasty names, there is a serious chronological error (directly following the Yuan Dynasty after the Tang Dynasty, omitting the Song, Liao, Jin, etc.), and regimes from the Three Kingdoms and Northern and Southern Dynasties periods are counted as separate dynasties, deviating from the conventional historical classification logic. Calculation result: Euclidean distance = 1.9 (less than 0.5 is a high fit, greater than 2 is almost a complete deviation), score: 40 points. The question asks for the calculation of the Pearson correlation coefficient between the given and the conclusion. Analysis: The question asks for the number of dynasties, and the conclusion states "25." However, the context contains instances of double counting (e.g., counting the Three Kingdoms of Wei, Shu, and Wu separately) and omissions (e.g., not including the Song Dynasty). Furthermore, historical consensus recognizes approximately 18 major dynasties. The conclusion's count is incorrect. Calculation result: Pearson correlation coefficient = 0.25 (minimum 0, maximum 1), score: 25 points. Overall score: (75+40+25)÷3≈47 points. Conclusion: This model's answer contains significant misleading information, resulting in a low overall score.

[0082] The lowest-scoring answer and the scoring criteria were input into the Test Model, which then re-answered question 63 based on this, resulting in a new answer: "There have been 22 major dynasties in Chinese history. Among them, the Xia, Shang, Western Zhou, and Eastern Zhou were the early dynasties, followed by the Qin, Western Han, and Eastern Han dynasties. The Three Kingdoms period is usually referred to as the Three Kingdoms. Then came the Western Jin, Eastern Jin, and Northern and Southern Dynasties, followed by the Sui, Tang, and Five Dynasties and Ten Kingdoms, then the Song, Yuan, Ming, and Qing dynasties. In addition, the Liao and Jin dynasties were not widely recorded. Therefore, there are a total of 22 dynasties in China."

[0083] Input question 63 and this new answer into the evaluation model Qwen3, and have it evaluate and score again. The new score is 58. Perform a conditional judgment: the average score of the new answer is 58, which is an improvement over the average score of the original answer of 47. Therefore, the model has been optimized to a certain extent. Retain the model parameters to obtain the optimized new Test Model.

[0084] Step 4: Use the new Test Model to iterate through the question set and answer the questions to obtain a new answer dataset. End one loop, x=0+1. Calculate the overall average score: score_ans_s1=73>score_ans_s0=65. Proceed to the next optimization loop and repeat the above steps.

[0085] Wherein, the 56th cycle: at this time, the number of cycles x=56, the average score of the overall answer score_ans_s56=94, judging the end condition: the average score of the previous round score_ans_s55=94 equals score_ans_s56, the number of boundary touches t_m in the overall optimization process is increased by 1, and at this time t_m=5 equals the maximum number of boundary touches T_m for overall optimization, the number of cycles x<X, and the end condition is satisfied.

[0086] The structure and parameters of the Test Model at this time are the optimal ones, which is the target model obtained by the model hallucination mitigation solution.

[0087] Please refer to Figure 2 , Figure 2 is a schematic structural diagram of a model hallucination mitigation apparatus provided by the second embodiment of the present application. The model hallucination mitigation apparatus provided by the second embodiment of the present application comprises: a data acquisition module 201, a score generation module 202 and a model optimization module 203; the data acquisition module 201 is configured to acquire a question set and an initial answer set of a target model; the score generation module 202 is configured to input the question set and the initial answer set into a scoring model to obtain the average score of all classes of answers of the target model and the average score of the whole initial answer set; wherein the average score of each class of answers is obtained according to the scores of all answers corresponding to each question in the question set, and the average score of the whole initial answer set is obtained according to the scores of all answers in the initial answer set; the model optimization module 203 is configured to optimize the target model according to the average score of all classes of answers and the average score of the whole initial answer set based on the gradient descent optimization rule.

[0088] In an optional embodiment, said acquiring the question set of the target model comprises: acquiring questions corresponding to the application scenario and business field of the target model to obtain the question set.

[0089] In an optional embodiment, said acquiring the initial answer set of the target model comprises: inputting the question set into the target model to obtain a plurality of answers corresponding to each question in the question set, and determining all answers corresponding to each question as one class of answers; summarizing all classes of answers to obtain the initial answer set.

[0090] In an optional embodiment, the scoring generation module 202 is further configured to set the scoring rules of the scoring model before inputting the input question set and initial answer set to the scoring model to obtain the average score of all types of answers in the target model and the overall average score of the initial answer set; the setting of the scoring rules of the scoring model includes: splitting the questions and answers to obtain split elements; wherein, the split elements of the questions include question background, question requirements, and question objects, and the split elements of the answers include answer objects, answer context, and answer conclusions; vectorizing each split element to obtain the vector corresponding to each split element; determining the cosine similarity between the vector of the question object and the vector of the answer object; determining the Euclidean distance between the vector of the question background and the vector of the answer context; determining the Pearson correlation coefficient between the vector of the question requirements and the vector of the answer conclusion; and determining the score of the answer based on the cosine similarity, Euclidean distance, and Pearson correlation coefficient.

[0091] In an optional embodiment, the step of inputting the question set and the initial answer set into the scoring model to obtain the average score of all types of answers in the target model and the overall average score of the initial answer set includes: inputting each question in the question set and all answers corresponding to the current question into the scoring model to obtain the scores of all answers output by the scoring model after scoring each answer according to the scoring rules; determining the average score of all answers corresponding to the current question based on the scores of all answers corresponding to the current question to obtain the average score of one type of answer; and determining the overall average score of the initial answer set based on the scores of all answers in the initial answer set.

[0092] In an optional embodiment, the gradient descent-based optimization rule optimizes the target model based on the average score of all answer classes and the overall average score of the initial answer set. This includes: using all answer classes as the optimization space and selecting the class with the lowest average score as the current optimization direction; sorting the class with the lowest average score from lowest to highest score, and selecting the lowest-scoring first predetermined answer from the current class; inputting the first predetermined answer and its corresponding scoring criteria into the target model to obtain a second predetermined answer output by the target model after learning and re-answering the question corresponding to the current class; wherein the scoring criteria are generated by a scoring model, including scoring dimensions and scoring details; inputting the second predetermined answer into the scoring model to obtain the average score of the second predetermined answer; comparing the average score of the first predetermined answer and the average score of the second predetermined answer to determine whether the target model has been optimized; if the target model has been optimized, then using the second predetermined answer and its corresponding scoring criteria to optimize the target model, and replacing the unoptimized target model with the optimized target model; continuing to optimize the target model using the current class of answers until the average score of subsequent predetermined answers generated by the target model is less than or equal to the overall average score of the previous answer set.

[0093] In an optional embodiment, comparing the average score of the first set quantity answer and the average score of the second set quantity answer to determine whether the target model has been optimized includes: if the average score of the first set quantity answer is less than the average score of the second set quantity answer, then the target model has been optimized; if the average score of the first set quantity answer is greater than or equal to the average score of the second set quantity answer, then the number of boundary touches in one optimization direction is recorded, and the target model is continued to be optimized using the current type of answer.

[0094] In an optional embodiment, the device further includes a loop execution module, configured to perform the following operations: inputting a question set into the optimized target model to obtain a new answer set; inputting the new answer set into the scoring model for scoring and obtaining the scoring basis; reselecting the answer with the lowest average score as the current optimization direction to optimize the target model until the termination condition for optimization is met; and saving the structure and parameters of the current target model to obtain the final optimized target model.

[0095] In an optional embodiment, the termination condition for reaching the end of optimization includes: reaching the termination condition for reaching the end of optimization when the number of boundary touches reaches a preset maximum value; and reaching the termination condition for reaching the end of optimization when the number of iterations reaches a preset maximum number of rounds.

[0096] The specific implementation process of the functions and roles of each module in the above-mentioned device can be found in the implementation process of the corresponding steps in the method described in the first embodiment of this application, and will not be repeated here.

[0097] The third embodiment of this application provides a computer program product, which includes instructions that, when executed by a computer, cause the computer to perform the method described in the first embodiment of this application and achieve the same beneficial effects.

[0098] The methods described in the first embodiment of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the various embodiments of this application are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, a core network device, an OAM (Open Application Model), or other programmable devices.

[0099] Computer programs or instructions can be stored in or transferred from one computer-readable storage medium to another. For example, a computer program or instructions can be transferred from one website, computer, server, or data center to another via wired or wireless means. A computer-readable storage medium can be any usable medium that a computer can access, or a data storage device such as a server or data center that integrates one or more usable media. Usable media can be magnetic media, such as floppy disks, hard disks, and magnetic tapes; optical media, such as digital video discs; or semiconductor media, such as solid-state drives. The computer-readable storage medium can be volatile or non-volatile, or may include both types.

[0100] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of this application. The fourth embodiment of this application provides an electronic device 30, including a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301; when the processor 301 executes the computer program, it implements the method described in the first embodiment of this application and can achieve the same beneficial effects.

[0101] When the processor 301 reads a computer program from the memory 302 via the bus 303 and executes the computer program, it can implement any of the methods described in the first embodiment of this application.

[0102] Processor 301 can process digital signals and may include various computing architectures. For example, it may be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 301 may be a microprocessor.

[0103] The memory 302 can be used to store instructions executed by the processor 301 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 301 of this disclosure embodiment can be used to execute instructions in the memory 302 to implement the method described in the first embodiment of this application. The memory 302 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.

[0104] The fifth embodiment of this application provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to perform the method described in the first embodiment of this application, and can achieve the same beneficial effects.

[0105] The method described in the first embodiment of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the various embodiments of this application are executed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, a core network device, an OAM (Open Application Model), or other programmable devices.

[0106] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0107] In summary, embodiments of this application provide a model illusion mitigation method, program product, device, and medium. The model illusion mitigation method includes: obtaining a question set and an initial answer set for a target model; inputting the question set and the initial answer set into a scoring model to obtain the average score of all answer classes of the target model and the overall average score of the initial answer set; wherein, the average score of each answer class is obtained based on the scores of all answers corresponding to each question in the question set, and the overall average score of the initial answer set is obtained based on the scores of all answers in the initial answer set; and optimizing the target model based on gradient descent optimization rules, according to the average score of all answer classes and the overall average score of the initial answer set. This application embodiment inputs the question set and initial answer set of the target model into the scoring model to obtain the average score of all answer classes in the target model and the overall average score of the initial answer set. Based on the gradient descent optimization rule, the target model is optimized according to the average score of all answer classes and the overall average score of the initial answer set. It can efficiently obtain the scoring results of each answer class and the overall scoring results of the initial answer set using the scoring model. Based on the gradient descent optimization rule, the target model is comprehensively optimized by combining these scoring results, thereby alleviating the illusion of the target model and reducing the cost of computational resources and human resources invested in the target model.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0109] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0110] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for alleviating model hallucinations, characterized in that, The method includes: Obtain the question set and initial answer set for the target model; Input the question set and the initial answer set into the scoring model to obtain the average score of all answer classes in the target model and the average score of the initial answer set as a whole; where the average score of each answer class is obtained based on the scores of all answers corresponding to each question in the question set, and the average score of the initial answer set as a whole is obtained based on the scores of all answers in the initial answer set; The optimization rule based on gradient descent optimizes the target model according to the average score of all answer classes and the overall average score of the initial answer set. The gradient descent-based optimization rule optimizes the target model based on the average score of all answer classes and the overall average score of the initial answer set, including: The optimization space is defined by all types of answers, and the current optimization direction is the type of answer with the lowest average score. Sort the answers with the lowest average score from lowest to highest, and select the first set quantity answer with the lowest score in the current category. Input the first set-value answer and the corresponding scoring criteria into the target model, and obtain the second set-value answer output by the target model after learning and re-answering the question corresponding to the current class of answers; wherein, the scoring criteria are generated by the scoring model, and the scoring criteria include scoring dimensions and scoring details; Input the second set of answers into the scoring model to obtain the average score of the second set of answers; Compare the average score of the first setpoint answer with the average score of the second setpoint answer to determine whether the target model has been optimized; If the target model is optimized, the second set quantity answer and the corresponding scoring criteria are used to optimize the target model, and the optimized target model replaces the unoptimized target model. Continue to optimize the target model using the current type of answer until the average score of the subsequent set of answers generated by the target model is less than or equal to the overall average score of the previous answer set; The method further includes: selecting and numbering a set of questions highly relevant to the business based on the application scenarios and business domains of the target model; wherein the set of questions covers the application scenarios of the target model and the types of questions that produce hallucinations.

2. The method according to claim 1, characterized in that, The problem set for obtaining the target model includes: Obtain the questions corresponding to the application scenarios and business domains of the target model to obtain a question set.

3. The method according to claim 1, characterized in that, The process of obtaining the initial answer set for the target model includes: Input a set of questions into the target model, obtain multiple answers for each question in the set of questions, and classify all answers for each question into one type of answer; Summarize all the answers to obtain the initial answer set.

4. The method according to claim 1, characterized in that, Before inputting the input question set and initial answer set into the scoring model to obtain the average score of all classes of answers in the target model and the overall average score of the initial answer set, the following steps are also included: Set the scoring rules for the scoring model; The scoring rules for setting the scoring model include: The question and answer are broken down into their constituent elements. The constituent elements of the question include the question background, question requirements, and question object. The constituent elements of the answer include the answer object, answer context, and answer conclusion. Each splitting element is vectorized to obtain the vector corresponding to each splitting element; Determine the cosine similarity between the vector of the question object and the vector of the answer object; Determine the Euclidean distance between the vectors representing the problem context and the vectors representing the answer context; Determine the Pearson correlation coefficient between the vector required by the question and the vector of the answer conclusion; The score for the answer is determined based on cosine similarity, Euclidean distance, and Pearson correlation coefficient.

5. The method according to claim 1, characterized in that, The input question set and initial answer set are fed into the scoring model to obtain the average score of all answer classes in the target model and the overall average score of the initial answer set, including: Input each question in the question set and all the answers corresponding to the current question into the scoring model, and obtain the scores of all answers output by the scoring model after scoring each answer according to the scoring rules; Based on the scores of all answers to the current question, determine the average score of all answers to the current question to obtain the average score of a category of answers; Determine the average score of the initial answer set based on the scores of all answers in the initial answer set.

6. The method according to claim 1, characterized in that, The step of comparing the average score of the first setpoint answer and the average score of the second setpoint answer to determine whether the target model has been optimized includes: If the average score of the first set quantity answers is less than the average score of the second set quantity answers, then the target model is optimized; If the average score of the first set quantity answer is greater than or equal to the average score of the second set quantity answer, then record the number of boundary touches in one optimization direction, and continue to use the current type of answer to optimize the target model.

7. The method according to claim 6, characterized in that, Also includes: Input the question set into the optimized target model to obtain a new answer set; Input a new set of answers into the scoring model for scoring and obtain the scoring criteria; The lowest average score of the answer type is selected as the current optimization direction, and the target model is optimized until the termination condition for optimization is met. Save the structure and parameters of the current target model to obtain the final optimized target model.

8. The method according to claim 7, characterized in that, The termination conditions for reaching the end of optimization include: When the number of boundary touches reaches the preset maximum value, the optimization ends. When the number of iterations reaches the preset maximum limit, the optimization ends.

9. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; when the processor executes the computer program, it implements the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 8.

Images