Corpus optimization method and device for generative engine, storage medium and electronic equipment

Abstract

The application relates to a corpus optimization method and device of a generative engine, a storage medium and an electronic device. The method comprises the following steps: acquiring a corpus and a test question subjected to disturbance processing; inputting the test question into multiple generative engines respectively to obtain multiple answers; determining a robustness representation value based on the corpus and the multiple answers; in the case that the robustness representation value is less than a preset value, rewriting the corpus based on the multiple answers to obtain an updated corpus, so as to improve the consistency degree of answers generated by the multiple generative engines based on the corpus and the consistency degree of the answers generated by the multiple generative engines based on the corpus; and repeatedly performing the test step, the determination step and the rewriting step for at least one round until the robustness representation value is not less than the preset value. The method can reduce the answer deviation caused by the noise of a user input question and the difference between engines in the application of the generative engine.

Description

Technical Field

[0001] This application relates to the field of generative artificial intelligence technology, and in particular to a corpus optimization method, apparatus, storage medium and electronic device for a generative engine. Background Technology

[0002] With the explosive growth of generative artificial intelligence (AIGC) technology, generative engines (such as large language models) are widely used in various fields. Currently, the following problems exist in the practical application of generative engines: Firstly, the questions input by users are noisy (e.g., contain typos, are colloquial, omit some information, or are logically incomplete), which causes the answers generated by the generative engine based on such questions to deviate from the corpus. Secondly, different generative engines have different architectures and training logics. Therefore, the answers generated by different types of generative engines when analyzing the same corpus for the same question may have discrepancies.

[0003] In summary, reducing the noise in user input questions and the answer bias caused by cross-engine differences has become an urgent problem to be solved in generative engine applications. Summary of the Invention

[0004] This application provides a corpus optimization method, apparatus, computer-readable storage medium, and electronic device for generative engines, to reduce answer bias caused by noise in user input questions and cross-engine differences in generative engine applications.

[0005] Firstly, this application provides a corpus optimization method for generative engines. The method includes: an acquisition step, acquiring corpus and perturbation-processed test questions; a testing step, inputting the test questions into multiple generative engines to obtain multiple answers, each answer corresponding to a generative engine, the generative engine generating the answer based on the corpus; a determination step, determining a robustness representation value based on the corpus and the multiple answers, the robustness representation value representing the consistency between the multiple answers and the corpus, and the consistency of the multiple answers; a rewriting step, rewriting the corpus based on the multiple answers when the robustness representation value is less than a preset value, obtaining an updated corpus to improve the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus, and the consistency of the answers generated by the multiple generative engines based on the corpus; and a repetition step, sequentially executing the testing step, the determination step, and the rewriting step at least once until the robustness representation value is not less than the preset value, the updated corpus being used to guide the generative engines in generating answers to the questions.

[0006] Optionally, there are multiple test questions, and the corpus consists of multiple text fragments. One test question corresponds to one text fragment. The generative engine is used to generate answers based on the text fragments corresponding to the test questions. Based on the corpus and the multiple answers, a robustness representation value is determined, including: for each test question, based on the text fragments corresponding to the test question and the multiple answers, determining a first target content representation value for the test question, where the first target content representation value represents the degree of consistency between the content of the text fragments corresponding to the test question and the content of the multiple answers; for each test question, based on the text fragments corresponding to the test question and the multiple answers, determining a target fact claim representation value for the test question, where the target fact claim representation value represents the degree of consistency between the fact claims of the text fragments corresponding to the test question and the fact claims of the multiple answers; for each test question, based on the text fragments corresponding to the test question and the multiple answers, determining a target logic representation value for the test question, where the target logic representation value represents the consistency between the logic of the text fragments corresponding to the test question and the logic of the multiple answers. The consistency of the content of the multiple answers is determined based on the following: For each test question, a second target content representation value is determined based on the multiple answers, whereby the second target content representation value represents the consistency of the content of the multiple answers; the average of the first target content representation values ​​corresponding to all test questions is calculated to obtain a first average content representation value; the average of the target fact claim representation values ​​corresponding to all test questions is calculated to obtain an average fact claim representation value; the average of the target logic representation values ​​corresponding to all test questions is calculated to obtain an average logic representation value; the average of the second target content representation values ​​corresponding to all test questions is calculated to obtain a second average target content representation value; the first average content representation value, the average fact claim representation value, the average logic representation value, and the second average target content representation value are then normalized sequentially to obtain a first standard content representation value, a standard fact claim representation value, a standard logic representation value, and a second standard content representation value; a weighted sum is then performed on the first standard content representation value, the standard fact claim representation value, the standard logic representation value, and the second standard content representation value to obtain the robustness representation value.

[0007] Optionally, for each test question, based on the text segment corresponding to the test question and the multiple answers, a first target content representation value corresponding to the test question is determined, including: for each test question, obtaining the text vector of the text segment corresponding to the test question and the text vector of the multiple answers; for each test question, calculating the cosine similarity between the text vector of each answer and the text vector of the text segment corresponding to the test question to obtain an initial content representation value corresponding to each answer, wherein the initial content representation value represents the degree of consistency between the content of the answer and the content of the text segment corresponding to the test question; for each test question, calculating the average of the initial content representation values ​​corresponding to all the answers to obtain the first target content representation value corresponding to the test question.

[0008] Optionally, for each test question, based on the text fragment corresponding to the test question and the multiple answers, a target fact claim representation value corresponding to the test question is determined, including: for each test question and each answer, inputting the answer and the text fragment corresponding to the test question into a trained natural language inference model to obtain a fact claim score corresponding to the answer, wherein the fact claim score represents the degree of consistency between the fact claim of the answer and the fact claim of the text fragment corresponding to the test question; for each test question, calculating the fact claim scores corresponding to all the answers to obtain the target fact claim representation value corresponding to the test question.

[0009] Optionally, for each test question, based on the text fragment corresponding to the test question and the multiple answers, a target logical representation value corresponding to the test question is determined, including: setting a thought prompt chain for the large language model, the thought prompt chain being configured to guide the large language model to sequentially execute: constructing a logical chain for the text fragment, constructing a logical chain for the answer, and scoring based on the consistency between the logical chain of the text fragment and the logical chain of the answer; for each test question and each answer, inputting the answer and the text fragment corresponding to the test question into the large language model to obtain a logical score corresponding to the answer; for each test question, calculating the logical scores corresponding to all the answers to obtain the target logical representation value corresponding to the test question.

[0010] Optionally, for each test question, based on the plurality of answers, a second target content representation value corresponding to the test question is determined, including: for each test question, obtaining the text vectors of the plurality of answers; for each test question, constructing a content similarity matrix based on the text vectors of the plurality of answers; for each test question, calculating the variance of the off-diagonal elements of the content similarity matrix to obtain the second target content representation value corresponding to the test question.

[0011] Optionally, the rewriting step, when the robustness representation value is less than a preset value, rewrites the corpus based on the multiple answers to obtain an updated corpus, including: selecting multiple incorrect answers from the multiple answers, wherein the multiple incorrect answers are answers that satisfy at least one type of preset condition, the types of preset conditions include: a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition, wherein the first preset condition is that the initial content representation value corresponding to the answer is less than a first preset content representation value, the second preset condition is that the factual claim score corresponding to the answer is less than a first preset score, the third preset condition is that the logical score corresponding to the answer is less than a second preset score, and the fourth preset condition is that the second target content representation value corresponding to the test question to which the answer belongs is greater than a second preset content representation value; determining the text fragment corresponding to the incorrect answer as a vulnerable text fragment; for each vulnerable text fragment, determining a rewriting strategy for the vulnerable text fragment based on the type of preset condition satisfied by the incorrect answer corresponding to the vulnerable text fragment; and for each vulnerable text fragment, rewriting the vulnerable text fragment based on the rewriting strategy to obtain the updated corpus.

[0012] Secondly, this application provides a corpus optimization device for a generative engine. The device includes: an acquisition unit for acquiring corpus and perturbation-processed test questions; a testing unit for inputting the test questions into multiple generative engines to obtain multiple answers, each answer corresponding to a generative engine, the generative engine generating the answer based on the corpus; a determination unit for determining a robustness representation value based on the corpus and the multiple answers, the robustness representation value representing the consistency between the multiple answers and the corpus, and the consistency of the multiple answers; a rewriting unit for rewriting the corpus based on the multiple answers when the robustness representation value is less than a preset value, to obtain an updated corpus, thereby improving the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus, and the consistency of the answers generated by the multiple generative engines based on the corpus; and a repetition unit for sequentially calling the testing unit, the determination unit, and the rewriting unit at least once until the robustness representation value is not less than the preset value, the updated corpus being used to guide the generative engines in generating answers to the questions.

[0013] Thirdly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the generative engine corpus optimization methods described above.

[0014] Fourthly, this application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any of the generative engine corpus optimization methods described above.

[0015] In this embodiment, the following steps are included: an acquisition step, acquiring corpus and perturbation-processed test questions; a testing step, inputting the test questions into multiple generative engines to obtain multiple answers; a determination step, determining robustness representation values ​​based on the corpus and multiple answers; and a rewriting step, where if the robustness representation value is less than a preset value, it is determined that the consistency between the answers generated by the generative engines based on the corpus and the corpus is low when the perturbation-processed test questions are input, and the consistency between the answers generated by multiple generative engines based on the corpus is low. Therefore, the corpus is rewritten to improve the consistency between the answers generated by the generative engines based on the corpus and the corpus, as well as the consistency between the answers generated by different generative engines based on the corpus. The process is repeated, executing the testing step, determination step, and rewriting step at least once, until the robustness representation value is not less than the preset value. This ensures that the consistency between the answers generated by the generative engines based on the corpus and the corpus, as well as the consistency between the answers generated by different generative engines based on the corpus, meets the requirements, thereby reducing answer bias caused by user input question noise and cross-engine differences in the generative engine application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1 A flowchart illustrating a corpus optimization method for a generative engine provided in an embodiment of this application; Figure 2 This is a structural block diagram of a generative engine corpus optimization device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0022] To reduce the answer bias caused by noise in user input questions and differences across engines in generative engine applications, this application provides a corpus optimization method for generative engines, a corpus optimization device for generative engines, a computer-readable storage medium, and an electronic device.

[0023] Figure 1 A corpus optimization method for a generative engine provided in this application embodiment includes: Step S101: Obtain the corpus and the perturbation-processed test questions; For example, the corpus can be product manuals, brand marketing copy, enterprise knowledge base entries, technical white papers, service introductions, etc.

[0024] For example, the corpus consists of multiple text fragments, each of which answers one or more test questions. For instance, the corpus could be a user manual for an air purifier, containing the text: "This air purifier uses a double-layer filter structure, which can effectively remove PM2.5 and formaldehyde. It is recommended to replace the filter every 6-12 months. Do not block the air inlet when turning it on. It is not suitable for flammable or explosive environments." The corpus consists of text fragment 1: "This air purifier uses a double-layer filter structure, which can effectively remove PM2.5 and formaldehyde," text fragment 2: "It is recommended to replace the filter every 6-12 months," and text fragment 3: "Do not block the air inlet when turning it on." The test question answered by text fragment 1 could be "How effective is this double-layer filter air purifier?", the test question answered by text fragment 2 could be "What is the recommended replacement cycle for the filter?", and the test question answered by text fragment 3 could be "What precautions should be taken when turning it on?". When dividing the text fragments, it is sufficient to ensure that each text fragment answers at least one test question. This application does not further limit the method of dividing the text fragments.

[0025] For example, the perturbation-processed test question is used to simulate a noisy question input by a user. The perturbation process can include paraphrasing, syntactic noise injection, content dilution, and round-trip translation perturbation. Taking a corpus of an air purifier's instruction manual as an example, for paraphrasing, for instance, the original test question is "How effective is this dual-layer filter air purifier?", and the paraphrased test question is "Is this dual-layer filter air purifier good at purifying?". For syntactic noise injection, for instance, the original test question is "How effective is this air purifier?", and the syntactic noise-injected test question is the abbreviated form "How effective is this air purifier?". The inverted sentence "How effective is your dual-layer filter air purifier?" and the inclusion of common typos "How effective is the purification of this dual-layer filter air purifier?" are used for content dilution (removing key information). For example, the original test question was "How often do I need to replace the filter of this air purifier?", while the diluted test question is "Does the filter of this air purifier need to be replaced?". The back-and-forth translation process (Chinese to English, English to Chinese) is also used. For example, the original test question was "How often do I need to replace the filter of this air purifier?", while the translated test question is "Approximately how often do I need to replace the filter of this air purifier?".

[0026] Step S102, the testing step, input the above test questions into multiple generative engines respectively to obtain the above multiple answers; The above answers correspond one-to-one with the above generative engine, which is used to generate the above answers based on the above corpus. For example, the generative engine parses text fragments related to the test question from the corpus to generate an answer.

[0027] Step S103: Determine the robustness representation value based on the above corpus and the above multiple responses; Among them, the robustness characterization value represents the degree of consistency between the above multiple answers and the above corpus, as well as the degree of consistency between the above multiple answers; Step S104, rewriting step: when the robustness representation value is less than the preset value, the corpus is rewritten based on the multiple answers to obtain an updated corpus, so as to improve the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus and the consistency between the answers generated by the multiple generative engines based on the corpus. For example, the consistency between the updated corpus and the answers generated by the multiple generative engines based on the updated corpus is greater than the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus, and the consistency between the answers generated by the multiple generative engines based on the updated corpus is greater than the consistency between the answers generated by the multiple generative engines based on the corpus.

[0028] Step S105: Repeat the steps, and execute the above test steps, the above determination steps and the above rewrite steps at least once in sequence until the above robustness characterization value is not less than the above preset value. The updated corpus is used to guide the generative engine in generating answers to questions.

[0029] Through the above embodiments, the following steps are implemented: First, the corpus and perturbated test questions are acquired. Second, the test questions are input into multiple generative engines to obtain multiple answers. Third, a robustness representation value is determined based on the corpus and the multiple answers. Fourth, a rewriting step is performed where, if the robustness representation value is less than a preset value, it is determined that the consistency between the answers generated by the generative engines based on the corpus and the corpus is low, and the consistency between the answers generated by multiple generative engines based on the corpus is also low. Therefore, the corpus is rewritten to improve the consistency between the answers generated by the generative engines based on the corpus and the corpus, as well as the consistency between the answers generated by different generative engines based on the corpus. Fifth, the steps are repeated, executing the test step, determination step, and rewriting step at least once, until the robustness representation value is not less than the preset value. This ensures that the consistency between the answers generated by the generative engines based on the corpus and the corpus, as well as the consistency between the answers generated by different generative engines based on the corpus, meets the requirements, thereby reducing answer bias caused by user input question noise and cross-engine differences in the generative engine application.

[0030] In an optional embodiment, there are multiple test questions, the corpus consists of multiple text segments, one test question corresponds to one text segment, and the generative engine is used to generate answers based on the text segments corresponding to the test questions. Step S103 can be implemented as follows: Step S1031: For each of the above test questions, based on the text fragment corresponding to the above test question and the above multiple answers, determine the first target content representation value corresponding to the above test question. The first target content representation value represents the degree of consistency between the content of the text fragment corresponding to the above test question and the content of the above multiple answers. Step S1032: For each of the above test questions, based on the text fragment corresponding to the above test question and the above multiple answers, determine the target fact claim representation value corresponding to the above test question. The target fact claim representation value represents the degree of consistency between the fact claim of the text fragment corresponding to the above test question and the fact claim of the above multiple answers. For example, when the content of a text fragment is highly consistent with the content of an answer, the consistency between the factual claims in the text fragment and the factual claims in the answer may be low. For instance, the text fragment is "The air purifier filter should be replaced every 6 months," and the answer is "The air purifier filter should be replaced every 12 months." The content of the two is highly consistent, but their factual claims are completely inconsistent. Conversely, when the content of a text fragment is low, the consistency between the factual claims in the text fragment and the answer may be high. For instance, the text fragment is "This air purifier filter should be replaced every 6 months," and the answer is "For this air purifier, it is recommended to replace the filter every six months to ensure its effectiveness." The content of the two is low, but their factual claims are completely consistent.

[0031] Step S1033: For each of the above test questions, based on the text fragment corresponding to the above test question and the above multiple answers, determine the target logical representation value corresponding to the above test question. The target logical representation value represents the degree of consistency between the logic of the text fragment corresponding to the above test question and the logic of the above multiple answers. For example, logic refers to the reasoning path formed by the sequence, causal relationship, conditional relationship, and execution flow between various information units in a text.

[0032] Step S1034: For each of the above test questions, based on the above multiple answers, determine the second target content representation value corresponding to the above test question. The second target content representation value represents the degree of consistency of the content of the above multiple answers. Step S1035: Calculate the average value of the first target content representation value corresponding to all the above test questions to obtain the first average content representation value; calculate the average value of the target fact claim representation value corresponding to all the above test questions to obtain the average fact claim representation value; calculate the average value of the target logic representation value corresponding to all the above test questions to obtain the average logic representation value; calculate the average value of the second target content representation value corresponding to all the above test questions to obtain the second average target content representation value. Step S1036: The first average content representation value, the average fact claim representation value, the average logic representation value, and the second average target content representation value are normalized sequentially to obtain the first standard content representation value, the standard fact claim representation value, the standard logic representation value, and the second standard content representation value. For example, the first average content representation value, the aforementioned average factual claim representation value, the aforementioned average logical representation value, and the aforementioned second average target content representation value are normalized to the same preset value range.

[0033] Step S1037: The first standard content representation value, the standard fact claim representation value, the standard logic representation value, and the second standard content representation value are weighted and summed to obtain the robustness representation value.

[0034] In this embodiment, for each test question, a representation value is calculated based on four evaluation dimensions: the consistency of content between the corresponding text fragment and the answers from multiple generative engines, the consistency of factual claims, the consistency of logic, and the consistency of content between the generative engine's answer. This avoids misjudgments of the consistency between the text fragment and the answer due to a single evaluation dimension. By averaging the various representation values ​​for all test questions, the random bias of the various representation values ​​for individual test questions can be eliminated, making the evaluation results more representative and statistically significant. Normalizing the various average representation values ​​unifies the units and numerical ranges of data from different dimensions, ensuring the comparability of the various average representation values ​​in subsequent weighted calculations. The final robustness representation value is obtained by weighted summation of the normalized average representation values. The weights of each evaluation dimension can be flexibly adjusted according to actual application needs to adapt to different scenarios.

[0035] In an optional embodiment, step S1031 above can be implemented as follows: For each of the above test questions, obtain the text vector of the text segment corresponding to the above test question and the text vector of the above multiple answers; For example, the text fragment corresponding to the test question is input into the Sentence-BERT model (SentenceBidirectional Encoder Representations from Transformers) to obtain the text vector of the text fragment corresponding to the test question, and the answer is input into the Sentence-BERT model to obtain the text vector of the answer.

[0036] For each of the above test questions, the cosine similarity between the text vector of each of the above answers and the text vector of the text segment corresponding to the above test question is calculated to obtain the initial content representation value corresponding to each of the above answers. The initial content representation value represents the degree of consistency between the content of the above answer and the content of the text segment corresponding to the above test question. For each of the above test questions, calculate the average of the initial content representation values ​​corresponding to all the above answers to obtain the first target content representation value corresponding to the above test question.

[0037] In this embodiment, for each test question, the average of the initial content representation values ​​corresponding to the answers of multiple generative engines is calculated to obtain the first target content representation value. The first target content representation value is used to represent the consistency between the content of the text fragment corresponding to the test question and the content of the answers of multiple generative engines. This can offset the random deviation of the initial content representation value corresponding to a single answer. The first target content representation value has stability and statistical representativeness.

[0038] In an optional embodiment, step S1032 above can be implemented as follows: For each of the above test questions and each of the above answers, the text fragments corresponding to the above answers and the above test questions are input into the trained natural language reasoning model to obtain the fact claim score corresponding to the above answer. The fact claim score represents the degree of consistency between the fact claim of the above answer and the fact claim of the text fragment corresponding to the above test question. For example, a Natural Language Inference (NLI) model is used to determine the semantic relationship between two texts, such as implication, contradiction, or neutrality. Implication indicates that the factual claims between the two texts are highly consistent, contradiction indicates that the factual claims between the two texts are completely contradictory, and neutrality indicates that the factual claims between the two texts are not directly related. A natural language inference model with a softmax layer can be used. The softmax layer outputs the probability distribution of the three types of relationships. The probability of implication is used as the factual claim score. The natural language inference model is trained using training data, which includes: text fragments, the corresponding answers to the text fragments, and the factual claim scores corresponding to the answers.

[0039] For each of the above test questions, calculate the fact claim scores corresponding to all the above answers to obtain the target fact claim representation value corresponding to the above test questions.

[0040] In this embodiment, for each test question, the average of the fact claim scores corresponding to the answers of multiple generative engines is calculated to obtain the target fact claim representation value. The target fact claim representation value is used to represent the consistency between the content of the text fragment corresponding to the test question and the content of the answers of multiple generative engines. This can offset the random bias of the fact claim score corresponding to a single answer. The target fact claim representation value has stability and statistical representativeness.

[0041] In an optional embodiment, step S1033 above can be implemented as follows: A thought prompt chain is set for the Large Language Model (LLM). The thought prompt chain is configured to guide the Large Language Model to execute the following sequentially: the logical chain of breaking down text fragments, the logical chain of breaking down answers, and the consistency between the logical chain of text fragments and the logical chain of answers. For example, a logical chain refers to the reasoning path formed by the sequence, causal relationship, conditional relationship, and execution flow between various information units in a text. For instance, a text fragment might read, "When the fan starts, the air first passes through a pre-filter to remove large dust particles, then enters a HEPA filter to remove PM2.5 and pollen, and subsequently passes through an activated carbon filter to adsorb formaldehyde and odors. The purified air is then discharged from the outlet." Let's first extract the information units: fan starts, air passes through a pre-filter, the pre-filter removes large dust particles, air enters a HEPA filter to remove PM2.5 and pollen, then passes through an activated carbon filter to adsorb formaldehyde and odors, and the purified air is discharged from the outlet. The logic chain of this text fragment includes: Trigger condition: fan starts; Execution step 1: Air → pre-filter → filters large dust particles; Execution step 2: Air → HEPA filter → removes PM2.5 and pollen; Execution step 3: Air → activated carbon filter → adsorbs formaldehyde and odors; Final result: air purification completed → discharged from the air outlet.

[0042] For each of the above test questions and each of the above answers, input the text fragments corresponding to the above answers and test questions into the above large language model to obtain the logical score corresponding to the above answers; For each of the above test questions, calculate the logical score corresponding to all the above answers to obtain the target logical representation value corresponding to the above test question.

[0043] In this embodiment, for each test question, the average of the logical scores corresponding to the answers of multiple generative engines is calculated to obtain the target logical representation value. The target logical representation value is used to represent the consistency between the content of the text fragment corresponding to the test question and the content of the answers of multiple generative engines. This can offset the random deviation of the logical score corresponding to a single answer. The target logical representation value has stability and statistical representativeness.

[0044] In an optional embodiment, step S1034 above can be implemented as follows: For each of the above test questions, obtain the text vectors of the multiple answers; For example, the answer is input into the Sentence-BERT model to obtain the text vector of the answer.

[0045] For each of the above test questions, a content similarity matrix is ​​constructed based on the text vectors of the multiple answers. For example, the same test question is input into three generative engines, resulting in three answers, denoted as A, B, and C. The cosine similarity of the text vectors of any two answers is calculated to obtain a content similarity matrix. , where V AA V represents the cosine similarity between text vectors of A and A'. BB V represents the cosine similarity between the text vectors of B and B'. CC V represents the cosine similarity between the text vectors of C and C'. AB and V BA Both represent the cosine similarity between the text vectors of A and B, V AC and V CA V represents the cosine similarity between the text vectors of A and C. BC and V CB Both represent the cosine similarity between the text vectors of B and C.

[0046] For each of the above test questions, the variance of the off-diagonal elements of the content similarity matrix is ​​calculated to obtain the second target content representation value corresponding to the above test question.

[0047] In this embodiment, for each test question, the variance of the off-diagonal elements of the content similarity matrix is ​​calculated to obtain the second target content representation value corresponding to the test question. The second target content representation value corresponding to the test question can accurately quantify the degree of consistency of the content between the answers of multiple generative engines for the test question. The smaller the variance, the higher the degree of consistency of the content between the answers of multiple generative engines.

[0048] In an optional embodiment, step S104 above can be implemented as follows: Multiple incorrect answers were selected from the above multiple answers. The above multiple incorrect answers are answers that meet at least one type of preset condition. The types of preset conditions include: a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition. The first preset condition is that the initial content representation value corresponding to the above answer is less than the first preset content representation value. The second preset condition is that the factual claim score corresponding to the above answer is less than the first preset score. The third preset condition is that the logical score corresponding to the above answer is less than the second preset score. The fourth preset condition is that the second target content representation value corresponding to the test question to which the above answer belongs is greater than the second preset content representation value. The text fragments corresponding to the above incorrect answers have been identified as fragile text fragments. For each of the above vulnerable text fragments, the rewriting strategy for the above vulnerable text fragments is determined based on the type of preset conditions satisfied by the wrong answer corresponding to the above vulnerable text fragments; For example, if an incorrect answer meets the first preset condition (the initial content representation value of the answer is less than the first preset content representation value), it means that the content of the incorrect answer is very inconsistent with the content of the corresponding text fragment. The core reason is that the expression of the core information (core value) of the text fragment is singular and lacks limiting information of the core information (e.g., applicable scenario, effect). As a result, although the generative engine captures the core meaning, the content of the generated answer deviates too much from the content of the corresponding text fragment. For example, the fragile text fragment is "This air purifier filter should be replaced every 6 months", and the answer is "For this air purifier, it is recommended to replace the filter every six months to ensure the effect". The rewriting strategy is to supplement the synonym of the core value (six months) and the limiting information of the core value (scenario: standard use scenario of daily household use and 8 hours of daily operation, effect: replacing the filter can ensure that the device continues to maintain a good purification effect). The rewritten text fragment is "In the standard use scenario of daily household use and 8 hours of daily operation, it is recommended to replace the filter every 6 months (six months). Replacing the filter can ensure that the device continues to maintain a good purification effect".

[0049] For example, if the type of preset condition met by the incorrect answer is the second preset condition (the initial fact claim representation value corresponding to the answer is less than the preset fact claim representation value), it means that the fact claim of the answer is not consistent with the fact claim of the corresponding text fragment. The core reason is that the attributes of the core information (e.g., core values, core conclusions) in the text fragment are not highlighted, which leads to the generative engine not accurately capturing them. Therefore, in this case, the rewriting strategy is to add emphasis words before the core information to highlight the attributes of the core information. For example, the fragile text fragment is "It is recommended to replace the filter every 6-12 months", and the answer is "The filter can be replaced after 12 uses". The rewriting strategy is to add the attribute emphasis words "shortest replacement cycle and longest replacement cycle" before the core value. The rewritten text fragment is "The shortest replacement cycle of the filter is 6 months, and the longest replacement cycle is 12 months".

[0050] For example, if an incorrect answer meets the third preset condition (the logical score corresponding to the answer is less than the second preset score), it indicates that the logic of the answer is not consistent enough with the logic of the corresponding text fragment. The core reason is that the logical relationships in the text fragment (the order, causal relationship, conditional relationship, and execution flow between various information units in the text) are unclear, causing the generative engine to misunderstand the logical relationship of the text fragment. The rewriting strategy is to use explicit logical relationship connectors to express the implicit logical relationship. For example, the fragile text fragment is "This purifier adopts a double-layer filter structure, which can effectively remove PM2.5 and formaldehyde." The answer is "This purifier can remove PM2.5 and formaldehyde, therefore it adopts a double-layer filter structure." The rewriting strategy is to add causal relationship connectors (because and therefore), and the rewritten text fragment is "Because this purifier adopts a double-layer filter structure, it can effectively remove PM2.5 and formaldehyde."

[0051] For example, if the type of preset condition met by the incorrect answer is the fourth preset condition (the second target content representation value corresponding to the test question to which the answer belongs is greater than the second preset content representation value), that is, the consistency of the answers from multiple generative engines to the same test question is too low, the core reason is that different generative engines have different understandings of the key concepts in the text fragment, and the key concepts in the text fragment are not fully explained. The rewriting strategy is to provide a detailed explanation of the key concepts in the text fragment. For example, the vulnerable text fragment is "This purifier supports intelligent cruise mode, which can automatically adjust the purification range." The first generative engine's answer is "The intelligent cruise mode of this purifier can automatically adjust the wind speed," and the second generative engine's answer is "The intelligent cruise mode of this purifier can automatically move to the heavily polluted area." This difference stems from the fact that the text fragment does not explain the key concept of "intelligent cruise mode," and different generative engines have different understandings of it. The rewriting strategy is used to provide a brief synonymous explanation of the key concept. The rewritten text fragment is "This purifier supports intelligent cruise mode (i.e., a mode that automatically adjusts the purification area coverage by detecting the indoor pollution distribution through built-in sensors), which can automatically adjust the purification range."

[0052] For each of the aforementioned fragile text segments, the aforementioned fragile text segments are rewritten based on the aforementioned fragile text segment rewriting strategy to obtain the aforementioned updated corpus.

[0053] For example, a rewriting strategy for fragile text fragments can be implemented by setting a cueing thought chain for a large language model. The cueing thought chain is used to guide the large language model to rewrite the aforementioned fragile text fragments based on the rewriting strategy for the fragile text fragments.

[0054] In this embodiment, four preset conditions are used to accurately filter out incorrect answers, covering four dimensions: content, factual claims, logic, and cross-engine deviation. This ensures that all types of answer anomalies are captured without omission, enabling precise location of fragile text fragments and avoiding blind rewriting. A dedicated rewriting strategy is matched for different preset conditions, so that the rewritten corpus can effectively guide the generative engine to output answers with consistent content, accurate facts, and clear logic, thereby reducing cross-engine answer deviation.

[0055] Figure 2 A corpus optimization apparatus for a generative engine provided in this application embodiment includes: Acquisition unit 10 is used to acquire the corpus and perturbated test questions; Test unit 20 is used to input the above test questions into multiple generative engines to obtain the above multiple answers; The above answers correspond one-to-one with the above generative engine, which is used to generate the above answers based on the above corpus. The determination unit 30 is used to determine the robustness representation value based on the above corpus and the above multiple responses; Among them, the robustness characterization value represents the degree of consistency between the above multiple answers and the above corpus, as well as the degree of consistency between the above multiple answers; The rewriting unit 40 is used to rewrite the corpus based on the multiple answers when the robustness representation value is less than a preset value, so as to obtain an updated corpus, thereby improving the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus and the consistency between the answers generated by the multiple generative engines based on the corpus. The repeating unit 50 is used to sequentially call the above-mentioned test unit, the above-mentioned determination unit and the above-mentioned rewrite unit at least once, until the above-mentioned robustness characterization value is not less than the above-mentioned preset value. The updated corpus is used to guide the generative engine in generating answers to questions.

[0056] Through the above embodiments, the following steps are implemented: First, the corpus and perturbated test questions are acquired. Second, the test questions are input into multiple generative engines to obtain multiple answers. Third, a robustness representation value is determined based on the corpus and the multiple answers. Fourth, a rewriting step is performed where, if the robustness representation value is less than a preset value, it is determined that the consistency between the answers generated by the generative engines based on the corpus and the corpus is low, and the consistency between the answers generated by multiple generative engines based on the corpus is also low. Therefore, the corpus is rewritten to improve the consistency between the answers generated by the generative engines based on the corpus and the corpus, as well as the consistency between the answers generated by different generative engines based on the corpus. Fifth, the steps are repeated, executing the test step, determination step, and rewriting step at least once, until the robustness representation value is not less than the preset value. This ensures that the consistency between the answers generated by the generative engines based on the corpus and the corpus, as well as the consistency between the answers generated by different generative engines based on the corpus, meets the requirements, thereby reducing answer bias caused by user input question noise and cross-engine differences in the generative engine application.

[0057] In one optional embodiment, there are multiple test questions, the corpus consists of multiple text segments, one test question corresponds to one text segment, the generative engine is used to generate answers based on the text segments corresponding to the test questions, and the determining unit includes: The first determining module is used to determine, for each of the above test questions, a first target content representation value corresponding to the above test question based on the text fragment corresponding to the above test question and the above multiple answers. The first target content representation value represents the degree of consistency between the content of the text fragment corresponding to the above test question and the content of the above multiple answers. The second determining module is used to determine, for each of the above test questions, a target fact claim representation value corresponding to the above test question based on the text fragment corresponding to the above test question and the above multiple answers. The target fact claim representation value represents the degree of consistency between the fact claim of the text fragment corresponding to the above test question and the fact claim of the above multiple answers. The third determining module is used to determine the target logical representation value corresponding to each of the above test questions based on the text fragment corresponding to the above test question and the above multiple answers. The target logical representation value represents the degree of consistency between the logic of the text fragment corresponding to the above test question and the logic of the above multiple answers. The fourth determining module is used to determine, for each of the above test questions, a second target content representation value corresponding to the above test question based on the above multiple answers, wherein the second target content representation value represents the degree of consistency of the content of the above multiple answers; The first calculation module is used to calculate the average value of the first target content representation value corresponding to all the above test questions to obtain the first average content representation value; calculate the average value of the target fact claim representation value corresponding to all the above test questions to obtain the average fact claim representation value; calculate the average value of the target logical representation value corresponding to all the above test questions to obtain the average logical representation value; and calculate the average value of the second target content representation value corresponding to all the above test questions to obtain the second average target content representation value. The second calculation module is used to normalize the first average content representation value, the average fact claim representation value, the average logic representation value and the second average target content representation value in sequence to obtain the first standard content representation value, the standard fact claim representation value, the standard logic representation value and the second standard content representation value. The third calculation module is used to perform a weighted summation of the above-mentioned first standard content representation value, the above-mentioned standard fact claim representation value, the above-mentioned standard logic representation value, and the above-mentioned second standard content representation value to obtain the above-mentioned robustness representation value.

[0058] In an optional embodiment, the first determining module includes: The first acquisition submodule is used to acquire, for each of the above test questions, the text vector of the text segment corresponding to the above test question and the text vector of the above multiple answers; The first calculation submodule is used to calculate the cosine similarity between the text vector of each of the above-mentioned test questions and the text vector of the text segment corresponding to the above-mentioned test questions, so as to obtain the initial content representation value corresponding to each of the above-mentioned answers. The initial content representation value represents the degree of consistency between the content of the above-mentioned answer and the content of the text segment corresponding to the above-mentioned test questions. The second calculation submodule is used to calculate the average value of the initial content representation values ​​corresponding to all the above answers for each of the above test questions, so as to obtain the first target content representation value corresponding to the above test questions.

[0059] In one optional embodiment, the second determining module includes: The first input submodule is used to input the text fragments corresponding to the above test questions and the above test questions into the trained natural language reasoning model for each of the above test questions and each of the above answers, so as to obtain the fact claim score corresponding to the above answer. The fact claim score represents the degree of consistency between the fact claim of the above answer and the fact claim of the text fragment corresponding to the above test questions. The third calculation submodule is used to calculate the fact claim score corresponding to all the above answers for each of the above test questions, and obtain the target fact claim representation value corresponding to the above test questions.

[0060] In an optional embodiment, the third determining module includes: The configuration submodule is used to set up thought prompts for the Large Language Model (LLM). These thought prompts are configured to guide the LLM to execute the following sequentially: a logic chain for breaking down text fragments, a logic chain for breaking down answers, and a score for the consistency between the logic chain for text fragments and the logic chain for the answers. The second input submodule is used to input the text fragments corresponding to the above-mentioned test questions and test questions into the above-mentioned large language model for each of the above-mentioned test questions and each of the above-mentioned answers, so as to obtain the logical score corresponding to the above-mentioned answers. The fourth calculation submodule is used to calculate the logical score corresponding to all the above answers for each of the above test questions, and obtain the target logical representation value corresponding to the above test questions.

[0061] In an optional embodiment, the fourth determining module includes: The second acquisition submodule is used to acquire the text vector of the multiple answers for each of the above test questions; A submodule is constructed to build a content similarity matrix for each of the above test questions, based on the text vectors of the multiple answers. The fifth calculation submodule is used to calculate the variance of the off-diagonal elements of the content similarity matrix for each of the above test questions, and obtain the second target content representation value corresponding to the above test questions.

[0062] In an optional embodiment, the rewriting unit includes: The filtering module is used to filter out multiple incorrect answers from the above multiple answers. The above multiple incorrect answers are answers that meet at least one type of preset condition. The types of preset conditions include: a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition. The first preset condition is that the initial content representation value corresponding to the above answer is less than a first preset content representation value. The second preset condition is that the factual claim score corresponding to the above answer is less than a first preset score. The third preset condition is that the logical score corresponding to the above answer is less than a second preset score. The fourth preset condition is that the second target content representation value corresponding to the test question to which the above answer belongs is greater than a second preset content representation value. The fifth determination module is used to determine whether the text fragments corresponding to the above incorrect answers are fragile text fragments; The sixth determining module is used to determine the rewriting strategy for each of the above-mentioned vulnerable text fragments based on the type of preset conditions satisfied by the incorrect answers corresponding to the above-mentioned vulnerable text fragments. The rewriting module is used to rewrite each of the aforementioned fragile text segments based on the rewriting strategy for the aforementioned fragile text segments, thereby obtaining the aforementioned updated corpus.

[0063] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described generative engine corpus optimization methods: This application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of any of the above-described generative engine corpus optimization methods.

[0064] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0066] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0067] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A corpus optimization method for a generative engine, characterized in that, The method includes: The acquisition steps involve obtaining the corpus and perturbation-processed test questions; The testing steps involve inputting the test question into multiple generative engines to obtain multiple answers. Each answer corresponds one-to-one with a generative engine, and the generative engine is used to generate the answer based on the corpus. The determination step involves determining a robustness representation value based on the corpus and the multiple answers. The robustness representation value represents the degree of consistency between the multiple answers and the corpus, as well as the degree of consistency between the multiple answers. The rewriting step involves rewriting the corpus based on the multiple answers when the robustness representation value is less than a preset value, to obtain an updated corpus, thereby improving the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus, as well as the consistency between the answers generated by the multiple generative engines based on the corpus. Repeat the steps, performing the test step, the determination step, and the rewrite step at least once in sequence, until the robustness representation value is not less than the preset value. The updated corpus is used to guide the generative engine to generate answers to questions.

2. The method according to claim 1, characterized in that, There are multiple test questions, and the corpus consists of multiple text fragments. Each test question corresponds to one text fragment. The generative engine generates answers based on the text fragments corresponding to the test questions. Based on the corpus and the multiple answers, it determines robustness representation values, including: For each test question, based on the text fragment corresponding to the test question and the multiple answers, a first target content representation value is determined for the test question. The first target content representation value represents the degree of consistency between the content of the text fragment corresponding to the test question and the content of the multiple answers. For each test question, based on the text fragment corresponding to the test question and the multiple answers, a target fact claim representation value is determined for the test question. The target fact claim representation value represents the degree of consistency between the fact claim of the text fragment corresponding to the test question and the fact claim of the multiple answers. For each test question, based on the text fragment corresponding to the test question and the multiple answers, a target logical representation value corresponding to the test question is determined. The target logical representation value represents the degree of consistency between the logic of the text fragment corresponding to the test question and the logic of the multiple answers. For each test question, based on the multiple answers, a second target content representation value is determined for the test question, wherein the second target content representation value represents the degree of consistency of the content of the multiple answers; Calculate the average of the first target content representation values ​​corresponding to all the test questions to obtain the first average content representation value; calculate the average of the target fact claim representation values ​​corresponding to all the test questions to obtain the average fact claim representation value; calculate the average of the target logical representation values ​​corresponding to all the test questions to obtain the average logical representation value; calculate the average of the second target content representation values ​​corresponding to all the test questions to obtain the second average target content representation value. The first average content representation value, the average fact claim representation value, the average logical representation value, and the second average target content representation value are normalized sequentially to obtain the first standard content representation value, the standard fact claim representation value, the standard logical representation value, and the second standard content representation value. The robustness representation value is obtained by weighted summing of the first standard content representation value, the standard fact claim representation value, the standard logic representation value, and the second standard content representation value.

3. The method according to claim 2, characterized in that, For each test question, based on the text fragment corresponding to the test question and the multiple answers, a first target content representation value corresponding to the test question is determined, including: For each test question, obtain the text vector of the text segment corresponding to the test question and the text vector of the multiple answers; For each test question, the cosine similarity between the text vector of each answer and the text vector of the text segment corresponding to the test question is calculated to obtain the initial content representation value for each answer. The initial content representation value represents the degree of consistency between the content of the answer and the content of the text segment corresponding to the test question. For each test question, the average of the initial content representation values ​​corresponding to all the answers is calculated to obtain the first target content representation value corresponding to the test question.

4. The method according to claim 3, characterized in that, For each test question, based on the text fragment corresponding to the test question and the multiple answers, a target fact claim representation value corresponding to the test question is determined, including: For each test question and each answer, the text fragments corresponding to the answer and the test question are input into a trained natural language reasoning model to obtain a fact claim score corresponding to the answer. The fact claim score represents the degree of consistency between the fact claim of the answer and the fact claim of the text fragment corresponding to the test question. For each test question, calculate the fact claim score corresponding to all the answers to obtain the target fact claim representation value corresponding to the test question.

5. The method according to claim 4, characterized in that, For each test question, based on the text fragment corresponding to the test question and the multiple answers, a target logical representation value corresponding to the test question is determined, including: A thought prompt chain is set for the large language model, which is configured to guide the large language model to execute sequentially: constructing a logical chain for a text fragment, constructing a logical chain for a response, and scoring the consistency between the logical chain for the text fragment and the logical chain for the response. For each test question and each answer, the text fragments corresponding to the answer and the test question are input into the large language model to obtain the logical score corresponding to the answer; For each test question, calculate the logical score corresponding to all the answers to obtain the target logical representation value corresponding to the test question.

6. The method according to claim 5, characterized in that, For each of the test questions, based on the multiple answers, a second target content representation value corresponding to the test question is determined, including: For each of the test questions, obtain the text vectors of the multiple answers; For each of the test questions, a content similarity matrix is ​​constructed based on the text vectors of the multiple answers; For each test question, the variance of the off-diagonal elements of the content similarity matrix is ​​calculated to obtain the second target content representation value corresponding to the test question.

7. The method according to claim 6, characterized in that, The rewriting step involves rewriting the corpus based on the multiple responses when the robustness representation value is less than a preset value, to obtain an updated corpus, including: Multiple incorrect answers are selected from the multiple answers. The multiple incorrect answers are answers that meet at least one type of preset condition. The types of preset conditions include: a first preset condition, a second preset condition, a third preset condition, and a fourth preset condition. The first preset condition is that the initial content representation value corresponding to the answer is less than a first preset content representation value. The second preset condition is that the factual claim score corresponding to the answer is less than a first preset score. The third preset condition is that the logical score corresponding to the answer is less than a second preset score. The fourth preset condition is that the second target content representation value corresponding to the test question to which the answer belongs is greater than a second preset content representation value. The text fragment corresponding to the incorrect answer was determined to be a fragile text fragment; For each of the fragile text segments, a rewriting strategy for the fragile text segment is determined based on the type of preset conditions satisfied by the incorrect answer corresponding to the fragile text segment. For each of the fragile text segments, the fragile text segments are rewritten based on the rewriting strategy of the fragile text segments to obtain the updated corpus.

8. A corpus optimization device for a generative engine, characterized in that, The device includes: The acquisition unit is used to acquire the corpus and perturbated test questions; The testing unit is used to input the test question into multiple generative engines to obtain multiple answers, each answer corresponding to one of the generative engines, and the generative engine is used to generate the answer based on the corpus. A determining unit is configured to determine a robustness representation value based on the corpus and the multiple answers, wherein the robustness representation value represents the degree of consistency between the multiple answers and the corpus, and the degree of consistency between the multiple answers. The rewriting unit is used to rewrite the corpus based on the multiple answers when the robustness representation value is less than a preset value, so as to obtain an updated corpus, thereby improving the consistency between the corpus and the answers generated by the multiple generative engines based on the corpus and the consistency between the answers generated by the multiple generative engines based on the corpus. The repeating unit is used to sequentially call the test unit, the determination unit, and the rewrite unit at least once, until the robustness representation value is not less than the preset value. The updated corpus is used to guide the generative engine to generate answers to questions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the corpus optimization method of the generative engine as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the corpus optimization method of the generative engine as described in any one of claims 1 to 7.

Images