Domain question and answer pair automatic extraction method and system

Abstract

The application provides a field question and answer pair automatic extraction method and system, relates to the technical field of data processing, encodes evidence nodes of field text into active indexes and generates candidate question and answers, determines verification risks according to comparison results, obtains updated prompt versions according to the verification risks, selects target evidence nodes according to prompt version changes and node verification risks, re-encodes the target evidence nodes and writes them into incremental indexes, respectively schedules the active indexes and the incremental indexes for retrieval according to migration service states, generates or verifies candidate question and answers after score calibration and merging of corresponding retrieval results, updates a version pointer of the active indexes based on the merged results, restores corresponding evidence nodes to be served by the active indexes when writing or switching abnormities, iteratively executes the above steps and outputs a question and answer pair data set, and can realize accurate positioning of a question source and selective updating of an evidence base in a construction process of the question and answer pair, and guarantees continuity of retrieval service.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to a method and system for automatically extracting domain-specific question-answer pairs. Background Technology

[0002] With the deep application of artificial intelligence technology in vertical industries, automatically constructing question-answer pair datasets from non-conversational texts such as textbooks, technical documents, and institutional documents has become a key step in promoting the implementation of domain-specific intelligent question-answering systems.

[0003] Currently, existing domain question answering schemes typically treat quality feedback and evidence base updates as two separate processes during iterative optimization. When a candidate question answer is detected to lack original text support or contradict the domain text content, the source of the quality problem cannot be located to the specific erroneous document fragment and fed back to the evidence base update stage. This leads to a disconnect between generation strategy optimization and evidence base maintenance. Even after multiple iterations, document fragments that repeatedly cause quality problems are still treated equally and cannot be prioritized for correction. After the suggestion strategy is updated, the evidence base also needs to be fully rebuilt, resulting in the ineffective consumption of computing resources. During the reconstruction, the retrieval service may also be forced to be interrupted, affecting the continuity and stability of question answer generation.

[0004] Therefore, how to accurately locate the source of questions and selectively update the evidence base during the construction of question-answer pairs, while ensuring the continuity of retrieval services, has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a method and system for automatically extracting domain-specific question-answer pairs, which can accurately locate the source of questions and selectively update the evidence base during the construction of question-answer pairs, while ensuring the continuity of retrieval services.

[0006] A first aspect of the present invention provides a method for automatically extracting domain question-answer pairs, comprising:

[0007] Evidence nodes of the domain text are encoded into the activity index. Candidate questions and answers are generated based on the current prompt version and the activity index. Verification risk is determined based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index.

[0008] The current prompt version is updated based on the verification risk to obtain an updated prompt version, and the verification risk is assigned to the associated evidence node to obtain the node verification risk. The target evidence node is selected based on the change amount of the prompt version and the node verification risk.

[0009] Based on the update prompt version, the selected target evidence nodes are re-encoded and written into the incremental index, and the migration service status of the target evidence nodes is recorded.

[0010] According to the migration service status, the active index and the incremental index are scheduled to be retrieved respectively. The corresponding retrieval results are merged after score calibration. Candidate questions and answers are generated or verified based on the merged results. The version pointer of the active index is updated when the merged results meet the index switching conditions.

[0011] In the event of a write or switch failure, the corresponding evidence node is restored to be served by the active index according to the migration service status, and the above steps are iteratively executed until the preset convergence condition is met before outputting the question-answer pair dataset.

[0012] Optionally, in one possible implementation of the first aspect, encoding the evidence nodes of the domain text into an activity index, generating candidate questions and answers based on the current prompt version and the activity index, and determining the verification risk based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index, includes:

[0013] The domain text is divided into multiple levels to obtain evidence nodes. The relationship between each evidence node is established. Each evidence node is encoded together with the current prompt version and then loaded into the activity index.

[0014] The initial evidence node is obtained by retrieving the activity index based on the labeled question and answer, and a positive evidence node set is obtained by expanding along the association relationship. Candidate questions and answers are generated based on the positive evidence node set, and the candidate answers of the candidate questions and answers contain the source mapping of each answer fragment and the corresponding evidence node.

[0015] After encoding the candidate questions and answers together with the current prompt version, the activity index is retrieved to obtain a set of reverse evidence nodes;

[0016] The verification risk is determined based on the differences in node overlap between the positive evidence node set and the negative evidence node set, the source support of each answer fragment in the positive evidence node set, and the nodes in the negative evidence node set that contradict the candidate question and answer.

[0017] Optionally, in one possible implementation of the first aspect, the step of retrieving the activity index based on the labeled question and answer to obtain initial evidence nodes, expanding along the association relationship to obtain a set of positive evidence nodes, and generating candidate questions and answers based on the set of positive evidence nodes, wherein the candidate answers of the candidate questions and answers include source mappings of each answer fragment and the corresponding evidence node, including:

[0018] The labeled questions and answers are encoded into a query vector under the current prompt version, and the activity index is retrieved based on the query vector to obtain the initial evidence node;

[0019] The initial evidence nodes are extended along the association relationship to obtain candidate nodes. Based on the coverage of each candidate node to the labeled questions and answers and the degree of repetition between each candidate node, the set of positive evidence nodes is selected within a preset text length range.

[0020] Extract answer fragments from the set of positive evidence nodes, record the evidence node identifier and original text position corresponding to each answer fragment, and form a source mapping;

[0021] Candidate questions are generated by using the answer fragment as the answer constraint, the set of positive evidence nodes as the context constraint, and the question-answer type corresponding to the labeled question and answer as the question type constraint. The answer fragment and the candidate questions are then combined to obtain the candidate question and answer.

[0022] Optionally, in one possible implementation of the first aspect, determining the verification risk based on the node overlap difference between the positive evidence node set and the negative evidence node set, the source support of each answer fragment in the positive evidence node set, and the nodes in the negative evidence node set that contradict the candidate question and answer includes:

[0023] Calculate the node overlap degree between the positive evidence node set and the negative evidence node set, and obtain the node overlap difference based on the difference between the node overlap degree and the preset overlap threshold. When the node overlap degree is not lower than the preset overlap threshold, the node overlap difference value is zero.

[0024] The proportion of missing source support is obtained based on the proportion of each answer fragment that does not have the corresponding original text position in the set of positive evidence nodes. The source support situation includes the proportion of missing source support.

[0025] The number of evidence nodes in the reverse evidence node set that contradict the candidate answer of the candidate question and answer in terms of value, condition, or version is counted, and the proportion of the number of contradictory evidence nodes to the total number of nodes in the reverse evidence node set is calculated to obtain the contradictory node score.

[0026] The verification risk is obtained by weighted summation of the node overlap difference, the proportion of missing source support, and the score of the contradictory node.

[0027] Optionally, in one possible implementation of the first aspect, the step of updating the current prompt version according to the verification risk to obtain an updated prompt version, and assigning the verification risk to associated evidence nodes to obtain node verification risks, and selecting target evidence nodes according to the prompt version change amount and the node verification risks, includes:

[0028] Based on the verification risk, the prompt parameters of the current prompt version are adjusted to obtain an updated prompt version;

[0029] The verification risk is assigned to each associated evidence node corresponding to the candidate question and answer, and the verification risk assigned to the same evidence node in different candidate questions and answers is aggregated to obtain the node verification risk of each evidence node.

[0030] Calculate the degree of difference between the prompt parameters of the current prompt version and the prompt parameters of the updated prompt version to obtain the change amount of the prompt version;

[0031] Based on the change amount of the prompt version and the node verification risk of each evidence node, the recoding priority of each evidence node is determined, and the target evidence node is selected within the preset update quantity range.

[0032] Optionally, in one possible implementation of the first aspect, adjusting the prompt parameters of the current prompt version based on the verification risk to obtain the updated prompt version includes:

[0033] Construct a joint loss function that includes semantic loss, bidirectional evidence consistency loss, and positive-negative evidence comparison loss;

[0034] Based on the verification risk, the weights of the bidirectional evidence consistency loss and the positive-negative evidence comparison loss in the joint loss function are weighted and adjusted to obtain the target joint loss function.

[0035] The updated suggestion version is obtained by performing gradient updates on the suggestion parameters of the current suggestion version based on the target joint loss function;

[0036] The step of determining the recoding priority of each evidence node based on the version change amount and the node verification risk of each evidence node, and selecting target evidence nodes within a preset update quantity range, includes:

[0037] Obtain the query hit frequency of each evidence node within a preset time period;

[0038] The recoding priority of each evidence node is obtained by normalizing and weighting the change amount of the prompt version, the node verification risk of each evidence node, and the query hit frequency.

[0039] A preset number of evidence nodes are selected from high to low according to the recoding priority to obtain the target evidence nodes.

[0040] Optionally, in one possible implementation of the first aspect, the step of re-encoding the selected target evidence node based on the update prompt version and writing it into the incremental index, and recording the migration service status of the target evidence node, includes:

[0041] The update prompt version and the selected target evidence node are re-encoded to obtain a new version vector for each target evidence node, and each new version vector is written into the incremental index;

[0042] Record the migration service status of each of the target evidence nodes. The migration service status includes at least "writing", "verified", and "service new version". The migration service status of evidence nodes that are not selected is that the activity index provides retrieval services.

[0043] When the migration service status is "writing" or "verified", the active index continues to provide retrieval services for the target evidence node;

[0044] When the migration service status changes to a new service version, the corresponding target evidence node is retrieved by the incremental index.

[0045] Optionally, in one possible implementation of the first aspect, the step of scheduling the active index and the incremental index for retrieval according to the migration service status, merging the corresponding retrieval results after score calibration, generating or verifying candidate questions and answers based on the merged results, and updating the version pointer of the active index when the merged results meet the index switching conditions includes:

[0046] Based on the migration service status, retrieve the unselected evidence nodes and the target evidence nodes whose migration service status is "writing" or "verified" from the activity index to obtain the first retrieval result; retrieve the target evidence nodes whose migration service status is "service new version" from the incremental index to obtain the second retrieval result.

[0047] The first search result includes the retrieved evidence nodes and their corresponding search scores; the second search result includes the retrieved evidence nodes and their corresponding search scores.

[0048] The search scores of the first search result and the search scores of the second search result are normalized respectively. The processed first search result and the second search result are deduplicated and merged to obtain the merged search result.

[0049] The merged search results are used as context to generate candidate questions and answers, or the source support verification of the answer fragments in the generated candidate questions and answers is performed based on the evidence nodes in the merged search results.

[0050] When the merged search results meet the preset index switching conditions, the new version vectors of each target evidence node in the incremental index are merged into the active index, and the version pointer of the active index is updated.

[0051] Optionally, in one possible implementation of the first aspect, the step of restoring the corresponding evidence node to be served by the active index according to the migration service status when a write or switch exception occurs, and iteratively executing the above steps until a preset convergence condition is met before outputting the question-answer pair dataset, includes:

[0052] When the new version vector fails to be written to the incremental index, the migration service status of the corresponding target evidence node is marked as write failure, and the active index continues to provide retrieval services for the target evidence node;

[0053] When the version pointer update of the active index fails, the target evidence node whose migration service status is the new version will be restored to having the retrieval service provided by the active index.

[0054] Repeat the above steps of generating and verifying candidate questions and answers, updating the prompt version, selecting and recoding the target evidence node, writing the incremental index, scheduling, retrieving and merging the active index and the incremental index, and updating the version pointer.

[0055] When the rate of change of the verification risk is lower than a preset change threshold or the number of iterations reaches the preset maximum number of iterations, it is determined that the preset convergence condition is met and the iteration stops.

[0056] Output candidate questions and answers whose verification risk is lower than a preset risk threshold to obtain a question-answer pair dataset.

[0057] A second aspect of the present invention provides a domain question-answering pair automatic extraction system, comprising:

[0058] The generation module is used to encode the evidence nodes of the domain text into the activity index, generate candidate questions and answers based on the current prompt version and the activity index, and determine the verification risk based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index.

[0059] The update module is used to update the current prompt version according to the verification risk to obtain an updated prompt version, and to allocate the verification risk to the associated evidence node to obtain the node verification risk, and to select the target evidence node according to the change amount of the prompt version and the node verification risk;

[0060] The recording module is used to re-encode the selected target evidence node based on the update prompt version and write it into the incremental index, and record the migration service status of the target evidence node.

[0061] The retrieval module is used to schedule the active index and the incremental index to perform retrieval according to the migration service status, merge the corresponding retrieval results after score calibration, generate or verify candidate questions and answers based on the merged results, and update the version pointer of the active index when the merged results meet the index switching conditions.

[0062] The output module is used to restore the corresponding evidence node to be served by the active index according to the migration service status when a write or switch failure occurs, and to iteratively execute the above steps until the preset convergence condition is met before outputting the question-answer pair dataset.

[0063] The beneficial effects of this invention are as follows:

[0064] 1. This invention can generate candidate questions and answers through forward retrieval and verify candidate questions and answers through reverse retrieval. It can comprehensively determine the verification risk by combining three dimensions: node overlap difference, source support missing ratio and contradictory node score. Furthermore, when generating candidate questions and answers, it can ensure that each fragment in the candidate answer can be traced back to the specific evidence node and its original text position in the domain text through answer fragment extraction and source mapping record, thereby improving the accuracy of domain question answer automatic extraction.

[0065] 2. This invention can accurately locate evidence nodes that repeatedly cause quality problems by attributing verification risks to associated evidence nodes and aggregating them. Then, it can determine the recoding priority of each evidence node by normalizing and weighting the three dimensions of prompt version change, node verification risk and query hit frequency. Within the preset update range, target evidence nodes are selected for local recoding. This can reduce the waste of computing resources while improving the optimization efficiency of prompt version and the generation quality of candidate questions and answers.

[0066] 3. This invention can ensure the continuity of retrieval services by restoring the corresponding node to be served by the active index when the new version vector fails to be written to the incremental index, and by reverting the node that has been switched to the incremental index to the active index when the version pointer update fails. Attached Figure Description

[0067] Figure 1 This is a flowchart illustrating an automatic extraction method for domain question-answering pairs provided in an embodiment of the present invention;

[0068] Figure 2 This is a schematic diagram of the structure of an automatic domain question-answering pair extraction system provided in an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0070] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0071] See Figure 1This is a flowchart illustrating an automatic domain question-answering pair extraction method provided in an embodiment of the present invention. Figure 1 The method shown can be implemented by software and / or hardware devices. It includes steps S1 to S5, as detailed below:

[0072] S1, Encode the evidence nodes of the domain text into the activity index, generate candidate questions and answers based on the current prompt version and the activity index, and determine the verification risk based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index.

[0073] Among them, domain text refers to a collection of non-conversational documents that carry specific industry knowledge, such as textbooks, technical manuals, and policy documents; evidence nodes refer to independently searchable text units divided from domain text, with each evidence node recording its text content and position information in the original text; the active index refers to an index structure used to store the vector representation of evidence nodes and provide retrieval functions, which can provide query services to external parties; the current prompt version refers to the set of parameters used to guide coding and question-answer generation in the current round; candidate questions and answers refer to question-answer pairs generated based on the current prompt version and the active index that have not yet undergone quality verification; the comparison result refers to the matching situation obtained after comparing the candidate questions and answers with the evidence nodes in the active index; and the verification risk refers to the quality risk value of the candidate questions and answers determined based on the comparison result. The higher the verification risk, the greater the possibility that the candidate questions and answers have quality problems.

[0074] Understandably, domain text is unstructured content in natural language form, making it difficult for computers to directly retrieve and match based on semantics. By encoding evidence nodes as vectors and loading them into the active index, the semantic information of the text can be transformed into a computable numerical representation. The index structure supports efficient similarity retrieval, enabling the rapid finding of evidence nodes related to the query content, whether generating candidate questions and answers or performing reverse verification. However, automatically generated candidate questions and answers may have issues such as inconsistencies between the answers and the original text or a lack of supporting evidence. Directly outputting these as the final result would affect the accuracy and credibility of the question-and-answer dataset. Therefore, after generating candidate questions and answers, they can be compared with the evidence in the active index to determine the degree of deviation from the evidence.

[0075] Specifically, the evidence nodes obtained from the domain text segmentation are first encoded into vector representations along with the current prompt version and loaded into the active index. Then, under the current prompt version, candidate questions and answers are generated based on the active index. After the candidate questions and answers are generated, they are compared with the evidence nodes in the active index. The comparison can include the degree of overlap between the two sets of evidence nodes, whether each content fragment in the candidate answer can find corresponding original text support in the evidence nodes, and whether there are evidence nodes in the search results that contradict the content of the candidate answer. The comparison results are obtained, and the verification risk value corresponding to the candidate question and answer is calculated based on the comparison results.

[0076] In some embodiments, step S1 can be implemented as follows:

[0077] S11, the domain text is divided into multiple levels to obtain evidence nodes, the relationship between each evidence node is established, and each evidence node is encoded together with the current prompt version and loaded into the activity index.

[0078] Among them, multi-level division refers to the process of segmenting domain text according to a hierarchical structure from large to small. For example, a document can be divided into text units of different granularities such as chapters, paragraphs, and sentences. Each level of text unit can be used as an evidence node. Association refers to the attribution and reference relationship between evidence nodes of different levels. For example, a sentence belongs to a paragraph, and a paragraph belongs to a chapter. Through association, scattered text units can be organized into a hierarchical structure.

[0079] Understandably, domain texts are usually quite long. If the entire document is used as the unit of retrieval, the search results will not be precise enough and it will be difficult to locate the specific supporting statements. By dividing the text into fine-grained evidence nodes according to the hierarchy and establishing the relationships between them, the most relevant nodes can be found first during the retrieval, and then the context can be obtained along the relationships, so that the search results are both targeted and complete.

[0080] Specifically, the domain text is divided into chapters, paragraphs, and sentences to obtain evidence nodes of different granularities. The text content and start and end positions of each node in the original text are recorded. Then, the relationships between the evidence nodes are established, including establishing parent-child relationships for nodes with inclusion relationships, establishing sequential relationships for adjacent nodes at the same level, and establishing semantic relationships for nodes that mention the same entity or clause. Finally, each evidence node and the current prompt version are encoded into a vector and loaded into the activity index.

[0081] Common encoding can be performed in the following ways:

[0082] S111, the current prompt version, the text content of each evidence node and the title path are concatenated to obtain the encoded input corresponding to each evidence node.

[0083] Among them, text content refers to the original text contained in the evidence node, title path refers to the position path of the evidence node in the document structure, such as the title level of the chapter to which it belongs, and encoded input refers to the complete text formed by concatenating the current prompt version, text content and title path in a preset order.

[0084] Understandably, simply encoding the text content of an evidence node into a vector would lose the node's structural position information within the document. For example, the same sentence appearing in the disclaimer section and the liability determination section might have completely different meanings. Therefore, during encoding, the title path can also be included as input information and encoded along with the text content, so that the generated vector contains both semantic and structural position information. In addition, the current prompt version determines the direction of encoding, so it can also be concatenated, allowing the encoding model to output different vector representations when the same evidence node is paired with different prompt versions.

[0085] Specifically, for each evidence node, its text content and title path are obtained, and the current prompt version, text content and title path are concatenated into a complete text sequence in chronological order. This text sequence is the encoded input corresponding to the evidence node.

[0086] S112, input each of the aforementioned encoding inputs into the encoding model to obtain the node vector of each evidence node under the current prompt version.

[0087] The encoding model refers to a model that can convert a text sequence into a numerical vector representation. Its function is to take a text as input and output a fixed-dimensional vector. This vector can represent the semantic information of the input text, so that texts with similar semantics are also close in distance in the vector space. The node vector refers to the numerical vector output by the encoding model after encoding the input text sequence. This vector is a comprehensive representation of the semantic and structural information of the evidence node under the current prompt version.

[0088] Understandably, the encoded input obtained by concatenation in step S111 is still in text form. Computers cannot directly calculate the similarity of text and need to use an encoding model to convert it into a numerical vector. The role of the encoding model is to read the input text sequence and compress its semantic and structural information into a numerical vector of fixed dimensions. The values ​​of each dimension in the vector jointly characterize the features of the text under the current prompt version. The same evidence node paired with different prompt versions will result in different encoded inputs and different vectors output by the encoding model. This solution does not restrict which specific encoding model to use. Commonly used pre-trained language models in this field can be used as encoding models, such as BERT, T5, and LLaMA models. When using it, the encoded input is fed into the model, and then the hidden state vector output by the last layer of the model or the vector after pooling can be taken as the node vector.

[0089] Specifically, the encoded input corresponding to each evidence node is fed into the encoding model. After processing the input sequence layer by layer, the encoding model outputs a numerical vector of fixed dimensions. This vector is the node vector of the evidence node in the current prompt version. For example, when using the BERT model as the encoding model, the encoded input is segmented and fed into the BERT model. The vector corresponding to the [CLS] tag output by the BERT model is taken as the node vector of the evidence node. When using the T5 model, the encoded input can be fed into the T5 encoder. The sequence representation output by the encoder is averaged and pooled as the node vector. The encoding model is used to encode the existing technology, which will not be elaborated on here.

[0090] S113, associate each node vector with the node identifier of the corresponding evidence node and the version identifier of the current prompt version and store them in the activity index.

[0091] Among them, the version identifier is a mark used to distinguish different prompt versions. The version identifier is updated every time the prompt version is updated. The node identifier is a mark used to identify each evidence node.

[0092] Specifically, for each evidence node, its node identifier, the version identifier of the current prompt version, and the encoded node vector are associated and stored together in the active index. In this way, during subsequent searches, it can be known which node the retrieved vector comes from and under which prompt version it was encoded, providing a basis for index switching and anomaly recovery.

[0093] S12, retrieve the activity index based on the labeled question and answer to obtain the initial evidence node, expand along the association relationship to obtain the positive evidence node set, and generate candidate questions and answers based on the positive evidence node set. The candidate answers of the candidate questions and answers contain the source mapping of each answer fragment and the corresponding evidence node.

[0094] Among them, labeled question-and-answer refers to pre-prepared question-and-answer pairs containing questions and standard answers, which serve as a reference for generating candidate question-and-answer. Initial evidence nodes refer to the evidence nodes most relevant to the labeled question-and-answer obtained after indexing the labeled question-and-answer retrieval activity. The positive evidence node set refers to a set of evidence nodes selected after expanding along the association relationship based on the initial evidence nodes. Candidate answers refer to the answer part in the candidate question-and-answer. Answer fragments refer to text paragraphs extracted from the positive evidence node set that can be directly used as candidate answers. Source mapping refers to recording which evidence node each answer fragment comes from and its corresponding specific position in the original text.

[0095] Understandably, the activity index stores node vectors for each evidence node, with each node vector representing the semantic information of that evidence node. Retrieving the activity index essentially involves calculating the similarity between the query content and the node vectors stored in the activity index. The higher the similarity, the closer the query content is to the semantics of that evidence node. Then, the evidence node with the highest similarity is returned as the search result. Labeled questions and answers serve as existing high-quality question and answer samples, guiding the retrieval of relevant evidence nodes from the activity index, thus providing a material basis for generating new candidate questions and answers. After retrieving the initial evidence node, expansion along the association relationship can supplement the relevant context around the initial node, making the evidence on which the candidate questions and answers are based more complete. When generating candidate questions and answers, answer fragments are first extracted from the evidence nodes and their sources are recorded before generating the questions. This order ensures that each answer fragment has a clear original source.

[0096] Based on the above embodiments, step S12 can be implemented in the following ways:

[0097] S121, the labeled questions and answers are encoded into a query vector under the current prompt version, and the activity index is retrieved based on the query vector to obtain the initial evidence node.

[0098] The query vector refers to the vector representation obtained by inputting the labeled question and answer along with the current prompt version into the encoding model, and is used for similarity retrieval in the active index.

[0099] It is understandable that the activity index stores node vectors obtained by co-encoding each evidence node with the current hint version. These vectors can be understood as the coordinates of each evidence node in the semantic space. The closer the spatial distance between two node vectors, the closer their semantic content is. The labeled question and answer itself is natural language text, and the computer cannot directly match it with the node vectors. Therefore, it is necessary to first co-encode the labeled question and answer with the current hint version and convert it into a query vector in the same semantic space. Then, the distance between the query vector and all the node vectors stored in the activity index is calculated one by one. The smaller the distance, the higher the similarity. Since the hint version determines the position of the text mapped in the semantic space during encoding, the query vector and the node vector need to be encoded using the same hint version for the comparison of their similarity to be meaningful. The above-mentioned technique of similarity retrieval by calculating vector distance is an existing technology and will not be elaborated on here.

[0100] Specifically, the labeled questions and answers, along with the current prompt version, are input into the encoding model. The encoding model outputs a query vector of fixed dimensions. Then, the distance between this query vector and each evidence node vector stored in the activity index is calculated one by one. The nodes are sorted from smallest to largest distance, and the evidence nodes corresponding to the node vectors with the smallest distance are the initial evidence nodes.

[0101] S122, extend the initial evidence nodes along the association relationship to obtain candidate nodes, and select the positive evidence node set within a preset text length range based on the coverage of each candidate node to the labeled questions and answers and the degree of repetition between each candidate node.

[0102] Among them, candidate nodes refer to all relevant evidence nodes obtained by expanding outward along the established relationship from the initial evidence node; coverage degree refers to the degree of matching between the text content of the candidate node and the knowledge points or factual elements involved in the labeled questions and answers; repetition degree refers to the repetition ratio of the text content between different candidate nodes; and the preset text length range refers to the upper limit of the sum of the text lengths of all nodes in the positive evidence node set.

[0103] Understandably, the initial evidence nodes are obtained directly through vector similarity retrieval. Although they are semantically related to the labeled question and answer, their number is limited and may only cover part of the information needed for the answer. For example, in a question about carrier liability, the initial evidence may only retrieve the definition paragraph, but related disclaimers, applicable conditions, etc., may be in other paragraphs. By expanding the association relationship, the parent node, adjacent node, or node sharing the same entity of the initial evidence node can also be included in the candidate scope. However, the number of candidate nodes increases after expansion, and not all of them can be selected. Within the limited text length, a set of nodes that most comprehensively covers the labeled question and answer and whose content is non-repetitive needs to be selected. The coverage degree is used to indicate the contribution of a single candidate node to answering the question, and the repetition degree is used to avoid selecting nodes with similar content and wasting text length. Combining the two can obtain the evidence combination with the most information within a limited length.

[0104] Specifically, starting from each initial evidence node, the process expands outward along the established hierarchical, adjacency, and entity association relationships. The expanded nodes are added to the candidate node set. Then, the coverage of key information in the labeled question and answer, as well as the text repetition between candidate nodes, are evaluated. When evaluating coverage, key information elements in the labeled question and answer can be identified, such as entity names, values, condition keywords, and step numbers. The number of these key information elements appearing in the text content of each candidate node is then counted; the more elements appearing, the higher the coverage. When evaluating repetition, the content overlap between two candidate node texts can be calculated. The overlap ratio, for example, is the proportion of the number of words or phrases that appear in both texts to the total number of words in one of the texts. The higher the ratio, the higher the degree of repetition. When selecting, you can first select the node with the highest coverage from the candidate nodes and add it to the positive evidence node set. Then, examine the remaining candidate nodes one by one and calculate the degree of repetition between each candidate node and the selected node. If the degree of repetition is lower than the preset repetition threshold and the total text length after adding it does not exceed the upper limit of the preset text length range, then add it to the positive evidence node set as well. Repeat this process until the total text length reaches the upper limit of the preset text length range or there are no candidate nodes that meet the conditions. Finally, the positive evidence node set is obtained.

[0105] S123, extract answer fragments from the set of positive evidence nodes, record the evidence node identifier and original text position corresponding to each answer fragment, and form a source mapping.

[0106] Among them, the evidence node identifier refers to the number used to identify each evidence node. This number can be used to locate the vector of the evidence node in the activity index and its corresponding original text content. The original text position refers to the specific location information of the answer fragment in the domain text, such as the chapter name, paragraph number, etc.

[0107] Specifically, text paragraphs that can independently express complete facts or conclusions can be identified from the set of positive evidence nodes and used as answer fragments. For each answer fragment, the evidence node identifier of its source and the start and end positions of the fragment in the original text are recorded. All answer fragments are associated with their corresponding evidence node identifiers and original text positions to form a source mapping.

[0108] S124, using the answer fragment as the answer constraint, the positive evidence node set as the context constraint, and the question-and-answer type corresponding to the labeled question and answer as the question type constraint, generate candidate questions, and combine the answer fragment with the candidate questions to obtain the candidate question and answer.

[0109] Among them, answer constraint refers to the restriction that the generated question must be answerable by the determined answer fragment when generating candidate questions; context constraint refers to the restriction that the knowledge scope of the generated question must not exceed the content contained in the positive evidence node set when generating candidate questions; question-answer type refers to the question type category to which the question and answer belong, such as multiple choice, fill-in-the-blank, true / false, short answer, etc.; question type constraint refers to the restriction that the question type of the generated question must be consistent with the question type of the marked question and answer when generating candidate questions; and candidate question refers to the question generated under the above three constraints that matches the answer fragment.

[0110] Understandably, determining the answer fragment first and then generating the corresponding question based on the answer fragment ensures that the answer to each candidate question and answer has a clear original source, avoiding the situation where the answer cannot be found in the original text when generating the question first and then looking for the answer. At the same time, setting context constraints can prevent the generated questions from involving knowledge outside the set of positive evidence nodes, and setting question type constraints can maintain the consistency of the generated candidate questions and answers with the labeled questions and answers in terms of form. The three constraints work together to control the generation quality of candidate questions and answers from multiple dimensions.

[0111] Specifically, the extracted answer fragments are used as target answers, the set of positive evidence nodes is used as the scope of knowledge that can be referenced, and the question-answer type corresponding to the labeled questions and answers is used as the question type requirements. Under these three constraints, candidate questions that match the answer fragments are generated. After generation, the answer fragments are used as candidate answers and combined with the candidate questions to form complete candidate question-answer pairs.

[0112] S13, after encoding the candidate questions and answers together with the current prompt version, retrieve the activity index to obtain the reverse evidence node set.

[0113] The reverse evidence node set refers to a set of evidence nodes obtained by retrieving the activity index under the same current prompt version using candidate questions and answers as query conditions, which is used to compare and verify with the positive evidence node set.

[0114] Understandably, the set of positive evidence nodes serves as a reference when generating candidate questions and answers. However, whether the generated results truly match the original text needs to be verified from another perspective. Using the generated candidate questions and answers as query conditions, the active index is searched again to see if the retrieved evidence nodes are consistent with the positive evidence nodes used during generation. If the candidate questions and answers are of high quality, the evidence nodes obtained through reverse retrieval should highly overlap with the positive evidence nodes. If the candidate questions and answers deviate from the original text, reverse retrieval will find nodes different from the positive evidence, or even nodes that contradict the content of the candidate answers. This reverse verification method is equivalent to using the generated questions and answers to perform a reverse search of the active index to see if it can be supported by the original text evidence. Reverse retrieval must be performed under the same prompt version because the vector representation of the same text is different under different prompt versions. Only by using the same prompt version encoding can the similarity ranking obtained through reverse retrieval be in the same representation space as that used during generation, and only then will the comparison results be meaningful.

[0115] Specifically, the candidate questions and answers in the candidate question and answer pool are concatenated and input into the encoding model along with the current prompt version to obtain a reverse query vector. Then, the distance between this reverse query vector and each node vector stored in the activity index is calculated one by one. The nodes are sorted in ascending order of distance, and the evidence nodes corresponding to the node vectors with the smallest distance are the reverse evidence node set.

[0116] S14. Based on the differences in node overlap between the positive evidence node set and the negative evidence node set, the source support of each answer fragment in the positive evidence node set, and the nodes in the negative evidence node set that contradict the candidate question and answer, the verification risk is determined.

[0117] Among them, node overlap difference refers to the difference in the degree of node matching between the positive evidence node set and the negative evidence node set. The greater the overlap difference, the more inconsistent the evidence referenced during generation is with the evidence retrieved during reverse verification. Source support refers to whether each answer fragment can be found in the positive evidence node set at the corresponding position in the original text. Content contradiction node refers to the evidence node found in the negative evidence node set that conflicts with the candidate answer in factual description, such as inconsistent values, missing constraints, or version conflicts.

[0118] Specifically, the positive evidence node set is compared with the negative evidence node set, and the differences in node composition between the two are calculated to obtain the node overlap difference. At the same time, according to the source mapping of the records, each answer fragment is checked one by one to see if there is a corresponding original source in the positive evidence node set to obtain the source support. By checking whether there are nodes in the negative evidence node set that contradict the candidate answer in factual description, the content contradiction nodes are identified. Combining these three aspects of information, the verification risk can be calculated.

[0119] Based on the above embodiments, step S14 can be implemented in the following ways:

[0120] S141, calculate the node overlap degree between the positive evidence node set and the negative evidence node set, and obtain the node overlap difference based on the difference between the node overlap degree and the preset overlap threshold. When the node overlap degree is not lower than the preset overlap threshold, the node overlap difference value is zero.

[0121] Among them, node overlap refers to the proportion of the number of identical nodes in the positive evidence node set and the negative evidence node set to the total number of nodes in the positive evidence node set. The preset overlap threshold is the minimum requirement for overlap set in advance. If it is lower than the threshold, the deviation between the two sets of evidence is considered unacceptable.

[0122] Understandably, node overlap reflects the degree of overlap between positive and negative evidence. If the node composition of positive and negative evidence is basically the same, and the overlap is close to 1, it means that the generated result is highly consistent with the original text. If the overlap is very low, it means that the original text on which the generation was based does not match the original text found during reverse verification, and there may be problems with the candidate questions and answers. By calculating the difference between the overlap and the preset threshold, the severity of this inconsistency can be determined. When the overlap is lower than the preset overlap threshold, the larger the difference obtained by subtracting the overlap from the threshold, the more inconsistent the two sets of evidence are, and the greater the contribution of this dimension to the verification risk. When the overlap has reached or exceeded the preset overlap threshold, it means that the consistency between positive and negative evidence has met the requirements, and this dimension should no longer contribute to the verification risk. Therefore, the node overlap difference can be set to zero.

[0123] Specifically, the number of identical nodes in the positive evidence node set and the negative evidence node set is counted, and the proportion of identical nodes to the total number of nodes in the positive evidence node set is calculated to obtain the node overlap degree. This overlap degree is compared with a preset overlap threshold. If the overlap degree is lower than the preset overlap threshold, the overlap degree is subtracted from the preset overlap threshold, and the difference is taken as the node overlap difference. The larger the difference, the more inconsistent the two sets of evidence are. If the overlap degree is not lower than the preset overlap threshold, the node overlap difference is zero.

[0124] S142, based on the proportion of each answer fragment that does not have the corresponding original text position in the positive evidence node set, the source support missing ratio is obtained, and the source support situation includes the source support missing ratio.

[0125] Among them, the missing source support ratio refers to the proportion of the number of answer fragments in the candidate answers that cannot be found in the set of positive evidence nodes to the total number of answer fragments. The higher the ratio, the more content in the answer lacks original text support.

[0126] It is understandable that each fragment in the candidate answer should be able to find its original source in the set of positive evidence nodes. If some fragments cannot find their source, it means that these contents may have been added or speculated during the generation process and lack original textual basis. By statistically analyzing the proportion of unsupported fragments, the degree to which the answer lacks original textual support can be determined.

[0127] Specifically, based on the source mapping of the records, each answer fragment in the candidate answers is checked one by one to see if there is a corresponding original text position in the positive evidence node set. The number of answer fragments without original text position support is counted, and their proportion to the total number of answer fragments is calculated to obtain the source support missing ratio.

[0128] S143, count the number of evidence nodes in the reverse evidence node set that contradict the candidate answer of the candidate question and answer in terms of value, condition or version, calculate the proportion of the number of contradictory evidence nodes to the total number of nodes in the reverse evidence node set, and obtain the contradictory node score.

[0129] Among them, the number of evidence nodes refers to the number of evidence nodes in the set of reverse evidence nodes that contradict the candidate answer, and the contradiction node score refers to the score calculated based on the number of evidence nodes that contradict the candidate answer. The more contradiction nodes there are, the higher the score, indicating that the conflict between the candidate answer and the original text is more serious.

[0130] Understandably, the evidence nodes obtained through reverse retrieval are the nodes most relevant to the semantics of the candidate question and answer. If these nodes contain content that contradicts the candidate answer in terms of numerical values, applicable conditions, or version information, it indicates that the candidate answer may contradict the facts in the domain text. For example, if the candidate answer states that the compensation amount is 30% of the total contract price, while a certain evidence node obtained through reverse retrieval clearly states that the compensation amount is 20% of the total contract price, there is a numerical contradiction between the two. The more contradictory nodes there are, the greater the possibility that the candidate answer contains factual errors.

[0131] Specifically, the text content of each evidence node in the reverse evidence node set can be compared with the candidate answers of the candidate questions and answers to identify evidence nodes that conflict with each other in terms of numerical value, applicable conditions or version information, count the number of contradictory evidence nodes, and use this number as the score of contradictory nodes.

[0132] S144, the weighted sum of the node overlap difference, the proportion of missing source support, and the score of the contradictory node is used to obtain the verification risk.

[0133] Specifically, preset weight coefficients can be assigned to the differences in node overlap, the proportion of missing source support, and the scores of contradictory nodes. Then, the three dimensions are multiplied by their respective weight coefficients, and the three products are added together to obtain the verification risk corresponding to the candidate question and answer.

[0134] S2, update the current prompt version according to the verification risk to obtain an updated prompt version, and assign the verification risk to the associated evidence node to obtain the node verification risk. Select the target evidence node according to the change amount of the prompt version and the node verification risk.

[0135] Among them, the updated prompt version refers to the new prompt version obtained after adjusting the current prompt version based on the feedback information of the verification risk; the node verification risk refers to the risk value obtained by assigning the verification risk of each candidate question and answer to the associated evidence node and accumulating the verification risk obtained by the same evidence node in multiple assignments; the prompt version change amount refers to the degree of difference between the current prompt version and the updated prompt version; and the target evidence node refers to the part of the evidence nodes that need to be recoded from all evidence nodes based on the prompt version change amount and the node verification risk.

[0136] Understandably, the verification risk reflects the shortcomings of the current prompt version in guiding question-and-answer generation. If the prompt version is not adjusted, the same generation defects will repeatedly appear in subsequent rounds. Therefore, the current prompt version can be optimized based on the verification risk to obtain an updated prompt version. At the same time, the verification risk is not evenly distributed across all evidence nodes. Some nodes may repeatedly cause quality problems, while other nodes have stable quality. If all nodes are treated indiscriminately, it will be impossible to accurately locate the root cause of the problem. Therefore, the verification risk can be attributed to specific related evidence nodes to obtain the node verification risk of each node. After the prompt version is updated, due to the change in encoding conditions, the vectors of some evidence nodes need to be re-encoded. However, the computational cost of full re-encoding is extremely high. Therefore, the target evidence nodes that most need to be re-encoded can be accurately selected by combining the change in the prompt version and the node verification risk to achieve local updates.

[0137] Based on the above embodiments, step S2 can be implemented in the following ways:

[0138] S21, Based on the verification risk, adjust the prompt parameters of the current prompt version to obtain an updated prompt version.

[0139] The prompt parameters refer to the specific adjustable content that constitutes the current prompt version. The prompt parameters can be continuous soft prompt vectors or discrete prompt templates. Soft prompt vectors refer to a set of learnable continuous numerical vectors that are concatenated with the text content and input into the model during the encoding and generation process. Prompt templates refer to discrete text instructions that include content such as domain, question type, answer range, and output fields. The prompt parameters determine the behavior of encoding and question-answer generation. Adjusting the prompt parameters changes the content of the current prompt version, thereby obtaining an updated prompt version.

[0140] Understandably, the obtained verification risk value reflects the degree of quality deviation of the generated results under the current prompt version. If the prompt parameters are not adjusted, using the same prompt version in the next iteration will continue to produce similar quality problems. Therefore, the prompt parameters can be optimized and adjusted based on the verification risk, so that the updated prompt version can reduce the verification risk and improve the quality of question and answer in the next generation.

[0141] Specifically, the verification risk of each candidate question and answer is used as the basis for adjustment. The prompt parameters of the current prompt version are optimized. The goal of the adjustment is to reduce the verification risk of the candidate questions and answers generated in the next round. After the adjustment is completed, an updated prompt version is obtained.

[0142] Based on the above embodiments, step S21 can be implemented in the following ways:

[0143] S211, construct a joint loss function that includes semantic loss, bidirectional evidence consistency loss, and positive-negative evidence comparison loss.

[0144] Among them, semantic loss refers to the degree of difference between candidate question answer and labeled question answer in semantic representation, which is used to represent the deviation between the generated result and the expected result. Two-way evidence consistency loss refers to the degree of consistency between the positive evidence node set and the negative evidence node set at the semantic level, which is used to represent the difference between the evidence referenced when generating candidate question answer and the evidence retrieved when verifying the reverse. Positive and negative evidence comparison loss refers to the difference between the positive evidence node and the negative evidence node in the retrieval score.

[0145] Understandably, adjusting the suggestion parameters requires a comprehensive metric that reflects the performance of the current suggestion version across various aspects. Semantic loss measures the generation quality from the question-and-answer content itself; bidirectional evidence consistency loss measures the credibility of the generated results from the consistency between positive and negative evidence; and positive-negative evidence contrast loss measures the retrieval quality from the discriminative power of the evidence. Combining these three metrics allows for a comprehensive evaluation of the current suggestion version's overall performance in both generation and retrieval stages. Semantic loss can be calculated by distributing the vector representations of candidate questions and answers to those of labeled questions and answers. A larger distance indicates a greater semantic difference and a higher semantic loss. Bidirectional evidence consistency loss can be calculated by distributing the average representation of the vectors of each node in the positive evidence node set to the average representation of the vectors of each node in the negative evidence node set. A larger distance indicates greater inconsistency between the positive and negative evidence and a higher consistency loss. Positive-negative evidence contrast loss can be calculated by distributing the retrieval scores between positive and negative evidence nodes. Positive evidence refers to nodes in the positive evidence node set, while negative evidence refers to nodes with contradictory content or those without supporting original text identified during the verification process. A smaller score difference between positive and negative evidence results in a higher contrast loss.

[0146] It should be noted that the calculation of the above three losses all depend on the vector representation of each text in the current prompt version. These vector representations are jointly encoded by the encoding model based on the input text and the prompt parameters of the current prompt version. Therefore, when the prompt parameters change, the vector representation of all texts will change accordingly, and the calculation results of the three losses will also change accordingly. So the joint loss function is essentially a function of the prompt parameters, and the prompt parameters can be optimized and adjusted according to this functional relationship in the future.

[0147] Specifically, first, the vector representations of candidate questions and answers and labeled questions and answers in the current prompt version are obtained, and the distance between them is calculated as the semantic loss. Then, the vector representation of each node in the positive evidence node set is obtained, and its average vector is calculated. The vector representation of each node in the negative evidence node set is obtained, and its average vector is calculated. The distance between the two average vectors is calculated as the bidirectional evidence consistency loss. The vector representations of each node in the positive and negative evidence are obtained, and their similarity with the labeled question and answer query vector is calculated as the retrieval score. The positive and negative evidence comparison loss is calculated based on the difference between the positive and negative evidence retrieval scores. Finally, the semantic loss, bidirectional evidence consistency loss, and positive and negative evidence comparison loss are combined to construct a joint loss function.

[0148] S212, Based on the verification risk, the weights of the bidirectional evidence consistency loss and the positive-negative evidence comparison loss in the joint loss function are weighted and adjusted to obtain the target joint loss function.

[0149] The target joint loss function refers to the joint loss function after verification risk weighting adjustment.

[0150] Understandably, the initial weights of the three loss terms in the joint loss function are equal. However, different candidate questions and answers have different verification risks, so they should be treated differently when adjusting the prompt parameters. Candidate questions and answers with high verification risks have more serious problems in terms of evidence consistency and evidence comparison, so the proportion of these two losses in the total loss can be increased, so that the prompt parameters will focus more on these aspects when adjusting them. Candidate questions and answers with low verification risks are already of good quality, so the weights of the various losses can be kept relatively balanced. In this way, the target joint loss function can more effectively reflect the areas where the current prompt version needs the most improvement.

[0151] Specifically, the verification risk corresponding to each candidate question and answer can be obtained. Based on the magnitude of the verification risk, the weight adjustment coefficients of the two-way evidence consistency loss and the positive-negative evidence comparison loss corresponding to each candidate question and answer can be determined. The higher the verification risk, the larger the weight adjustment coefficient. Then, the adjusted weights are used to replace the original weights of the two-way evidence consistency loss and the positive-negative evidence comparison loss in the joint loss function to obtain the target joint loss function.

[0152] S213, Based on the target joint loss function, the prompt parameters of the current prompt version are updated by gradient to obtain the updated prompt version.

[0153] Gradient update refers to the optimization method that calculates the gradient direction of the joint loss function of the target relative to the cue parameters and gradually adjusts the cue parameters along the gradient direction. When the cue parameters are differentiable numerical vectors, gradient update directly optimizes the numerical vector. When the cue parameters are discrete text templates, gradient update indirectly achieves optimization by evaluating the changes in verification risk under different templates and making the best replacement.

[0154] Understandably, the target joint loss function obtained in step S212 is a function of the cue parameters. The smaller the loss value, the better the performance of the current cue version. When the cue parameters are soft cue vectors, the soft cue vectors are a set of continuous numerical vectors. The value of the target joint loss function changes with the value of the soft cue vectors, and there is a differentiable mathematical relationship between the two. Therefore, the partial derivative of the target joint loss function with respect to each value in the soft cue vector can be calculated. The partial derivative indicates how the loss function changes when this value increases or decreases slightly. Adjusting the soft cue vector in the opposite direction of the partial derivative can gradually decrease the value of the loss function. As the quality of the question-and-answer selection gradually improves, when the prompt parameter is a text template, the objective joint loss function cannot directly take the partial derivative with respect to the text content. In this case, a template selection method can be used for optimization. The adjustable part of the text template is designed as multiple candidate options. In each iteration, we try to replace the relevant options in the current prompt template to generate multiple candidate prompt versions. We calculate the objective joint loss value corresponding to each candidate prompt version and select the candidate prompt version with the lowest loss value as the updated prompt version. In any case, the updated prompt version changes in the direction of reducing the verification risk. It is expected that higher quality candidate questions and answers will be generated in the next iteration.

[0155] Specifically, the appropriate update method is selected according to the type of the prompt parameter. When the prompt parameter is a trainable numerical vector, the partial derivatives of the target joint loss function with respect to each element in the current prompt parameter are calculated to obtain the gradient vector. The current prompt parameter is then moved by a preset step in the opposite direction of the gradient vector to obtain the updated prompt version.

[0156] When the prompt parameter is a discrete text template, since the text content is not differentiable, the partial derivative cannot be directly calculated. Multiple candidate templates can be generated, such as by replacing the answer range constraint, adjusting the evidence reference field, or modifying the question type slot. The joint loss function value of the objective under each candidate template can be calculated, and the candidate template with the lowest loss value can be selected as the updated prompt template. Gradient update is an existing technique in the field of machine learning, which will not be elaborated on here.

[0157] S22, the verification risk is assigned to each associated evidence node corresponding to the candidate question and answer, and the verification risk assigned to the same evidence node in different candidate questions and answers is aggregated to obtain the node verification risk of each evidence node.

[0158] Among them, associated evidence nodes refer to the evidence nodes involved in the generation and verification process of candidate questions and answers.

[0159] Understandably, the verification risk of a candidate question and answer is not arbitrary, but closely related to the evidence nodes used to generate it. If the text of a certain evidence node has problems such as ambiguous expression, incomplete information, or contradiction with other nodes, the candidate question and answer generated based on that node is prone to quality defects. Therefore, the verification risk of a candidate question and answer can be allocated to each associated evidence node used to generate it. At the same time, the same evidence node may be used by multiple candidate questions and answers. If the node is assigned a high verification risk in multiple candidate questions and answers, it indicates that the node itself has significant quality problems and needs to be given priority. Therefore, the verification risks assigned to the same evidence node in different candidate questions and answers can be accumulated and aggregated to obtain the node verification risk of that node.

[0160] Specifically, for each candidate question and answer, the verification risk of the candidate question and answer can be assigned to each evidence node in its positive evidence node set. The assignment method can be equal distribution or weighted distribution based on the contribution of each evidence node to the candidate answer. After the assignment is completed, for each evidence node, the verification risk assigned to the node in all candidate questions and answers is summed to obtain the node verification risk of the evidence node.

[0161] S23, calculate the degree of difference between the prompt parameters of the current prompt version and the prompt parameters of the updated prompt version, and obtain the change amount of the prompt version.

[0162] Understandably, the change in the hint version measures the extent of the modification to the hint parameters in this round of updates. If the change is large, it means that the difference between the old and new hint versions is significant, and more evidence nodes need to be re-encoded to adapt to the new encoding conditions. If the change is small, it means that the hint version has only been slightly adjusted, and the existing vectors of most evidence nodes can still be used. Only a small number of key nodes that are sensitive to changes need to be re-encoded.

[0163] Specifically, the prompt parameters of the current prompt version are compared item by item with the prompt parameters of the updated prompt version, and the overall degree of difference between the two is calculated. The method of calculating the degree of difference depends on the form of the prompt parameters. For example, when the prompt parameters are numerical vectors, the distance between the two vectors can be calculated. When the prompt parameters are discrete options, the proportion of the changed parameter items to the total number of parameter items can be counted. Based on the calculated difference value, the amount of change in the prompt version is determined.

[0164] S24. Based on the change amount of the prompt version and the node verification risk of each evidence node, determine the recoding priority of each evidence node, and select the target evidence node within the preset update quantity range.

[0165] Among them, recoding priority refers to the order in which each evidence node needs to be recoded. The higher the priority of the node, the more priority it needs to be updated. The preset update quantity range refers to the range of evidence nodes that are allowed to be recoded in each iteration.

[0166] It is understandable that after the prompt version is updated, some evidence nodes need to be recoded with the new prompt version. However, recoding all nodes in full would consume a lot of computing resources. Therefore, we can select the nodes that need to be updated the most and complete the partial update within a limited number of updates, thus allocating the limited computing resources to the nodes that need to be updated the most.

[0167] Specifically, the recoding priority of each evidence node can be determined based on the amount of version change and the node verification risk of each evidence node. Then, within the preset update range, the evidence node with the highest recoding priority is selected as the target evidence node.

[0168] Based on the above embodiments, step S24 can be implemented in the following ways:

[0169] S241, obtain the query hit frequency of each evidence node within a preset time period.

[0170] The preset time period refers to the historical time period used to count the frequency of query hits, such as the most recent day or the most recent week. The query hit frequency refers to the number of times each evidence node is retrieved within the preset time period.

[0171] Understandably, in addition to indicating the amount of version changes and node verification risks, the frequency of node usage is also an important reference factor. If a frequently retrieved node has a problem, the impact will be wider and it should be updated first. Conversely, even if a rarely used node has a high verification risk, its impact on the overall quality is limited and its priority can be appropriately reduced.

[0172] Specifically, within a preset time period, the evidence nodes returned each time the activity index is retrieved are counted, the number of times each evidence node is retrieved is recorded, and the query hit frequency of each evidence node is obtained.

[0173] S242, normalize and weight the change amount of the prompt version, the node verification risk of each evidence node and the query hit frequency to obtain the recoding priority of each evidence node.

[0174] Normalized weighting refers to a calculation method that unifies multiple indicators with different value ranges and dimensions to the same scale, and then sums them according to their respective importance weights.

[0175] Understandably, the version change amount is a global value, while the node verification risk and query hit frequency are different values ​​for each evidence node. The value ranges of the three may vary greatly, so they cannot be directly weighted. Instead, the three can be normalized separately and mapped to the same value range. Then, they can be weighted and summed according to the pre-set weight coefficients. The recoding priority calculated in this way integrates information from three dimensions: the degree of change in coding conditions, the historical quality issues of nodes, and the frequency of node use.

[0176] S243, select a preset number of evidence nodes from high to low according to the recoding priority to obtain the target evidence node.

[0177] The preset quantity refers to the number of evidence nodes that need to be re-encoded in this round, which is determined within the preset update quantity range.

[0178] Specifically, each evidence node is sorted from high to low according to the calculated recoding priority score, and the top-ranked number of evidence nodes are selected as target evidence nodes.

[0179] S3, based on the update prompt version, re-encode the selected target evidence node and write it into the incremental index, and record the migration service status of the target evidence node.

[0180] The incremental index refers to an auxiliary index that exists in parallel with the active index. It is used to store the new version vector obtained after recoding based on the update prompt version. The migration service status refers to the identification information that records the stage of each evidence node during the migration process from the active index to the incremental index. The migration service status can determine which index is currently providing retrieval services for the node.

[0181] Understandably, due to the update of the hint version, the vector encoded by the target evidence node under the old hint version can no longer accurately reflect the semantic space of the new hint version, and needs to be re-encoded based on the updated hint version. However, if the new vector is directly overwritten into the active index, both old and new encoding results will exist in the active index before the recoding is fully completed, leading to chaotic search results. Therefore, the new vector can be written into an independent incremental index and exist in parallel with the active index, thereby avoiding interference with the search service being provided in the active index. At the same time, in order to accurately grasp the migration progress of each target evidence node in subsequent search scheduling, its migration service status can be recorded, providing a basis for the coordinated scheduling of the active index and the incremental index.

[0182] Based on the above embodiments, step S3 can be implemented in the following ways:

[0183] S31, re-encode the update prompt version and the selected target evidence node to obtain a new version vector for each target evidence node, and write each new version vector into the incremental index.

[0184] The new version vector refers to the vector representation obtained after re-encoding the target evidence node according to the update prompt version.

[0185] It is understandable that the vector of the target evidence node in the active index is encoded based on the current prompt version. When the prompt version is updated, the semantic representation of the same evidence node changes under the new prompt version, and it needs to be re-encoded to ensure that the query vector and the node vector are in the same representation space during subsequent retrieval. Specifically, for each target evidence node, its text content and title path are obtained. The updated prompt version, text content and title path are concatenated in the same way as in step S111 to obtain the encoding input. The encoding input is sent to the encoding model to obtain the new version vector of the target evidence node. Then, the new version vector is written into the incremental index. When writing, the node identifier and the version identifier of the updated prompt version are used as the association information to facilitate the identification of the vector source during subsequent retrieval.

[0186] S32, record the migration service status of each of the target evidence nodes. The migration service status includes at least "writing in progress", "verified", and "service new version". The migration service status of the evidence nodes that are not selected is "retrieval service provided by the activity index".

[0187] Among them, "writing in progress" refers to the state in which the new version vector of the target evidence node is being written to the incremental index or has been written but has not yet been verified. "verified" refers to the state in which the new version vector of the target evidence node has been written and passed verification, but has not yet been officially switched to provide retrieval services. "serving the new version" refers to the state in which the new version vector of the target evidence node has passed verification and has been officially switched to provide retrieval services by the incremental index. "Retrieval service" refers to the function that when a query comes in, the vector of the evidence node participates in similarity calculation and can be returned as a retrieval result.

[0188] Understandably, after the new version vector is written to the incremental index, it cannot be immediately provided to the public because there may still be risks such as incomplete writing or data verification failure. By recording the migration service status, it is possible to clearly identify which stage of the migration process each target evidence node is currently in. Both the "writing" and "verified" statuses indicate that the node has not yet been officially switched and is still being provided to the public by the old version vector in the active index. Only when the status changes to "serving the new version" does it mean that the new version vector of the node is ready to be used. By controlling the status of each node in this way, it can be ensured that each evidence node is provided to the public by only one index at any given time, avoiding confusion caused by both old and new vectors participating in the search. Evidence nodes that are not selected are not involved in this round of recoding, and their migration service status is directly assumed to be provided to the public by the active index.

[0189] S33, when the migration service status is in writing or verified, the active index continues to provide retrieval services for the target evidence node.

[0190] Understandably, when the migration service status of the target evidence node is "writing", it means that the new version vector has not yet been written and there is no available vector for that node in the incremental index. At this time, if the node is searched from the incremental index, the corresponding vector will not be found. Therefore, the active index continues to provide the search service. When the migration service status is "verified", although the new version vector has been written to the incremental index and passed the verification, the search service has not yet been officially switched. At this time, the active index still provides the search service, which can ensure that before the official switch, the search results always come from the stable old version vector, avoiding fluctuations in search results at the moment of switch.

[0191] S34, when the migration service status changes to a new service version, the corresponding target evidence node is provided with retrieval services by the incremental index.

[0192] Understandably, when the migration service status of the target evidence node changes to the new version of the service, it means that the new version vector of the node has been ready in the incremental index and passed the verification, and can officially take over the retrieval service. At this time, the retrieval request of the node should obtain the new version vector from the incremental index, rather than obtain the old version vector from the active index, because the new version vector is encoded according to the update prompt version, which can more accurately reflect the semantic space of the update prompt version, thereby improving the consistency between the retrieval results and the update prompt version.

[0193] S4. Based on the migration service status, schedule the active index and the incremental index for retrieval respectively, merge the corresponding retrieval results after score calibration, generate or verify candidate questions and answers based on the merged results, and update the version pointer of the active index when the merged results meet the index switching conditions.

[0194] The search results refer to the evidence nodes and corresponding search scores retrieved from the active index or incremental index. Score calibration refers to the process of adjusting the search scores returned by the active index and the incremental index to the same numerical scale. Index switching conditions refer to the pre-set conditions for determining whether the incremental index can replace the active index as the new main index. Version pointer refers to the indication information that identifies the prompt version corresponding to the current active index.

[0195] Understandably, during the migration of target evidence nodes from the active index to the incremental index, some nodes are still served by the active index, while others are served by the incremental index. If a search is conducted only from a single index, migrated or unmigrated nodes will be missed, resulting in incomplete search results. Therefore, the two indexes can be scheduled for retrieval based on the migration service status. However, since the vectors in the active index and the incremental index are encoded based on different prompt versions, the retrieval scores returned by the two are on different numerical scales. Direct merging may lead to sorting errors. Therefore, the retrieval scores can be calibrated to bring them to the same scale before merging. The merged complete search results can continue to support the generation and verification of candidate questions and answers. When the migration is completed and the quality of the merged results meets the preset requirements, the incremental index can be switched to the new active index to complete this round of migration.

[0196] Based on the above embodiments, step S4 can be implemented in the following ways:

[0197] S41, based on the migration service status, retrieve unselected evidence nodes and target evidence nodes with migration service status of writing or verification from the activity index to obtain a first retrieval result, and retrieve target evidence nodes with migration service status of new service version from the incremental index to obtain a second retrieval result.

[0198] The first search result includes the retrieved evidence nodes and their corresponding search scores, and the second search result includes the retrieved evidence nodes and their corresponding search scores.

[0199] The first search result refers to the evidence node retrieved from the active index and its corresponding search score, while the second search result refers to the evidence node retrieved from the incremental index and its corresponding search score. The search score is the similarity score between the query vector and the node vector of the retrieved evidence node. The higher the similarity, the higher the search score, indicating that the evidence node has a stronger semantic association with the query content.

[0200] Understandably, in step S32, the migration service status of each evidence node has been recorded. Unselected evidence nodes are always retrieved by the active index. Target evidence nodes with a migration service status of "writing" or "verified" are still retrieved by the active index. Target evidence nodes with a migration service status of "serving a new version" are retrieved by the incremental index. Therefore, during retrieval, it is necessary to determine which index to retrieve the node from based on its migration service status. If only the active index is retrieved, nodes already served by the incremental index may be missed. If only the incremental index is retrieved, nodes that have not yet completed migration may be missed. Only by scheduling the two indexes separately for retrieval can it be ensured that all evidence nodes are covered and the retrieval results are complete and without omissions.

[0201] Specifically, the similarity between the query vector and the node vectors of unselected evidence nodes in the active index and target evidence nodes whose migration service status is "writing" or "verified" can be calculated. The nodes with the highest similarity and their corresponding search scores are used as the first search result. At the same time, the similarity between the query vector and the node vectors of target evidence nodes whose migration service status is "new version of service" in the incremental index can be calculated. The nodes with the highest similarity and their corresponding search scores are used as the second search result.

[0202] S42, normalize the search scores of the first search result and the second search result respectively, and then merge the processed first search result and the second search result to obtain the merged search result.

[0203] Normalization refers to the process of mapping search scores of different numerical scales to the same numerical range. Deduplication and merging refers to merging the first and second search results together. For the same evidence node appearing in the two search results, only one record is retained. Merged search results refer to the complete search results obtained after deduplication and merging, which include all relevant evidence nodes and their calibrated search scores.

[0204] Understandably, the node vectors in the active index are encoded based on the current hint version, while the node vectors in the incremental index are encoded based on the updated hint version. Because the two hint versions are different, the encoding models map the text to the semantic space in different ways, resulting in the retrieval scores returned by the two indexes potentially falling on different numerical scales. For example, the retrieval scores returned by the active index might be concentrated between 0.6 and 0.9, while the retrieval scores returned by the incremental index might be concentrated between 0.3 and 0.7. If the two sets of scores are directly sorted together, nodes in the active index will occupy a higher ranking due to their naturally higher scores, while nodes in the incremental index will be ranked lower due to their naturally lower scores. This ranking bias is not due to the active index being more active. The nodes in the dynamic index are not truly more relevant, but rather because the score scales of the two indexes are different. Therefore, the two sets of scores can be normalized first, mapping them to the same numerical range so that the two sets of scores can be directly compared. The normalization process can use existing normalization methods such as max-min normalization or quantile mapping. This solution does not limit itself to these methods. After normalization, the two sets of search results are merged together. Since the same evidence node may appear in both the first and second search results, for example, if the migration service status of a certain target evidence node has just changed from verified to a new service version, both indexes may return the node during the search. In this case, deduplication is required, keeping only one record, and finally obtaining the merged search results.

[0205] S43, use the merged search results as context to generate candidate questions and answers, or verify the source support of the answer fragments in the generated candidate questions and answers based on the evidence nodes in the merged search results.

[0206] Among them, context refers to the text content of each evidence node in the merged search results, and source support verification refers to the verification process of checking whether each answer fragment in the generated candidate questions and answers can find the corresponding original text content in the evidence nodes of the merged search results.

[0207] Understandably, during the migration, the active index and the incremental index coexist. The merged search results contain all available evidence nodes at the current moment, representing the most complete set of evidence at that time. Using the merged search results as context to generate candidate questions and answers ensures that the evidence used in the generation process is complete and that key evidence is not missed due to some nodes being migrated. Similarly, when verifying the source support of the generated candidate questions and answers, it is also necessary to check whether the answer fragments are supported by the original text based on the evidence nodes in the merged search results. If only the active index or only the evidence nodes in the incremental index are used for verification, the verification results may be inaccurate due to the omission of some nodes.

[0208] Specifically, when generating candidate questions and answers, the text content of each evidence node in the merged search results can be used as context, combined with the current prompt version to generate candidate questions and answers. When verifying source support, each answer fragment in the generated candidate questions and answers can be checked one by one, and the corresponding text can be searched in the original text content of the evidence nodes in the merged search results. If a certain answer fragment cannot be found in the original text content of all evidence nodes in the merged search results, it is determined that the answer fragment lacks source support.

[0209] S44, when the merged search results meet the preset index switching conditions, the new version vectors of each target evidence node in the incremental index are merged into the active index, and the version pointer of the active index is updated.

[0210] Understandably, index switching is the final step in the migration process. Its purpose is to make the new version vector based on the update prompt version code officially effective, while ensuring that unselected evidence nodes are not lost and the retrieval service is not interrupted. If a complete replacement method is adopted, the incremental index directly replaces the active index. Since the incremental index only stores the new version vector of the target evidence node, the vectors of the unselected evidence nodes will be lost, making these nodes unsearchable. Therefore, this solution can use an incremental merging method to complete the switch. That is, during the switch, only the new version vector of each target evidence node in the incremental index is written to the active index, while the vectors of the unselected evidence nodes remain unchanged in the active index. After the merge is completed, the version pointer of the active index is updated to the version identifier of the update prompt version, and the prompt version corresponding to the vector that is currently effective in the active index is recorded. At the same time, the incremental index is cleared for the next iteration to write the new version vector.

[0211] Specifically, when the merged search results meet the preset index switching conditions, the new version vectors of each target evidence node in the incremental index are written into the active index one by one, while the vectors of unselected evidence nodes remain unchanged. Then, the version pointer of the active index is updated to the version identifier of the updated prompt version, thus completing this round of index switching.

[0212] The index switching conditions include:

[0213] The proportion of nodes in the target evidence nodes whose migration service status is the new version service reaches a preset coverage threshold.

[0214] Among them, the preset coverage threshold refers to the minimum percentage requirement that the number of target evidence nodes that have completed migration is set in advance.

[0215] Understandably, if most of the target evidence nodes have not yet completed migration, performing an index switch at this time will result in a large number of missing node vectors in the incremental index, leading to incomplete search results. Therefore, it is necessary to ensure that a sufficient proportion of the target evidence nodes have completed migration and are in the new service version state before performing a switch. For example, the preset coverage threshold can be set to 95%, that is, when the proportion of the number of target evidence nodes that have migrated to the new service version state to the total number of target evidence nodes reaches 95%, the coverage condition is considered to be met.

[0216] Specifically, the number of target evidence nodes whose migration service status is the new version can be counted, and then divided by the total number of target evidence nodes to obtain the percentage of nodes that have completed migration. This percentage is then compared with a preset coverage threshold. If the percentage is not lower than the preset coverage threshold, the coverage condition is deemed to be met.

[0217] The ranking overlap between the merged search results and the search results of the active index before migration reaches a preset stable threshold.

[0218] Among them, the ranking overlap refers to the degree of overlap between the top-ranked evidence nodes in the merged search results and the top-ranked evidence nodes obtained by using only the active index before migration. The preset stable threshold refers to the minimum proportion required for the ranking overlap to be achieved.

[0219] Understandably, the quality of search results should not be lower than before the index switch. If the evidence nodes retrieved after the switch are significantly different from those before, it indicates that the new version vectors in the incremental index may lead to a decrease in search quality. Therefore, it is necessary to compare the ranking of the merged search results with the ranking of the active index before migration to verify whether the rankings are basically consistent. When calculating the ranking overlap, a set of preset test queries can be used to search the active index before the migration begins. The list of top-ranked evidence nodes returned by each test query is recorded as the baseline ranking before migration. During the migration, the same test queries are used to perform a dual-index merged search, and the list of top-ranked evidence nodes in the merged search results is recorded. Then, the proportion of the number of identical nodes in the two lists to the total number of top-ranked nodes is calculated. This proportion is the ranking overlap.

[0220] Specifically, before the migration begins, a preset test query can be used to retrieve data from the active index, and the list of top-ranked evidence nodes corresponding to each test query can be recorded as the baseline ranking. During the migration, the same test query can be used to perform a dual-index merge retrieval, and the list of top-ranked evidence nodes in the merge retrieval results can be recorded. Then, the ranking overlap between the merge retrieval results corresponding to each test query and the baseline ranking can be calculated. The average ranking overlap of each test query can be taken, and when the average value is not lower than the preset stability threshold, the stability condition is considered met.

[0221] The bidirectional evidence consistency pass rate of candidate questions and answers generated based on merged search results is no less than the baseline level before migration.

[0222] Among them, the two-way evidence consistency pass rate refers to the proportion of candidate questions and answers that pass the two-way evidence consistency verification out of the total number of candidate questions and answers generated based on the merged search results. The baseline level refers to the two-way evidence consistency pass rate of candidate questions and answers generated using the active index before the migration begins.

[0223] Understandably, the ultimate goal of index switching is to improve the quality of question-answer pairs. Two-way evidence consistency verification is a verification mechanism established in step S1, which includes two directions: forward retrieval to generate candidate questions and answers and reverse retrieval to verify candidate questions and answers. Candidate questions and answers that pass the two-way evidence consistency verification indicate that their answers are supported by sufficient and stable evidence. Therefore, the two-way evidence consistency pass rate can be used to measure the overall quality of candidate questions and answers. The baseline level is the two-way evidence consistency pass rate of candidate questions and answers generated by the active index before the migration, which represents the quality benchmark before the switch. The pass rate after the switch should not be lower than this benchmark; otherwise, it indicates that the new version vector in the incremental index may have affected the generation quality of candidate questions and answers.

[0224] Specifically, several candidate questions and answers can be generated based on the merged search results. Then, the two-way evidence consistency verification described in steps S13 and S14 is performed on each candidate question and answer. The number of candidate questions and answers that pass the verification is counted and divided by the total number of candidate questions and answers generated to obtain the two-way evidence consistency pass rate. This pass rate is compared with the baseline level before migration. When the pass rate is not lower than the baseline level, the quality condition is determined to be met.

[0225] S5, In the event of a write or switch failure, the corresponding evidence node is restored to be served by the active index according to the migration service status, and the above steps are iteratively executed until the preset convergence condition is met, and then the question-answer pair dataset is output.

[0226] Among them, write or switch exception refers to the failure that occurs during the process of writing the new version vector to the incremental index or updating the version pointer; preset convergence condition refers to the pre-set condition for determining when the iterative optimization can stop; and question-answer pair dataset refers to the high-quality question-answer pair set finally output after multiple rounds of iterative verification.

[0227] It is understandable that during the process of writing new version vectors to the incremental index or performing index switching, unexpected reasons such as hardware failure or network fluctuations may cause write failure or switching failure. If these anomalies are not handled, the erroneous evidence nodes may not be effectively providing services in either the active index or the incremental index, or may be served by both indexes simultaneously, leading to errors or omissions in the search results. Therefore, when an anomaly occurs, the corresponding evidence node can be restored to continue providing services by the active index based on the migration service status to ensure that the search service is uninterrupted and error-free. Furthermore, the optimization of the version and the updating of evidence nodes is a gradual process. Performing only one round is often insufficient to achieve the ideal question-answer quality. Therefore, regardless of whether the current round of writing or switching is successfully completed or experiences anomaly recovery, it is necessary to continuously loop through the previous generation, verification, update, and migration steps until the verification risk tends to stabilize or reaches the preset iteration limit. Only then will the loop stop, and the candidate questions and answers that meet the quality standards will be selected, ultimately outputting the question-answer pair dataset.

[0228] Specifically, when performing operations to write incremental indexes or update version pointers, if a write or switching exception occurs, the evidence node can be restored to continue providing retrieval services by the active index based on the current migration service status of the corresponding evidence node. If no exception occurs, the write or switching operation of this round is completed normally. After completing this round of operations, it is determined whether the preset convergence condition is met. If the preset convergence condition is not met, the above steps are iterated again with the current updated prompt version and index status. If the preset convergence condition is met, the iteration is stopped, and candidate questions and answers with verification risks lower than the preset risk threshold are selected and the question and answer pair dataset is output.

[0229] Based on the above embodiments, step S5 can be implemented in the following ways:

[0230] S51, when the new version vector fails to be written to the incremental index, the migration service status of the corresponding target evidence node is marked as write failure, and the active index continues to provide retrieval services for the target evidence node.

[0231] Write failure refers to a situation where the write operation fails to complete successfully due to storage failure, network interruption, or data verification failure during the process of writing the new version vector of the target evidence node to the incremental index.

[0232] Specifically, during the process of writing the new version vector to the incremental index, if it is detected that the write operation has failed to complete, the migration service status of the corresponding target evidence node can be marked as write failure. The target evidence node continues to be provided with retrieval services by the active index, and the old version vector of the node stored in the active index remains unchanged. Subsequent retrieval requests involving the node will still be obtained from the active index.

[0233] S52, when the version pointer update of the active index fails, the target evidence node whose migration service status is the new version of the service is restored to be provided with retrieval services by the active index.

[0234] Understandably, updating the version pointer is the final action of index switching. When the version pointer update is successful, the incremental index will officially replace the active index as the new primary index. However, if the version pointer update fails, it means that the switching process has not been completed. At this time, the migration service status of some target evidence nodes may have changed to the new service version, that is, the retrieval service of these nodes has already pointed to the incremental index. However, due to the failure of the version pointer update, the incremental index has not yet been officially confirmed as the new active index. The retrieval service of these nodes is in an unstable intermediate state. In order to avoid abnormal retrieval results, the target evidence nodes whose migration service status is the new service version can be restored to the retrieval service provided by the active index, so that all nodes return to the stable state before the switch.

[0235] S53, repeat the above steps of generating and verifying candidate questions and answers, updating the prompt version, selecting and recoding the target evidence node, writing the incremental index, scheduling, retrieving and merging the active index and the incremental index, and updating the version pointer.

[0236] Understandably, the optimization of the prompt version and the updating of evidence nodes are a gradual process. In a single iteration, the adjustment range of the prompt version is limited, and it may only correct some generation defects. The quality of the candidate questions and answers may not yet reach an ideal state. At the same time, the target evidence nodes selected in this round are only a part of all evidence nodes. The quality issues of the remaining nodes may need to be addressed gradually in subsequent rounds. Therefore, it is necessary to repeat all the above steps. In each iteration, based on the verification risk of the previous round, the prompt version is optimized, new target evidence nodes are selected, and the code is recoded and migrated to gradually improve the quality of the candidate questions and answers in each iteration.

[0237] S54, when the rate of change of the verification risk is lower than the preset change threshold or the number of iterations reaches the preset upper limit, it is determined that the preset convergence condition is met and the iteration is stopped.

[0238] Among them, the preset change threshold is a critical value that is set in advance, and when the change in verification risk between two adjacent iterations is lower than this value, the optimization is considered to have reached a stable state. The preset number of iterations upper limit is the maximum number of iterations set in advance.

[0239] Understandably, iterative optimization cannot continue indefinitely and requires a clear stopping criterion. The rate of change of verification risk reflects the improvement in the quality of candidate questions and answers between two adjacent iterations. If the rate of change is already very low, it means that the quality improvement brought by continuing iteration is very limited. At this point, iteration can be stopped to avoid meaningless consumption of computational resources. At the same time, in order to prevent the number of iterations from becoming too many due to the continuous and slow decrease of verification risk, a preset upper limit on the number of iterations can be set as a fallback condition. When the number of iterations reaches the preset upper limit, iteration stops regardless of the rate of change of verification risk. Meeting either of the above two conditions is considered to meet the preset convergence condition.

[0240] Specifically, after each iteration, the difference between the average verification risk of the current iteration and the average verification risk of the previous iteration can be calculated. This difference can be divided by the average verification risk of the previous iteration to obtain the rate of change of verification risk. The rate of change can be compared with a preset change threshold. When the rate of change is lower than the preset change threshold, it is determined that the preset convergence condition is met and the iteration stops. At the same time, the current iteration number is recorded. When the number of iterations reaches the preset upper limit, regardless of the rate of change of verification risk, it is determined that the preset convergence condition is met and the iteration stops.

[0241] S55 outputs candidate questions and answers whose verification risk is lower than a preset risk threshold, thus obtaining a question-answer pair dataset.

[0242] Among them, the preset risk threshold refers to the pre-set critical value at which the verification risk of a candidate question and answer is considered to meet the quality standard when it is lower than the value.

[0243] Understandably, after multiple iterations, a large number of candidate questions and answers will be accumulated. However, not all candidate questions and answers meet the output standards. Some candidate questions and answers may still have problems such as lack of original text support, contradiction with domain text, or unstable evidence. Therefore, a preset risk threshold can be set as a quality screening standard. Only candidate questions and answers with verification risks below the threshold will be output to ensure that every question and answer pair in the final question and answer pair dataset has undergone quality verification and meets the preset quality standards.

[0244] See Figure 2 This is a schematic diagram of a domain question-answering pair automatic extraction system provided in an embodiment of the present invention. The system includes:

[0245] The generation module is used to encode the evidence nodes of the domain text into the activity index, generate candidate questions and answers based on the current prompt version and the activity index, and determine the verification risk based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index.

[0246] The update module is used to update the current prompt version according to the verification risk to obtain an updated prompt version, and to allocate the verification risk to the associated evidence node to obtain the node verification risk, and to select the target evidence node according to the change amount of the prompt version and the node verification risk;

[0247] The recording module is used to re-encode the selected target evidence node based on the update prompt version and write it into the incremental index, and record the migration service status of the target evidence node.

[0248] The retrieval module is used to schedule the active index and the incremental index to perform retrieval according to the migration service status, merge the corresponding retrieval results after score calibration, generate or verify candidate questions and answers based on the merged results, and update the version pointer of the active index when the merged results meet the index switching conditions.

[0249] The output module is used to restore the corresponding evidence node to be served by the active index according to the migration service status when a write or switch failure occurs, and to iteratively execute the above steps until the preset convergence condition is met before outputting the question-answer pair dataset.

[0250] Figure 2 The apparatus of the illustrated embodiment can be used to perform corresponding actions. Figure 1 The steps in the method embodiments shown are implemented in a similar manner and have similar technical effects, and will not be repeated here.

[0251] See Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device 30 includes: a processor 31, a memory 32, and a computer program; wherein...

[0252] The memory 32 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0253] Processor 31 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0254] Alternatively, the memory 32 can be either standalone or integrated with the processor 31.

[0255] When the memory 32 is a device independent of the processor 31, the device may further include:

[0256] Bus 33 is used to connect the memory 32 and the processor 31.

[0257] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically extracting domain-specific question-answering pairs, characterized in that, include: Evidence nodes of the domain text are encoded into the activity index. Candidate questions and answers are generated based on the current prompt version and the activity index. Verification risk is determined based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index. The current prompt version is updated based on the verification risk to obtain an updated prompt version, and the verification risk is assigned to the associated evidence node to obtain the node verification risk. The target evidence node is selected based on the change amount of the prompt version and the node verification risk. Based on the update prompt version, the selected target evidence nodes are re-encoded and written into the incremental index, and the migration service status of the target evidence nodes is recorded. According to the migration service status, the active index and the incremental index are scheduled to be retrieved respectively. The corresponding retrieval results are merged after score calibration. Candidate questions and answers are generated or verified based on the merged results. The version pointer of the active index is updated when the merged results meet the index switching conditions. In the event of a write or switch failure, the corresponding evidence node is restored to be served by the active index according to the migration service status, and the above steps are iteratively executed until the preset convergence condition is met before outputting the question-answer pair dataset.

2. The method according to claim 1, characterized in that, The process of encoding evidence nodes from the domain text into an activity index, generating candidate questions and answers based on the current prompt version and the activity index, and determining verification risks based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index includes: The domain text is divided into multiple levels to obtain evidence nodes. The relationship between each evidence node is established. Each evidence node is encoded together with the current prompt version and then loaded into the activity index. The initial evidence node is obtained by retrieving the activity index based on the labeled question and answer, and a positive evidence node set is obtained by expanding along the association relationship. Candidate questions and answers are generated based on the positive evidence node set, and the candidate answers of the candidate questions and answers contain the source mapping of each answer fragment and the corresponding evidence node. After encoding the candidate questions and answers together with the current prompt version, the activity index is retrieved to obtain a set of reverse evidence nodes; The verification risk is determined based on the differences in node overlap between the positive evidence node set and the negative evidence node set, the source support of each answer fragment in the positive evidence node set, and the nodes in the negative evidence node set that contradict the candidate question and answer.

3. The method according to claim 2, characterized in that, The process involves retrieving the activity index based on the labeled question-and-answer format to obtain initial evidence nodes, expanding along the association relationship to obtain a set of positive evidence nodes, and generating candidate questions and answers based on the set of positive evidence nodes. The candidate answers for each question and answer include a source mapping between each answer fragment and the corresponding evidence node, including: The labeled questions and answers are encoded into a query vector under the current prompt version, and the activity index is retrieved based on the query vector to obtain the initial evidence node; The initial evidence nodes are extended along the association relationship to obtain candidate nodes. Based on the coverage of each candidate node to the labeled questions and answers and the degree of repetition between each candidate node, the set of positive evidence nodes is selected within a preset text length range. Extract answer fragments from the set of positive evidence nodes, record the evidence node identifier and original text position corresponding to each answer fragment, and form a source mapping; Candidate questions are generated by using the answer fragment as the answer constraint, the set of positive evidence nodes as the context constraint, and the question-answer type corresponding to the labeled question and answer as the question type constraint. The answer fragment and the candidate questions are then combined to obtain the candidate question and answer.

4. The method according to claim 2, characterized in that, The step of determining verification risk based on the overlap difference between the positive evidence node set and the negative evidence node set, the source support of each answer fragment in the positive evidence node set, and the nodes in the negative evidence node set that contradict the candidate question and answer includes: Calculate the node overlap degree between the positive evidence node set and the negative evidence node set, and obtain the node overlap difference based on the difference between the node overlap degree and the preset overlap threshold. When the node overlap degree is not lower than the preset overlap threshold, the node overlap difference value is zero. The proportion of missing source support is obtained based on the proportion of each answer fragment that does not have the corresponding original text position in the set of positive evidence nodes. The source support situation includes the proportion of missing source support. The number of evidence nodes in the reverse evidence node set that contradict the candidate answer of the candidate question and answer in terms of value, condition, or version is counted, and the proportion of the number of contradictory evidence nodes to the total number of nodes in the reverse evidence node set is calculated to obtain the contradictory node score. The verification risk is obtained by weighted summation of the node overlap difference, the proportion of missing source support, and the score of the contradictory node.

5. The method according to claim 1, characterized in that, The process of updating the current prompt version based on the verification risk to obtain an updated prompt version, and allocating the verification risk to associated evidence nodes to obtain node verification risks, and selecting target evidence nodes based on the change in the prompt version and the node verification risks, includes: Based on the verification risk, the prompt parameters of the current prompt version are adjusted to obtain an updated prompt version; The verification risk is assigned to each associated evidence node corresponding to the candidate question and answer, and the verification risk assigned to the same evidence node in different candidate questions and answers is aggregated to obtain the node verification risk of each evidence node. Calculate the degree of difference between the prompt parameters of the current prompt version and the prompt parameters of the updated prompt version to obtain the change amount of the prompt version; Based on the change amount of the prompt version and the node verification risk of each evidence node, the recoding priority of each evidence node is determined, and the target evidence node is selected within the preset update quantity range.

6. The method according to claim 5, characterized in that, The step of adjusting the prompt parameters of the current prompt version based on the verification risk to obtain the updated prompt version includes: Construct a joint loss function that includes semantic loss, bidirectional evidence consistency loss, and positive-negative evidence comparison loss; Based on the verification risk, the weights of the bidirectional evidence consistency loss and the positive-negative evidence comparison loss in the joint loss function are weighted and adjusted to obtain the target joint loss function. The updated suggestion version is obtained by performing gradient updates on the suggestion parameters of the current suggestion version based on the target joint loss function; The step of determining the recoding priority of each evidence node based on the version change amount and the node verification risk of each evidence node, and selecting target evidence nodes within a preset update quantity range, includes: Obtain the query hit frequency of each evidence node within a preset time period; The recoding priority of each evidence node is obtained by normalizing and weighting the change amount of the prompt version, the node verification risk of each evidence node, and the query hit frequency. A preset number of evidence nodes are selected from high to low according to the recoding priority to obtain the target evidence nodes.

7. The method according to claim 1, characterized in that, The process of re-encoding the selected target evidence nodes based on the updated version and writing them into the incremental index, and recording the migration service status of the target evidence nodes, includes: The update prompt version and the selected target evidence node are re-encoded to obtain a new version vector for each target evidence node, and each new version vector is written into the incremental index; Record the migration service status of each of the target evidence nodes. The migration service status includes at least "writing", "verified", and "service new version". The migration service status of evidence nodes that are not selected is that the activity index provides retrieval services. When the migration service status is "writing" or "verified", the active index continues to provide retrieval services for the target evidence node; When the migration service status changes to a new service version, the corresponding target evidence node is retrieved by the incremental index.

8. The method according to claim 7, characterized in that, The process of scheduling the active index and the incremental index for retrieval based on the migration service status, merging the corresponding retrieval results after score calibration, generating or verifying candidate questions and answers based on the merged results, and updating the version pointer of the active index when the merged results meet the index switching conditions includes: Based on the migration service status, retrieve the unselected evidence nodes and the target evidence nodes whose migration service status is "writing" or "verified" from the activity index to obtain the first retrieval result; retrieve the target evidence nodes whose migration service status is "service new version" from the incremental index to obtain the second retrieval result. The first search result includes the retrieved evidence nodes and their corresponding search scores; the second search result includes the retrieved evidence nodes and their corresponding search scores. The search scores of the first search result and the second search result are normalized respectively. The processed first search result and the second search result are deduplicated and merged to obtain the merged search result. The merged search results are used as context to generate candidate questions and answers, or the source support verification of the answer fragments in the generated candidate questions and answers is performed based on the evidence nodes in the merged search results. When the merged search results meet the preset index switching conditions, the new version vectors of each target evidence node in the incremental index are merged into the active index, and the version pointer of the active index is updated.

9. The method according to claim 8, characterized in that, In the event of a write or switch anomaly, the corresponding evidence node is restored to be served by the active index according to the migration service status, and the above steps are iteratively executed until a preset convergence condition is met before outputting the question-answer pair dataset, including: When the new version vector fails to be written to the incremental index, the migration service status of the corresponding target evidence node is marked as write failure, and the active index continues to provide retrieval services for the target evidence node; When the version pointer update of the active index fails, the target evidence node whose migration service status is the new version will be restored to having the retrieval service provided by the active index. Repeat the above steps of generating and verifying candidate questions and answers, updating the prompt version, selecting and recoding the target evidence node, writing the incremental index, scheduling, retrieving and merging the active index and the incremental index, and updating the version pointer. When the rate of change of the verification risk is lower than a preset change threshold or the number of iterations reaches the preset maximum number of iterations, it is determined that the preset convergence condition is met and the iteration stops. Output candidate questions and answers whose verification risk is lower than a preset risk threshold to obtain a question-answer pair dataset.

10. A domain-specific question-answering pair automatic extraction system, characterized in that, include: The generation module is used to encode the evidence nodes of the domain text into the activity index, generate candidate questions and answers based on the current prompt version and the activity index, and determine the verification risk based on the comparison results between the candidate questions and answers and the evidence nodes in the activity index. The update module is used to update the current prompt version according to the verification risk to obtain an updated prompt version, and to allocate the verification risk to the associated evidence node to obtain the node verification risk, and to select the target evidence node according to the change amount of the prompt version and the node verification risk; The recording module is used to re-encode the selected target evidence node based on the update prompt version and write it into the incremental index, and record the migration service status of the target evidence node. The retrieval module is used to schedule the active index and the incremental index to perform retrieval according to the migration service status, merge the corresponding retrieval results after score calibration, generate or verify candidate questions and answers based on the merged results, and update the version pointer of the active index when the merged results meet the index switching conditions. The output module is used to restore the corresponding evidence node to be served by the active index according to the migration service status when a write or switch failure occurs, and to iteratively execute the above steps until the preset convergence condition is met before outputting the question-answer pair dataset.

Images