Optimization method of retrieval enhancement generation model

Abstract

The invention belongs to the technical field of computers, and provides an optimization method for a retrieval enhancement generation model, and the method comprises the steps: based on an input query request, retrieving a plurality of recall knowledge fragments related to the query request from a pre-constructed knowledge base; performing multi-dimensional quality labeling on the plurality of retrieved recall knowledge fragments to obtain respective quality labeling results of the plurality of recall knowledge fragments; the query request and the recall knowledge fragments are input into a to-be-optimized initial retrieval enhancement generation model, a plurality of initial copywriting are generated, and each initial copywriting comprises at least one recall knowledge fragment; based on the respective quality labeling results of the plurality of recall knowledge fragments and the plurality of initial copywritings, determining respective corresponding generation quality feedback results of the plurality of initial copywritings; and optimizing the initial retrieval enhancement generation model according to the generation quality feedback results corresponding to the plurality of initial copywritings to obtain a target retrieval enhancement generation model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer, more particularly, relates to an optimization method of a retrieval augmented generation model. BACKGROUND

[0002] With the rapid development of natural language processing technology, retrieval augmented generation (RAG) models are widely used in customer service question answering, content creation, industry consulting, and other fields due to their strong semantic understanding and text generation capabilities.

[0003] However, in related technologies, the knowledge base data of RAG models comes from various sources, and the quality of recalled knowledge fragments is uneven, with low relevance, factual errors, or fragment conflicts, which can easily generate scripts containing redundancies and errors. Moreover, RAG models rely on supervised learning for optimization, which has high labeling costs, low optimization efficiency, and poor optimization quality, resulting in poor quality of generated scripts by optimized RAG models. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide an optimization method of a retrieval augmented generation model, aiming to solve the technical problem of inefficient and poor quality of model optimization of related retrieval augmented generation models.

[0005] To achieve the above-mentioned purpose, according to the first aspect of the present application, an optimization method of a retrieval augmented generation model is provided, which comprises:

[0006] retrieving a plurality of recalled knowledge fragments related to the query request from a pre-constructed knowledge base based on the input query request;

[0007] performing quality labeling of the plurality of recalled knowledge fragments in multiple dimensions respectively to obtain quality labeling results of the plurality of recalled knowledge fragments respectively;

[0008] inputting the query request and the plurality of recalled knowledge fragments into an initial retrieval augmented generation model to be optimized to generate a plurality of initial scripts, wherein each initial script contains at least one recalled knowledge fragment;

[0009] determining a generation quality feedback result corresponding to each of the plurality of initial scripts based on the quality labeling results of the plurality of recalled knowledge fragments and the plurality of initial scripts, wherein the generation quality feedback result is used to quantitatively evaluate the generation quality of the initial script;

[0010] optimizing the initial retrieval augmented generation model according to the generation quality feedback result corresponding to each of the plurality of initial scripts to obtain a target retrieval augmented generation model.

[0011] In the above optimization method, a plurality of recall knowledge segments related to the query request are retrieved from a pre-constructed knowledge base based on the input query request; the plurality of retrieved recall knowledge segments are respectively marked in multiple dimensions to obtain respective quality marking results of the plurality of recall knowledge segments. And by inputting the query request and the plurality of recall knowledge segments into the initial retrieval enhancement generation model to be optimized, a plurality of initial scripts are generated; based on the respective quality marking results of the plurality of recall knowledge segments and the plurality of initial scripts, respective generation quality feedback results of the plurality of initial scripts are determined to accurately quantify the generation quality of the initial scripts and present the advantages and disadvantages of the initial retrieval enhancement generation model in the retrieval of the recall knowledge segments. Further, according to the respective generation quality feedback results of the plurality of initial scripts, the initial retrieval enhancement generation model is optimized in a targeted manner to improve the model optimization efficiency and optimization quality, so as to improve the script generation quality of the target retrieval enhancement generation model.

[0012] In a possible implementation manner, the plurality of dimensions include: a relevance dimension of the query request and the recall knowledge segment, a factual accuracy dimension of the recall knowledge segment, and a conflict dimension between different recall knowledge segments; and the marking of the plurality of retrieved recall knowledge segments in multiple dimensions to obtain respective quality marking results of the plurality of recall knowledge segments includes:

[0013] For the relevance dimension, the semantic similarity of each of the recall knowledge segments and the query request is scored to obtain respective first marking results of the plurality of recall knowledge segments;

[0014] For the factual accuracy dimension, the content authenticity of each of the recall knowledge segments is binary marked to obtain respective second marking results of the plurality of recall knowledge segments;

[0015] For the conflict dimension, whether each of the recall knowledge segments conflicts with other recall knowledge segments is judged to obtain respective third marking results of the plurality of recall knowledge segments.

[0016] In the above optimization method, mathematical operations are performed on the respective first feedback values and second feedback values of the plurality of initial scripts, and each initial script can obtain a unique generation quality feedback result. This result accurately quantifies the comprehensive quality of the script, and can be directly used as a feedback signal for parameter optimization of the initial retrieval enhancement generation model in subsequent reinforcement learning.

[0017] In a possible implementation manner, the determination of the respective generation quality feedback results of the plurality of initial scripts based on the respective quality marking results of the plurality of recall knowledge segments and the plurality of initial scripts includes:

[0018] determine a first feedback value corresponding to each of the initial scripts based on the quality annotation results of the plurality of recalled knowledge segments and the plurality of initial scripts, wherein the first feedback value comprises at least one of a relevance feedback value, a conflict resolution feedback value, a redundancy noise feedback value, and a fact error feedback value;

[0019] determine a second feedback value corresponding to each of the initial scripts, wherein the second feedback value comprises at least one of a fluency feedback value, a word count feedback value, and a language consistency feedback value;

[0020] perform mathematical operations on the first feedback value and the second feedback value corresponding to each of the initial scripts to obtain a generation quality feedback result corresponding to each of the initial scripts, wherein the mathematical operations comprise value addition, average taking, and normalization operation.

[0021] In the above optimization method, the quality evaluation of the initial script is converted into a quantifiable feedback signal, which provides an accurate basis for subsequent model training and realizes the deep correlation between the quality of the knowledge segment and the performance of the script generation.

[0022] In a possible implementation manner, the first feedback value comprises a relevance feedback value, the quality annotation result comprises a first annotation result for the relevance dimension, and the first annotation result at least comprises a semantic similarity score. The determination of the first feedback value corresponding to each of the initial scripts based on the quality annotation results of the plurality of recalled knowledge segments and the plurality of initial scripts comprises:

[0023] determine a first relevant segment in the plurality of recalled knowledge segments with a semantic similarity score higher than a first threshold according to the first annotation result corresponding to each of the plurality of recalled knowledge segments;

[0024] respectively calculate semantic knowledge overlap degrees between the plurality of initial scripts and the first relevant segment;

[0025] determine a relevance feedback value corresponding to a first initial script in the plurality of initial scripts according to a comparison result of the semantic knowledge overlap degree corresponding to the first initial script and a preset overlap degree.

[0026] In the above optimization method, the relevance annotation result of the recalled knowledge segment is associated with the knowledge reference of the initial script to accurately evaluate the utilization effect of the initial script on the high-relevance knowledge, thereby providing a basis for the subsequent generation quality feedback result.

[0027] In a possible implementation, the first feedback value includes a conflict resolution feedback value, the quality annotation result includes a second annotation result for a fact accuracy dimension and a third annotation result for a conflict dimension, and the determining of the first feedback value corresponding to each of the initial scripts based on the quality annotation result of each of the recall knowledge segments and the initial scripts includes:

[0028] determining whether a target recall knowledge segment with the second annotation result of content truthfulness exists in a third initial script of the initial scripts and a conflict exists between the target recall knowledge segment and other recall knowledge segments;

[0029] determining a conflict resolution feedback value corresponding to the third initial script according to whether the target recall knowledge segment in the third initial script is content truthful and a conflict exists between the target recall knowledge segment and other recall knowledge segments.

[0030] In the above optimization method, when the initial script faces a conflict of knowledge segments, the conflict processing capability of the generated script is quantified by associating the relationship among the fact accuracy annotation, the conflict annotation, and the script content reference.

[0031] In a possible implementation, the first feedback value includes a redundancy noise feedback value, the quality annotation result includes a first annotation result for a relevance dimension, and the determining of the first feedback value corresponding to each of the initial scripts based on the quality annotation result of each of the recall knowledge segments and the initial scripts includes:

[0032] determining a second relevance segment with a semantic similarity score lower than a second threshold from the recall knowledge segments according to the first annotation result corresponding to each of the recall knowledge segments;

[0033] determining a first determination result by judging whether at least one of the second relevance segments exists in a fourth initial script of the initial scripts and / or a duplicate recall knowledge segment exists in the fourth initial script;

[0034] determining a redundancy noise feedback value corresponding to the fourth initial script according to the first determination result.

[0035] Based on the above implementation, whether the initial script introduces low-relevance knowledge or contains duplicate information is evaluated, and the information purity and conciseness of the generated script are quantified by associating the relevance annotation result with the script content reference feature.

[0036] In a possible implementation manner, the first feedback value includes a factual error feedback value, the quality labeling result includes a second labeling result for a factual accuracy dimension, and the determining of the first feedback value corresponding to each of the initial scripts based on the quality labeling result of each of the recall knowledge segments and the initial scripts includes the following steps.

[0037] determining whether a fifth initial script in the initial scripts contains a recall knowledge segment with the second labeling result of content error, to obtain a second determination result;

[0038] determining a factual error feedback value corresponding to the fifth initial script according to the second determination result.

[0039] Based on the above implementation manner, it is determined whether the initial script references a knowledge segment with factual error, and the factual rigor of the generated script is quantified by associating the factual accuracy labeling result with the content reference of the script.

[0040] In a possible implementation manner, the second feedback value includes a fluency feedback value, and the determining of the second feedback value corresponding to each of the initial scripts includes the following steps.

[0041] verifying the grammatical correctness, the completeness of expression and the logical coherence of the sentences of the initial scripts respectively, to obtain a fluency verification result corresponding to each of the initial scripts;

[0042] determining a fluency feedback value corresponding to each of the initial scripts according to the fluency verification result corresponding to each of the initial scripts.

[0043] Based on the above implementation manner, the language expression quality of the initial script is evaluated, the fluency of the generated content is converted into a quantifiable fluency feedback value by systematically verifying the grammatical correctness, the completeness of expression and the logical coherence of the initial script.

[0044] In a possible implementation manner, the second feedback value includes a word count feedback value, and the determining of the second feedback value corresponding to each of the initial scripts includes the following steps.

[0045] if an actual word count of a sixth initial script in the initial scripts is greater than a preset script word count, calculating a word count difference between the actual word count of the sixth initial script and the preset script word count;

[0046] multiplying the word count difference corresponding to the sixth initial script by a preset word count feedback coefficient to obtain a word count feedback value corresponding to the sixth initial script.

[0047] Based on the above implementation manner, the word count feedback value can quantitatively evaluate the quality loss of the generated script due to excessive word count.

[0048] In a possible implementation manner, the second feedback value includes a language consistency feedback value, and the determining of the second feedback value corresponding to each of the plurality of initial scripts includes:

[0049] determining whether a seventh initial script in the plurality of initial scripts has a mixed situation of different national languages, to obtain a third determination result;

[0050] determining, according to the third determination result, a language consistency feedback value corresponding to the seventh initial script.

[0051] Based on the above implementation manner, the quality performance of the generated script in terms of language consistency can be quantitatively evaluated.

[0052] In a possible implementation manner, the optimizing of the initial retrieval-enhanced generation model according to the generation quality feedback result corresponding to each of the plurality of initial scripts to obtain a target retrieval-enhanced generation model includes:

[0053] constructing a plurality of training samples based on the plurality of initial scripts and the generation quality feedback result corresponding to each of the plurality of initial scripts;

[0054] dividing the plurality of training samples into a training data set and a verification data set;

[0055] inputting the training data set into the initial retrieval-enhanced generation model, updating each parameter in the initial retrieval-enhanced generation model to minimize a preset loss function, to obtain a model after parameter updating, wherein the preset loss function includes a generalized relative policy optimization (GRPO) loss function;

[0056] inputting the verification data set into the model after parameter updating, and detecting whether the model after parameter updating has converged;

[0057] in a case where the model after parameter updating has converged, determining that the model after parameter updating has been trained, and obtaining the target retrieval-enhanced generation model.

[0058] In the above optimization method, when receiving a new query request and recalling a knowledge fragment, the target retrieval-enhanced generation model can preferentially refer to a knowledge fragment with high relevance and high accuracy based on the rules learned in the training process, avoid redundancy, noise, and factual errors, and generate a high-quality script with high fluency, compliance with the word requirement, and language consistency, thereby significantly improving the overall quality of retrieval-enhanced generation. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0060] Figure 1 is a flowchart of an optimization method for retrieving an enhanced generation model provided by an embodiment of the present application;

[0061] Figure 2 is a flowchart of an optional optimization method for retrieving an enhanced generation model provided by an embodiment of the present application;

[0062] Figure 3 is a flowchart of an optional optimization method for retrieving an enhanced generation model provided by an embodiment of the present application;

[0063] Figure 4 is a flowchart of an optional optimization method for retrieving an enhanced generation model provided by an embodiment of the present application;

[0064] Figure 5 is a flowchart of an optional optimization method for retrieving an enhanced generation model provided by an embodiment of the present application;

[0065] Figure 6 is a structural schematic diagram of an optimization device for retrieving an enhanced generation model provided by an embodiment of the present application;

[0066] Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0067] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can be practiced without these specific details. In other instances, well-known systems, devices, circuits, and methods have not been described in detail so as not to obscure the description of the present application.

[0068] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0069] It should also be understood that, in the description of the application, unless otherwise specified, the use of the term "or" in the description and the claims of the application means a "and / or" relationship, for example, A or B can mean A or B or A and B. In this application, "and / or" is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which means that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. And in the description of the application, unless otherwise specified, "multiple" means two or more. "At least one of the following" or similar expressions means any combination of the items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0070] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the application, in the embodiments of the application, "first", "second" and the like are used to distinguish the same items or similar items with basically the same function and role. Those skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and are only used for distinction, and "first", "second" and the like do not necessarily mean different, and cannot be understood as indicating or implying relative importance.

[0071] As used in the specification and the appended claims of the application, the term "if" can be interpreted as "when" or "upon" or "in response to a determination" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]" depending on the context.

[0072] In the description of the application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in different parts of the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0073] The examples of the application provide an example of an optimization method for retrieving an enhanced generation model. Please refer to Figure 1 Figure 1 The application provides an example of an optimization method for retrieving an enhanced generation model. As an example but not limitation, the method can be applied to or run in an electronic device. The method includes:

[0074] S101, based on the input query request, retrieve a plurality of recall knowledge segments related to the query request from a pre-constructed knowledge base.

[0075] S102, respectively label the quality of the plurality of retrieved recall knowledge segments in multiple dimensions to obtain the quality labeling results of each of the plurality of recall knowledge segments.

[0076] S103, input the query request and the plurality of recall knowledge segments into an initial retrieval enhanced generation model to be optimized to generate a plurality of initial scripts, wherein each initial script contains at least one recall knowledge segment.

[0077] S104, based on the quality labeling results of each of the plurality of recall knowledge segments and the plurality of initial scripts, determine the generation quality feedback results corresponding to each of the plurality of initial scripts, wherein the generation quality feedback results are used to quantitatively evaluate the generation quality of the initial scripts.

[0078] S105, according to the generation quality feedback results corresponding to each of the plurality of initial scripts, optimize the initial retrieval enhanced generation model to obtain a target retrieval enhanced generation model.

[0079] It should be understood that the optimization method for retrieving an enhanced generation model provided by the embodiments of the application can be applied to, but not limited to, media retrieval, financial question answering, medical assistance and other generation scenarios that rely on external knowledge support to improve the quality of the output content of the retrieval enhanced generation model.

[0080] In some embodiments, the pre-constructed knowledge base can be built according to specific application scenarios, for example, in the media retrieval scenario, the knowledge base can include media basic attribute data (such as role setting, creative team information, content classification label, etc.), verified domain knowledge data (such as media industry specifications, classic work analysis, etc.), and legally acquired public information data (such as media-related comments, interpretation content, etc.).

[0081] ​In some embodiments, when receiving a query request input by a user, a semantic retrieval technique (such as vector similarity matching, keyword accurate matching, etc.) is used to retrieve the knowledge base, and knowledge fragments semantically associated with the query request are screened out. It should be noted that the retrieval result contains at least two or more recall knowledge fragments related to the query request. For example, taking the above query request “What are the works of a certain type in which actor A participates” as an example, three recall knowledge fragments are obtained, i.e., “Actor A plays a certain role in work M, and work M belongs to a certain type”, “Work N created by actor A is of a certain type and contains a certain plot”, and “Actor A does not participate in work P, and work P is of another type”.

[0082] In some embodiments, the plurality of dimensions includes at least the relevance of the query request to the knowledge fragment, the factual accuracy of the knowledge fragment, and the conflict between different knowledge fragments. Each recall knowledge fragment needs to be separately labeled with the above dimensions to ensure the independence and pertinence of the quality labeling results of each recall knowledge fragment.

[0083] In some embodiments, in the labeling process, the relevance dimension is used to judge the semantic fit degree of the knowledge fragment and the query request, which can be judged by professional labeling personnel in combination with the query intent, or a mature semantic matching model (such as a similarity calculation tool based on a pre-trained language model) is used to output a quantitative labeling result; the factual accuracy dimension is used to verify the authenticity of the content of the knowledge fragment, which can be completed by comparing authoritative data sources (such as officially published media information and certified field databases); the conflict dimension is used to identify whether there is a factual contradiction between multiple recall knowledge fragments (such as fragment 1 stating that “actor A participates in work M”, and fragment 2 stating that “actor A does not participate in work M”), which needs to be compared for each two factual points of the recall knowledge fragments, and finally the labeling result of whether there is a conflict and the conflict object corresponding to each recall knowledge fragment is formed.

[0084] As an example but not limited to, for example, for the above three retrieved recall knowledge fragments, the labeling can obtain the quality labeling results of fragment 1: high relevance, factual accuracy, and no conflict with other fragments, fragment 2: high relevance, factual accuracy, and no conflict with other fragments, and fragment 3: low relevance, factual accuracy, and no conflict with other fragments.

[0085] It should be understood that in the embodiments of the present application, the initial retrieval enhancement generation model to be optimized can be an existing open source retrieval enhancement generation model or be obtained based on the framework of the existing open source retrieval enhancement generation model. The query request and the recalled knowledge fragments are combined into a prompt and input into the initial retrieval enhancement generation model to be optimized. Then, the initial retrieval enhancement generation model simultaneously processes the query request and the plurality of recalled knowledge fragments to generate a plurality of initial scripts (such as 3-5 initial scripts), and each initial script includes one or more recalled knowledge fragments, that is, the content of each initial script quotes or integrates at least one recalled knowledge fragment, and different initial scripts can include the same or different recalled knowledge fragments.

[0086] For example, for the foregoing query request and three recalled knowledge fragments, the initial script 1 generated by the initial retrieval enhancement generation model can include the fragment 1 and the fragment 2 (the content is: the certain type of works of the actor A includes the work M and the work N, wherein the actor A plays a certain role in the work M, and the work N includes a certain plot), the initial script 2 can include the fragment 1 (the content is: the certain type of works of the actor A includes the work M, and the actor A plays a certain role in the work), and the initial script 3 can include the fragment 3 (the content is: the actor A does not participate in the work P, and the work does not belong to the certain type).

[0087] It should be understood that if an initial script can effectively use high-quality knowledge fragments (such as high-relevance and high-accuracy fragments) and avoid low-quality knowledge fragments (such as low-relevance and conflicting fragments), the generation quality feedback result is better; otherwise, the corresponding generation quality feedback result is poor.

[0088] In some embodiments, each initial script is taken as an independent evaluation object, and a quality label result of the recalled knowledge fragment corresponding to the initial script is combined to design a quantification rule from the aspects of whether the fact is accurately quoted, whether the query request is fitted, and whether the knowledge is effectively integrated. Then, each initial script obtains a unique and quantifiable generation quality feedback result as the basis for model training. For example, the initial script 1 quotes the fragment 1 and the fragment 2 (both are high-relevance and high-accuracy fragments), and the content fits the query request of “the certain type of works of the actor A”, and there is no factual error, so the generation quality feedback result of the initial script 1 can be set to a high score (such as 8 points, and the full score is 10 points); the initial script 2 only quotes the fragment 1 (a high-quality fragment), the content is accurate but the information coverage is incomplete, and the feedback result is set to 6 points; and the initial script 3 quotes the fragment 3 (a low-relevance fragment), the content is accurate but does not effectively respond to the query demand, and the feedback result is set to 3 points.

[0089] In some embodiments, the training process uses the generation quality feedback results corresponding to the plurality of initial texts as feedback signals, and adjusts the model parameters of the initial retrieval and enhancement generation model using a reinforcement learning algorithm (such as a policy gradient algorithm) to enable the initial retrieval and enhancement generation model to learn a generation strategy for generating texts with higher feedback results, thereby reducing the probability of the optimized target retrieval and enhancement generation model outputting low-quality texts.

[0090] For example, in the training process, the plurality of initial texts and their corresponding generation quality feedback results can be used as training samples to calculate the strategy loss of the initial retrieval and enhancement generation model when generating each initial text (i.e., the matching degree between the probability of the initial retrieval and enhancement generation model generating the initial text and the feedback result); then the model parameters are optimized through backpropagation to make the model more inclined to select high-quality knowledge fragments and generate content that meets the query requirements in subsequent generation processes; the above training process is iterated for a preset number of times (such as 1000 times) or until the average generation quality feedback result of the texts generated by the model reaches a preset threshold (such as an average score of 8 or above), and the training is stopped to obtain the optimized target retrieval and enhancement generation model. The target retrieval and enhancement generation model can directly output high-quality generated content when receiving subsequent query requests and recalling knowledge fragments, without the need for sampling and evaluation of initial texts.

[0091] In one possible implementation, the plurality of dimensions include: a relevance dimension of the query request and the recalled knowledge fragments, a factual accuracy dimension of the recalled knowledge fragments, and a conflict dimension between different recalled knowledge fragments; the plurality of recalled knowledge fragments are respectively marked in quality in the plurality of dimensions to obtain respective quality marking results of the plurality of recalled knowledge fragments, including:

[0092] For the relevance dimension, the semantic similarity between each recalled knowledge fragment and the query request is scored to obtain respective first marking results corresponding to the plurality of recalled knowledge fragments.

[0093] For the factual accuracy dimension, the content authenticity of each recalled knowledge fragment is binary marked to obtain respective second marking results corresponding to the plurality of recalled knowledge fragments.

[0094] For the conflict dimension, whether each recalled knowledge fragment conflicts with other recalled knowledge fragments is judged to obtain respective third marking results corresponding to the plurality of recalled knowledge fragments.

[0095] In some embodiments, a pre-trained language model (such as a large language model based on a Transformer architecture) can be used to calculate the semantic similarity between each recalled knowledge snippet and the query request: the query request text and the text of a single recalled knowledge snippet are respectively input into the pre-trained language model to obtain a first semantic vector of the query request text and a second semantic vector of the single recalled knowledge snippet; then the similarity value between the first semantic vector and the second semantic vector is calculated through a vector space distance (such as cosine similarity, Euclidean distance) calculation formula, and the similarity value is mapped to a set scoring interval (for example, 0-1 points, the higher the score, the stronger the relevance).

[0096] By way of example and not limitation, taking the query request “What are some works of a certain type that actor A has participated in” as an example, the recalled knowledge snippet corresponding to the query request is “Actor A plays a certain role in work M, and work M belongs to a certain type”. After calculation by the pre-trained language model, if the cosine similarity of the semantic vectors of the two is 0.85, then the first annotation result of the semantic similarity of the recalled knowledge snippet with respect to the query request is 0.85 points; if the cosine similarity of the semantic vectors of another recalled knowledge snippet “Actor A has not participated in work P, and work P is of another type” with respect to the query request is 0.2, then the first annotation result of the recalled knowledge snippet is 0.2 points. By performing the above operation on each recalled knowledge snippet individually, the first annotation result corresponding to each recalled knowledge snippet is ultimately obtained.

[0097] In another embodiment, content authenticity refers to the degree of fit between the information contained in the recalled knowledge snippet and the objective fact. In the embodiment of the present application, the annotation result is in binary form, i.e., only “real (marked as 1)” or “not real (marked as 0)” needs to be annotated. However, other forms can also be used to achieve the annotation effect of annotating whether it is real or not. Specifically, in implementation, the annotation of each recalled knowledge snippet can be achieved by comparison with an authoritative data source, but is not limited thereto: an authoritative data source (such as an official media database, a certified industry information platform, etc.) is constructed in advance, and the core information (such as “Actor A participates in work M”) of a single recalled knowledge snippet is extracted and compared with the corresponding information in the authoritative data source one by one; if the core information exists and is consistent in the authoritative data source, then the second annotation result of the recalled knowledge snippet with respect to content authenticity is “1”, and if the core information does not exist or is contradictory in the authoritative data source, then the second annotation result of the recalled knowledge snippet with respect to content authenticity is “0”.

[0098] In some embodiments, taking the recall knowledge piece "Actor A plays a certain role in Work M, which belongs to a certain type" as an example, if it is confirmed through comparison with the official media database that Actor A indeed participated in Work M and Work M belongs to a certain type, then the second labeling result of the recall knowledge piece in the "fact accuracy dimension" is 1; if another recall knowledge piece "Work N created by Actor A is of a certain type and contains a certain plot" is compared, there is no record of the association between Work N and Actor A in the authoritative data source, then the second labeling result of the other recall knowledge piece is 0. By repeating the above comparison and labeling process for each recall knowledge piece, the second labeling result corresponding to each of the recall knowledge pieces can be obtained.

[0099] In another embodiment, the third labeling result needs to reflect the conflict situation of the recall knowledge piece with other pieces, which can be in the form of conflict markers and conflict object identifiers: if there is a conflict between a certain recall knowledge piece and at least one other recall knowledge piece, the third labeling result is recorded as "there is a conflict, the conflict object is [specific recall knowledge piece number / identifier]"; if there is no conflict between a certain recall knowledge piece and all other recall knowledge pieces, the third labeling result is recorded as "there is no conflict".

[0100] It should be understood that a conflict refers to a contradiction between different recall knowledge pieces in describing the same fact. In specific implementation, for each recall knowledge piece, a fact comparison is performed with all other recall knowledge pieces one by one: first, the fact of a single recall knowledge piece is extracted (such as Actor A participating in Work M), then the fact of another recall knowledge piece is extracted (such as Actor A not participating in Work M), if the two facts cannot exist at the same time, it is determined that the third labeling result of the recall knowledge piece is that there is a conflict; if there is no such contradiction between all recall knowledge pieces, it is determined that the third labeling result of the recall knowledge piece is that there is no conflict.

[0101] Taking three recall knowledge pieces as examples, namely, piece 1 (Actor A participates in Work M), piece 2 (Actor A does not participate in Work M), and piece 3 (Actor A participates in Work N), when judging the conflict of piece 1, it is found that there is a contradiction in fact when compared with piece 2, therefore, the third labeling result of piece 1 is "there is a conflict, the conflict object is piece 2"; when judging piece 2, it is found that there is a contradiction in fact when compared with piece 1, therefore, the third labeling result of piece 2 is "there is a conflict, the conflict object is piece 1"; when judging piece 3, there is no contradiction in fact when compared with piece 1 and piece 2, therefore, the third labeling result of piece 3 is "there is no conflict". By completing the conflict judgment of each recall knowledge piece with other recall knowledge pieces, the third labeling result corresponding to each of the recall knowledge pieces can be finally obtained.

[0102] In a possible implementation manner, the generation quality feedback result corresponding to each of the plurality of initial scripts is determined based on the quality annotation result of each of the plurality of recall knowledge segments and the plurality of initial scripts, and the determining includes:

[0103] In S201, the first feedback value corresponding to each of the plurality of initial scripts is determined based on the quality annotation result of each of the plurality of recall knowledge segments and the plurality of initial scripts.

[0104] The first feedback value includes at least one of a relevance feedback value, a conflict resolution feedback value, a redundancy noise feedback value, and a fact error feedback value.

[0105] In some embodiments, the first feedback value is used to reflect the utilization quality of the initial script on the recall knowledge segment, and is specifically calculated in combination with the quality annotation result (the first annotation result "relevance", the second annotation result "fact accuracy", and the third annotation result "conflict") of the recall knowledge segment and the content reference of the initial script.

[0106] In some embodiments, the relevance feedback value is used to evaluate whether the initial script preferentially references the recall knowledge segment with high relevance. In specific implementation, the first annotation result (that is, the semantic similarity score) can be extracted from the quality annotation result of the recall knowledge segment, and a first threshold value (that is, a high relevance threshold value, such as 0.6 points, which can be adjusted according to the scene) is obtained, and the target recall knowledge segment with high relevance whose first annotation result is greater than or equal to the first threshold value is screened out, and the relevance feedback value of the target recall knowledge segment is set to 1 (positive feedback).

[0107] In another embodiment, after the target recall knowledge segment with high relevance whose first annotation result is greater than or equal to the first threshold value is selected, the content of the initial script can be analyzed to count the reference proportion of the information of the high-relevance segment referenced by the initial script. If the reference proportion is greater than or equal to 80%, the relevance feedback value of the target recall knowledge segment is set to 1 (positive feedback). If the reference proportion is between 50% and 80%, the relevance feedback value of the target recall knowledge segment is set to 0.5. If the reference proportion is less than 50% (that is, a large number of low-relevance segments are referenced), the relevance feedback value of the target recall knowledge segment is set to 0 or -1 (negative feedback).

[0108] In some embodiments, the conflict resolution feedback value is used to evaluate whether the initial script selects factually accurate information when facing fragment conflicts. For example, the second annotation result (factually accurate binary annotation) and the third annotation result (conflict judgment) of the recalled knowledge fragments can be extracted first, and the fragment groups with conflicts are located (for example, fragment X and fragment Y conflict, and the second annotation result of fragment X is 1 "true", and the second annotation result of fragment Y is 0 "not true"); then it is checked whether the initial script references the information of the conflict group. If the script only references the fragment information with the second annotation result of 1, the conflict resolution feedback value is set to 1; if the script references the information of both conflicts, the value is set to -0.5; if the script only references the fragment information with the second annotation result of 0, the value is set to -1.

[0109] For example, the conflict fragment group is "fragment X (actor A participates in work M, second annotation result 1)" and "fragment Y (actor A does not participate in work M, second annotation result 0)", and the initial script 3 only mentions "actor A participates in work M", so the conflict resolution feedback value of the initial script 3 is 1; the initial script 4 simultaneously mentions "actor A participates in and does not participate in work M", so the value is -0.5.

[0110] In some embodiments, the redundancy noise feedback value is used to evaluate whether the initial script has the problem of referencing low correlation fragments or repeated expressions. For example, the first annotation result of the recalled knowledge fragments can be combined to count the number of times the script references the first annotation result <0.3 (low correlation threshold) fragments, and the number of repeated information segments in the initial script. If there is no low correlation fragment and no repeated expression, the redundancy noise feedback value is set to 0 (no penalty); for each low correlation fragment or each repeated expression, the redundancy noise feedback value is reduced by 0.2, and the minimum value is -1. For example, the initial script 5 references a low correlation fragment and has one repetition, so the redundancy noise feedback value of the initial script 5 is -0.4; the initial script 6 references three low correlation fragments and has two repetitions, so the redundancy noise feedback value of the initial script 6 is -1.

[0111] In some embodiments, the fact error feedback value is used to evaluate whether the initial script references factually inaccurate fragment information. For example, the second annotation result of the recalled knowledge fragments is extracted, and it is checked whether each initial script contains fragment information with the second annotation result of 0 (not true). If an initial script does not contain fragment information with the second annotation result of 0 (not true), the fact error feedback value corresponding to the initial script is 0 (no penalty); for each error information in an initial script, the fact error feedback value of the initial script is reduced by 0.5, and the minimum value of the fact error feedback value is -1. For example, the initial script 7 contains one error fragment information, so the fact error feedback value of the initial script 7 is -0.5; the initial script 8 contains three error information, so the fact error feedback value of the initial script 7 is -1.

[0112] S202, determine a second feedback value corresponding to each of the plurality of initial scripts.

[0113] The second feedback value includes at least one of a fluency feedback value, a word count feedback value, and a language consistency feedback value.

[0114] It should be understood that the above-mentioned second feedback value focuses on the generation quality of the initial script itself, does not depend on the annotation result of the recalled knowledge fragment, and is directly evaluated based on the text features of the initial script.

[0115] In some embodiments, the fluency feedback value is used to evaluate the sentence smoothness and logical coherence of the initial script. For example, the initial script is input into a pre-trained text fluency evaluation model (such as a fluency scoring model based on a GPT series model fine-tuning), and the text fluency evaluation model outputs a fluency score of 0-1 (1 being the most fluent), which is the fluency feedback value. For example, the initial script 9 is smooth and logically coherent, and the text fluency evaluation model outputs a fluency score of 0.9, i.e., the fluency feedback value is 0.9; the initial script 10 has syntax errors and logical breaks, and the text fluency evaluation model outputs a fluency score of 0.3, i.e., the fluency feedback value is 0.3.

[0116] In some embodiments, the word count feedback value is used to evaluate whether the length of the initial script meets the preset requirement. For example, a scenario-based word count range is first set (such as 100-300 words for a media retrieval scenario), the actual word count of the initial script is counted, and if the actual word count is within the above-mentioned scenario-based word count range, the word count feedback value of the corresponding initial script is 0 (no penalty); if the actual word count is lower than the lower limit of the scenario-based word count range, the word count feedback value is reduced by 0.1 for every 10 words; if the actual word count is higher than the upper limit of the scenario-based word count range, the word count feedback value is reduced by 0.1 for every 10 words, and the minimum value of the word count feedback value is -0.5.

[0117] In some embodiments, the language consistency feedback value is used to evaluate whether the initial script has a problem of mixed languages. For example, a language detection tool can be used to scan the script text to identify whether it contains other language characters in addition to the target language (such as Chinese). If the initial script does not contain other language characters in addition to Chinese, the language consistency feedback value of the initial script is 0 (no penalty); if the initial script contains other language characters in addition to Chinese, according to the language mixing ratio, the language consistency feedback value of the initial script is determined to be -0.3 (a small amount of mixing) or -0.5 (a large amount of mixing).

[0118] S203, performing mathematical operations on the first feedback value and the second feedback value corresponding to each of the plurality of initial scripts to obtain a generation quality feedback result corresponding to each of the plurality of initial scripts.

[0119] The mathematical operations include value addition, average value taking, and normalization operation.

[0120] In some embodiments, the total number of the first feedback value and the second feedback value can be calculated first, and the feedback average value of each initial script can be obtained by directly adding all the dimension values in the first feedback value and the second feedback value and dividing the sum by the total number (the result range is consistent with the single dimension value range, such as -1 to 1).

[0121] The feedback average value of each initial script is then uniformly mapped to a fixed range (such as 0-1) as the generation quality feedback result, so as to stabilize the gradient of subsequent model training. For example, but not limited to, the Min-Max normalization formula can be used for normalization processing.

[0122] According to the above embodiments, the mathematical operations are performed on the first feedback value and the second feedback value corresponding to each of the plurality of initial scripts, and each initial script can obtain a unique generation quality feedback result. The result accurately quantifies the comprehensive quality of the script, which can be directly used as a feedback signal for reinforcement learning for parameter optimization of the initial retrieval enhanced generation model.

[0123] In a possible implementation manner, the first feedback value includes a relevance feedback value, and the quality annotation result includes a first annotation result for the relevance dimension, and the first annotation result at least includes a semantic similarity score. Figure 3 As shown, based on the quality annotation result of each of the plurality of recall knowledge segments and the plurality of initial scripts, the first feedback value corresponding to each of the plurality of initial scripts is determined, including:

[0124] S301, determining a first relevance segment with a semantic similarity score higher than a first threshold from the plurality of recall knowledge segments according to the first annotation result corresponding to each of the plurality of recall knowledge segments.

[0125] S302, calculating the semantic knowledge overlap between each of the plurality of initial scripts and the first relevance segment.

[0126] S303, determining the relevance feedback value corresponding to the first initial script in the plurality of initial scripts according to the comparison result of the semantic knowledge overlap corresponding to the first initial script and a preset overlap.

[0127] In some embodiments, the first annotation result for the relevance dimension in the quality annotation result of the recalled knowledge segments (which at least includes the semantic similarity score of each recalled knowledge segment with the query request, for example, a 0-1 interval value calculated by using the cosine similarity algorithm) is used to filter out the first relevance segments with a semantic similarity score higher than a first threshold from all the recalled knowledge segments.

[0128] Specifically, a first threshold for distinguishing high / low relevance is preset (which can be adjusted according to the accuracy requirements of the application scenario, for example, in the media retrieval scenario, considering the user's requirement for the accuracy of the query result, the first threshold can be set to 0.6), and then the semantic similarity score of each recalled knowledge segment is compared with the first threshold, and the recalled knowledge segment with a score higher than the first threshold is marked as a first relevance segment, which has a higher semantic fit with the user's query request and is a knowledge segment that should be preferentially quoted by the initial script. For example, when the user's query request is "What are some works of a certain type that actor A has participated in?", if there are 5 recalled knowledge segments, 3 of which have semantic similarity scores of 0.8, 0.75, and 0.65 (all higher than the first threshold of 0.6), then these 3 recalled knowledge segments are the first relevance segments, and the remaining 2 recalled knowledge segments with scores lower than 0.6 are low relevance segments and are not preferentially quoted.

[0129] After the selection of the first relevance segments is completed, the semantic knowledge overlap between each initial script and these first relevance segments is further analyzed to determine whether the initial script effectively quotes the high-relevance knowledge segments. It should be understood that the semantic knowledge overlap focuses on the overlap of knowledge content rather than simple text character repetition, and can be implemented by a pre-trained text matching model (such as a paragraph matching model based on BERT architecture fine-tuning): first, the knowledge points of the first relevance segments (such as "Actor A participates in work M, and work M belongs to a certain type" and "Actor A plays the leading role in work N, and work N is of a certain type") are extracted as structured knowledge units, and then the text content of the initial script is input into the text matching model, which compares the semantic association degree between the initial script and each first relevance segment knowledge unit sentence by sentence, and finally outputs a comprehensive semantic knowledge overlap value, for example, also in the 0-1 interval, the higher the value, the higher the overlap, that is, the more high-relevance knowledge quoted by the initial script.

[0130] For example, for the above 3 first correlation fragments, the content of the initial script 1 is "Actor A participated in certain type of works M and works N, among which, works M is his classic representative work", and the text matching model calculates that the initial script 1 has a high degree of coincidence with the knowledge units of the 3 first correlation fragments, and the semantic knowledge overlap degree is 0.9; while the content of the initial script 2 is "Actor A has been active in the film and television industry in recent years, and has participated in the creation of many works", the initial script 2 has weak knowledge association with the first correlation fragment, and the semantic knowledge overlap degree is only 0.3.

[0131] Finally, according to the comparison result of the semantic knowledge overlap degree corresponding to each initial script (taking the first initial script as an example) and the preset overlap degree, the correlation feedback value corresponding to the first initial script is determined. The preset overlap degree here needs to be set in combination with the number of first correlation fragments and the knowledge coverage range, for example, in the above case including 3 first correlation fragments, considering that the initial script needs to cover high correlation knowledge as much as possible, the preset overlap degree can be set to 0.7. If the semantic knowledge overlap degree of the first initial script is higher than the preset overlap degree (such as 0.9 of the initial script 1 described above > 0.7), it indicates that the first initial script effectively quotes the knowledge in the first correlation fragment, and the correlation degree with the query request is high, at this time the correlation feedback value of the first initial script is set to a positive value (such as 1, the specific value can be adjusted according to the scene demand, and the positive value represents the affirmation of the script in the correlation dimension); if the semantic knowledge overlap degree of the first initial script is lower than the preset overlap degree (such as 0.3 of the initial script 2 described above < 0.7), it indicates that the first initial script does not make full use of high correlation knowledge, and there is a problem of information deviation from the query request, at this time the correlation feedback value of the first initial script is set to a negative value (such as -0.5, the negative value represents the constraint of the script in the correlation dimension); if the semantic knowledge overlap degree is exactly equal to the preset overlap degree, the correlation feedback value can be set to 0, which represents that the script in the correlation dimension basically meets the standard but still has optimization space.

[0132] In a possible implementation manner, the first feedback value includes a conflict resolution feedback value, the quality marking result includes a second marking result for the fact accuracy dimension and a third marking result for the conflict dimension, and the first feedback value corresponding to each of the plurality of initial scripts is determined based on the quality marking result of each of the plurality of recall knowledge fragments and the plurality of initial scripts, including:

[0133] It is determined whether the target recall knowledge fragment with the second marking result of content true is contained in the third initial script of the plurality of initial scripts, and the target recall knowledge fragment has a conflict with other recall knowledge fragments.

[0134] The conflict resolution feedback value corresponding to the third initial script is determined according to whether the target recall knowledge fragment in the third initial script is content authentic and conflicts with other recall knowledge fragments.

[0135] In some embodiments, based on the quality annotation results of the recall knowledge fragments, a group of knowledge fragments with conflicts and distinguishable factual accuracy is located. The second annotation result for the factual accuracy dimension adopts a binary form (such as “1” representing content authenticity and “0” representing content inauthenticity), and the third annotation result for the conflict dimension is used to identify whether there is a factual contradiction between fragments (such as “conflict exists” or “conflict does not exist” and specific conflict objects). Then, the target recall knowledge fragment can be screened from all the recall knowledge fragments. The target recall knowledge fragment needs to meet two conditions: one is that the second annotation result of the target recall knowledge fragment is “1” (i.e., the content is authentic); and the other is that the third annotation result shows that the target recall knowledge fragment conflicts with at least one other recall knowledge fragment (i.e., there is a contradiction between the expression of another fragment on the same fact and the target recall knowledge fragment, and the second annotation result of the contradictory fragment is “0” or “1”).

[0136] For example, in a media retrieval scenario, the recall knowledge fragments include fragment X (“Actor A stars in work M”, second annotation result 1, third annotation result “conflicts with fragment Y”) and fragment Y (“Actor A does not star in work M”, second annotation result 0, third annotation result “conflicts with fragment X”), fragment X is content authentic and conflicts with fragment Y, then fragment X is the target recall knowledge fragment.

[0137] Subsequently, for the third initial script in the plurality of initial scripts, it is necessary to analyze whether the content of the third initial script contains the information of the above-mentioned target recall knowledge fragment and does not contain non-authentic information conflicting with the target fragment. Specifically, the semantic analysis of the text content of the third initial script is needed to extract the expression of the conflict fact point (such as whether actor A stars in work M) in the third initial script, and compare it with the target recall knowledge fragment and other fragments conflicting with it: if the third initial script only contains the information of the target recall knowledge fragment (content authentic) and does not mention the information of the non-authentic fragment conflicting with it, it means that the script successfully resolves the knowledge conflict and selects the authentic content; if the third initial script contains the information of the target recall knowledge fragment and the non-authentic fragment conflicting with it (such as mentioning “Actor A stars in work M” and “Actor A does not star in work M” at the same time), it means that the third initial script does not effectively resolve the conflict and there is information contradiction; if the third initial script does not contain the information of the target recall knowledge fragment but contains the information of the non-authentic fragment conflicting with it (such as only mentioning “Actor A does not star in work M”), it means that the third initial script selects the wrong information in the conflict processing.

[0138] Further, the conflict resolution feedback value corresponding to the third initial script is determined according to the analysis result. If the third initial script only contains the true information of the target recall knowledge fragment and avoids the non-true information of the conflict, it indicates that the conflict resolution capability of the third initial script is excellent, and the conflict resolution feedback value can be set as a positive reward value (such as 1, and the specific value can be adjusted according to the scene requirement); if the third initial script contains the information of both parties of the conflict, it shows that the third initial script cannot effectively handle the conflict, and the conflict resolution feedback value of the third initial script can be set as a neutral or slightly punished value (such as 0 or -0.5); if the third initial script selects the non-true information conflicting with the target recall knowledge fragment, it indicates that there is a serious problem in the conflict resolution feedback value, and the conflict resolution feedback value is a strong punishment value (such as -1).

[0139] In a possible implementation manner, the first feedback value includes a redundancy noise feedback value, and the quality annotation result includes a first annotation result for the relevance dimension, such as Figure 4 As shown in FIG. 1, based on the quality annotation results of the plurality of recall knowledge fragments and the plurality of initial scripts, the first feedback value corresponding to each of the plurality of initial scripts is determined, including:

[0140] S401, according to the first annotation result corresponding to each of the plurality of recall knowledge fragments, a second correlation fragment with a semantic similarity score lower than a second threshold is determined from the plurality of recall knowledge fragments.

[0141] S402, it is judged whether a fourth initial script in the plurality of initial scripts contains at least one second correlation fragment and / or there is a repeated recall knowledge fragment, to obtain a first judgment result.

[0142] S403, according to the first judgment result, a redundancy noise feedback value corresponding to the fourth initial script is determined.

[0143] In some embodiments, first, based on the first labeling result (including semantic similarity scores) of the relevance dimension of the recalled knowledge fragments, the knowledge fragments with low relevance are screened out. A second threshold is a preset threshold for defining low relevance, and the second threshold is equal to or less than the first threshold. The second threshold can be set according to the requirement of information accuracy in the scene, for example, in the media retrieval scene, in order to strictly filter irrelevant information, the second threshold can be set to 0.3, and all the fragments with semantic similarity scores lower than the second threshold in the recalled knowledge fragments are marked as second relevance fragments. Such fragments are prone to become redundant noise and interfere with the accuracy of generated content if they are cited by the initial script because of their weak semantic association with the user query request. For example, when the user query request is "an actor A starring in a certain type of movie", if there are fragments such as "a music album list of an actor B" (semantic similarity score 0.2) and "an introduction to the shooting location of a certain TV series" (semantic similarity score 0.15) in the recalled knowledge fragments, since the semantic similarity scores of the recalled knowledge fragments are all lower than the second threshold 0.3, they are all determined as second relevance fragments.

[0144] Next, for the fourth initial script in the plurality of initial scripts, two dimensions are analyzed to determine whether the fourth initial script contains redundant noise: on the one hand, it is determined whether the content of the fourth initial script contains information of at least one second relevance fragment, that is, the semantic association between the fourth initial script and all second relevance fragments is compared through text matching technology. If the fourth initial script contains expressions that coincide with the second relevance fragments (such as mentioning the album sales of actor B in the above case), it is determined that the fourth initial script contains the second relevance fragments. On the other hand, it is checked whether the fourth initial script contains repeated citation of recalled knowledge fragments, that is, it is analyzed whether there are multiple repeated expressions of information about the same recalled knowledge fragment in the fourth initial script (such as mentioning "the role setting of actor A in work M" twice in a row), or repeated citation of the same information in different recalled knowledge fragments (such as simultaneously citing consistent expressions about "the release time of work M" in two fragments).

[0145] Further, if the fourth initial script does not contain any information of the second relevant fragment and there is no repeated reference, it indicates that the fourth initial script has high information purity and no redundancy, and the redundancy noise feedback value of the fourth initial script can be set to 0 (no penalty); if the fourth initial script contains information of one second relevant fragment or there is one repeated reference, it indicates that there is slight redundant noise, and a lower penalty value (such as -0.2) can be set; if the fourth initial script contains information of two or more second relevant fragments or there are two or more repeated references, it indicates that the redundant noise is relatively serious, and the redundancy noise feedback value is increased accordingly (such as -0.5); if the fourth initial script contains a large amount of information of second relevant fragments and there are multiple repeated references, the redundancy noise feedback value of the fourth initial script is set to a higher penalty value (such as -1).

[0146] In a possible implementation manner, the first feedback value includes a fact error feedback value, the quality labeling result includes a second labeling result for a fact accuracy dimension, and the first feedback value corresponding to each of the plurality of initial scripts is determined based on the quality labeling result of each of the plurality of recall knowledge fragments and the plurality of initial scripts, including:

[0147] It is determined whether a fifth initial script in the plurality of initial scripts contains a recall knowledge fragment with a second labeling result of content error, to obtain a second determination result.

[0148] According to the second determination result, a fact error feedback value corresponding to the fifth initial script is determined.

[0149] In some embodiments, based on the second labeling result (in the form of binary labeling, such as "0" indicating content error and "1" indicating content truth) of each recall knowledge fragment for the fact accuracy dimension, the fact error fragment is distinguished from all recall knowledge fragments, that is, the recall knowledge fragment with the second labeling result of "0", which contains information inconsistent with the objective fact, and if it is referenced by the initial script, it will directly cause the generated content to have a factual error. For example, in the education question and answer scenario, the recall knowledge fragment contains "the sum of the internal angles of a triangle is 180 degrees" (second labeling result 1) and "the sum of the internal angles of a triangle is 200 degrees" (second labeling result 0), and the latter is the fact error recall knowledge fragment.

[0150] The content of each initial script (taking the fifth initial script as an example) is analyzed for fact tracing, to determine whether the fifth initial script contains information of the recall knowledge fragment with the second annotation result of "0". Specifically, the text content of the fifth initial script can be compared with all factually incorrect recall knowledge fragments through semantic matching technology, the core information (such as the specific value of the sum of the angles in a triangle) related to the factual statement in the fifth initial script is extracted, and it is checked whether the core information is derived from the fragment with the second annotation result of "0". If the fifth initial script contains a statement consistent with the core information of the factually incorrect fragment (such as "the sum of the angles in a triangle is 200 degrees"), it is determined that the fifth initial script contains the factually incorrect recall knowledge fragment; if the factual information in the fifth initial script is derived from the fragment with the second annotation result of "1", or does not involve any content of the factually incorrect fragment, it is determined that the fifth initial script does not contain the factually incorrect recall knowledge fragment.

[0151] If the fifth initial script does not contain any information of the recall knowledge fragment with the second annotation result of "0", it indicates that the fifth initial script is accurate and correct in terms of facts, and the factually incorrect feedback value of the fifth initial script can be set to 0 (no penalty); if the fifth initial script contains information from one factually incorrect fragment, it indicates that the fifth initial script has a slight factual error, and the factually incorrect feedback value of the fifth initial script can be set to be lower than 0, such as -0.5; if the fifth initial script contains information of two or more factually incorrect fragments, it indicates that the factual error of the fifth initial script is more serious, and the factually incorrect feedback value of the fifth initial script is adjusted to be lower accordingly, such as -1.

[0152] In a possible implementation, the second feedback value includes a fluency feedback value, and determining the second feedback value corresponding to each of the plurality of initial scripts includes:

[0153] The grammatical correctness, completeness of expression, and logical coherence of the sentences of the plurality of initial scripts are respectively checked to obtain a fluency checking result corresponding to each of the plurality of initial scripts;

[0154] The fluency feedback value corresponding to each of the plurality of initial scripts is determined according to the fluency checking result corresponding to each of the plurality of initial scripts.

[0155] In the embodiments of the present application, the language expression quality of the initial script is evaluated, the grammatical correctness, completeness of expression, and logical coherence of the initial script are systematically checked, and the fluency of the generated content is converted into a quantifiable fluency feedback value.

[0156] In some embodiments, the fluency-related multi-dimension checking is performed on each of the plurality of initial scripts. It should be understood that, for the checking of sentence fluency, the main focus is on whether there are grammatical errors, inappropriate word usage, mixed sentence patterns, etc. in each initial script. The pre-trained grammar correction model (such as a text correction model based on the Transformer architecture) can be used to scan each sentence in the initial script, identify and mark the sentences that are not fluent (for example, the subject-predicate-object combination error expression such as “Actor A works M starring”), and count the proportion of non-fluent sentences in the total sentences in the script. For the checking of expression completeness, the focus is on whether there are semantic defects, incomplete expressions, etc. in the initial script. For example, check whether there are sentences such as “Actor A won the award M” that lack an object (such as “an award”), or paragraphs that have obvious logical breaks. The proportion of the number of incomplete expressions is used as a checking indicator. For the checking of logical coherence, the focus is on whether the connection between sentences and paragraphs in the script is natural, whether there are logical jumps or contradictions. The semantic similarity between adjacent sentences (such as the cosine similarity of sentence vectors output by the BERT model) can be used to evaluate the coherence. If the similarity is lower than a preset threshold (such as 0.3), the sentence combination is considered to have a logical fault, and the number of occurrences of logical faults is counted.

[0157] After the above multi-dimension checking is completed, the checking results of the initial script need to be integrated into the fluency checking result. For example, the integration method can be, but is not limited to, using a weighted scoring system. The weights of sentence fluency, expression completeness, and logical coherence are set to 0.4, 0.3, and 0.3, respectively. According to the actual performance of each indicator (such as 90% of fluent sentences corresponding to 0.9 points, 80% of complete expressions corresponding to 0.8 points, and 70% of logically coherent segments corresponding to 0.7 points), the weighted total score (such as 0.9 x 0.4 + 0.8 x 0.3 + 0.7 x 0.3 = 0.81) is calculated. The total score is the fluency checking result of the initial script, and the value range can be set to 0-1. The higher the score, the better the fluency.

[0158] In some embodiments, the first fluency threshold and the second fluency threshold can be preset, wherein the first fluency threshold is greater than the second fluency threshold. If the fluency check result of the initial script is higher than the first fluency threshold (e.g., 0.8), it indicates that the initial script is fluent, complete in expression, logically coherent, and excellent in fluency, and the fluency feedback value of the initial script can be set as a positive reward value (e.g., 1). If the fluency check result of the initial script is between the second fluency threshold (e.g., 0.5) and the first fluency threshold, it indicates that the initial script has a small amount of incoherence or incompleteness, but the overall logic is identifiable, and the fluency is good, and the fluency feedback value of the initial script can be set as a moderate positive value (e.g., 0.5). If the check result is lower than the second fluency threshold, it indicates that the initial script has more syntax errors, incomplete expression, or logic confusion, and the fluency is poor, and the fluency feedback value of the initial script can be set as a negative penalty value (e.g., -0.5).

[0159] In a possible implementation manner, the second feedback value includes a word count feedback value, and determining the second feedback value corresponding to each of the plurality of initial scripts includes:

[0160] If the actual word count of the sixth initial script in the plurality of initial scripts is greater than the preset script word count, a word count difference between the actual word count of the sixth initial script and the preset script word count is calculated.

[0161] The word count difference corresponding to the sixth initial script is multiplied by a preset word count feedback coefficient to obtain a word count feedback value corresponding to the sixth initial script.

[0162] In order to evaluate whether the length of the initial script meets the preset requirement and avoid information redundancy or lack of emphasis due to too long content, in specific implementation, a preset script word count can be set for the initial script according to the requirements of the application scenario. For example, in a customer service Q&A scenario, in order to ensure the simplicity and efficiency of the reply, the preset script word count can be set as 200 words.

[0163] For the sixth initial script in the plurality of initial scripts, the actual word count contained in the sixth initial script is first counted. If the actual word count is greater than the preset script word count, a word count difference between the two is calculated, that is, the result obtained by subtracting the preset script word count from the actual word count. Then, the word count difference is multiplied by a preset word count feedback coefficient to obtain a word count feedback value corresponding to the sixth initial script.

[0164] The preset word number feedback coefficient is a negative coefficient (such as -0.01). When the actual word number of an initial script exceeds the preset script word number, it means that the initial script has a problem of being too long, and needs to be punished by a negative feedback value. The more the words exceed, the greater the punishment. For example, if the actual word number of the sixth initial script is 250 words, and the preset script word number is 200 words, the word number difference is 50 words, multiplied by the preset word number feedback coefficient -0.01, the word number feedback value obtained is -0.5, which can quantitatively evaluate the quality loss of the generated script due to too many words.

[0165] In a possible implementation, the second feedback value includes a language consistency feedback value, and determining the second feedback value corresponding to each of the plurality of initial scripts includes:

[0166] determining whether the seventh initial script in the plurality of initial scripts has a mixed situation of different national languages, to obtain a third determination result;

[0167] According to the third determination result, the language consistency feedback value corresponding to the seventh initial script is determined.

[0168] To evaluate whether the initial script maintains the unity of a single language and avoid affecting the reading experience and the accuracy of information transmission due to mixed languages. In specific implementation, for each initial script (the seventh initial script) in the plurality of initial scripts, a language recognition tool (such as a language classification model based on machine learning) is used to detect the language of the content of the seventh initial script, to determine whether the seventh initial script has a mixed situation of different national languages.

[0169] If the detection finds that only one national language (such as all Chinese) is used in the seventh initial script, and no characters or sentences of other national languages appear, it is determined that there is no mixed situation of different national languages. If the detection finds that two or more different national languages (such as Chinese sentences mixed with English words, short sentences, or Chinese and Japanese expressions) are included in the script, it is determined that there is a mixed situation of different national languages.

[0170] According to the above determination result, if it is determined that there is no mixed situation of languages in the seventh initial script, it means that the language of the seventh initial script is unified, and the language consistency feedback value corresponding to the seventh initial script can be set to 0 (no punishment). If it is determined that there is a mixed situation of languages in the seventh initial script, according to the different degrees of mixing, a corresponding negative punishment value is set, for example, slight mixing (such as only a few foreign words) can be set to -0.3, and severe mixing (such as multiple different languages appearing alternately) can be set to -0.8, to quantitatively evaluate the quality performance of the generated script in terms of language consistency.

[0171] In a possible implementation, as Figure 5As shown, the initial retrieval enhancement generation model is optimized according to the generation quality feedback results corresponding to the plurality of initial scripts to obtain a target retrieval enhancement generation model, including:

[0172] S501, a plurality of initial scripts, and a plurality of initial scripts corresponding to the generation quality feedback results, are constructed to obtain a plurality of training samples.

[0173] S502, the plurality of training samples are divided into a training data set and a validation data set.

[0174] S503, the training data set is input into the initial retrieval enhancement generation model, and each parameter in the initial retrieval enhancement generation model is updated to minimize the preset loss function to obtain a model after parameter update.

[0175] The preset loss function includes a generalized relative policy optimization (GRPO) loss function.

[0176] S504, the validation data set is input into the model after parameter update, and whether the model after parameter update has converged is detected.

[0177] S505, in the case where the model after parameter update has converged, it is determined that the model after parameter update has been trained, and a target retrieval enhancement generation model is obtained.

[0178] In some embodiments, a plurality of training samples are first constructed based on a plurality of initial scripts and their respective generation quality feedback results. Each training sample needs to include two parts: input information and feedback signal. The input information is the query request and the corresponding recall knowledge fragment used when generating the initial script, which ensures that the model can associate with the specific input context. The feedback signal is the generation quality feedback result (such as the quantitative value obtained by the foregoing addition, averaging or normalization operation) of the initial script, which is used to indicate the quality of the initial script. For example, for the query request "the core principle of a certain technology" and the related recall knowledge fragment, if the feedback result of the generated initial script A is 0.8 (high quality) and the feedback result of the initial script B is -0.3 (low quality), the training samples are constructed as: query request + recall knowledge fragment, 0.8, query request + recall knowledge fragment, -0.3, etc. A set containing a sufficient number of samples (such as 10,000 samples) is formed to cover various input scenarios and quality performances.

[0179] Next, the constructed multiple sets of training samples are divided into a training data set and a validation data set. Among them, the division ratio can be adjusted according to the total amount of samples, and a ratio of 7:3 or 8:2 is usually adopted, for example, 7000 groups are randomly selected from 10000 groups of samples as the training data set for updating learning of model parameters; the remaining 3000 groups are used as the validation data set to evaluate the performance of the model on data that does not participate in training, avoiding overfitting of the model. The distribution consistency of the two data sets should be ensured during the division process, that is, the samples in the training set and the validation set should be similar in terms of query request type, recalled knowledge fragment features, generated quality feedback result range, etc., to ensure the reliability of the validation result.

[0180] Subsequently, the training data set is input into the initial retrieval enhancement generation model to update the parameters of the initial retrieval enhancement generation model with the goal of minimizing the preset loss function. It should be understood that the preset loss function adopts a generalized relative policy optimization (GRPO) loss function, which is suitable for reinforcement learning scenarios and can adjust the generation strategy of the model through the generated quality feedback result as a reward signal.

[0181] Specifically, the GRPO loss function compares the probability of the model generating different initial scripts with the matching degree of the corresponding feedback results, so that the initial retrieval enhancement generation model is more inclined to output scripts with higher feedback results in subsequent generation. The parameter updating process adopts the gradient descent method: first, calculate the gradient of the GRPO loss function under the current model parameters, and then adjust the parameters (such as weights, biases, etc.) in the opposite direction of the gradient, and iterate this process until the loss value on the training data set tends to be stable.

[0182] After updating the model parameters, the validation data set needs to be input into the model after parameter updating to detect whether the model has converged. The core indicator of convergence detection is the stability of the model's performance on the validation data set: in consecutive multiple training rounds (such as 10 rounds), whether the fluctuation amplitude of the average generated quality feedback result of the model on the validation set is less than a preset threshold (such as ±0.02), and whether the loss function value tends to be stable (such as the continuous decrease amplitude is less than 0.01). If the above conditions are met, it indicates that the model has learned a stable generation rule and can maintain good performance on unseen data, i.e., the model has converged; if not, parameter updating iteration needs to continue until the convergence condition is met.

[0183] In the case where the model after parameter updating has converged, it is determined that the model training is completed, i.e., the optimized target retrieval enhancement generation model is obtained. When receiving new query requests and recalled knowledge fragments, the target retrieval enhancement generation model can preferentially reference knowledge fragments with high relevance and high accuracy based on the rules learned during the training process, avoid redundancy, noise and factual errors, and generate high-quality scripts with high fluency, compliance with word requirements and consistent language, thereby significantly improving the overall quality of retrieval enhancement generation.

[0184] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0185] The optimization method of the retrieval enhancement generation model corresponding to the above embodiment, Figure 6 is a structural schematic diagram of an optimization device of a retrieval enhancement generation model provided by the embodiments of the present application. The device can be realized by software, hardware or a combination of the two to become part or all of a computer device, which can be Figure 7 the electronic device shown in the figure.

[0186] Referring to Figure 6 , the optimization device of the retrieval enhancement generation model comprises:

[0187] The retrieval unit 601 is configured to retrieve a plurality of recall knowledge segments related to the query request from the pre-constructed knowledge base based on the input query request.

[0188] The labeling unit 602 is configured to perform quality labeling of a plurality of dimensions on the plurality of retrieved recall knowledge segments respectively, to obtain quality labeling results of the plurality of recall knowledge segments respectively.

[0189] The generation unit 603 is configured to input the query request and the plurality of recall knowledge segments into an initial retrieval enhancement generation model to be optimized, to generate a plurality of initial scripts, wherein each initial script contains at least one recall knowledge segment.

[0190] The determination unit 604 is configured to determine a generation quality feedback result corresponding to each of the plurality of initial scripts based on the quality labeling results of the plurality of recall knowledge segments respectively and the plurality of initial scripts, wherein the generation quality feedback result is used to quantitatively evaluate the generation quality of the initial script.

[0191] The optimization unit 605 is configured to optimize the initial retrieval enhancement generation model according to the generation quality feedback result corresponding to each of the plurality of initial scripts, to obtain a target retrieval enhancement generation model.

[0192] It can be understood that the retrieval enhancement generation model optimization device embodiment and any one of the implementation manners correspond to the retrieval enhancement generation model optimization method embodiment and any one of the implementation manners respectively. The technical effects corresponding to the retrieval enhancement generation model optimization device embodiment and any one of the implementation manners can be referred to the technical effects corresponding to the retrieval enhancement generation model optimization method embodiment and any one of the implementation manners, which will not be repeated here.

[0193] It should be noted that the above embodiment provides the optimization device for searching the enhanced generation model, and only the division of each function module is exemplified, and in actual application, the above function distribution can be completed by different function modules according to needs, that is, the internal structure of the device is divided into different function modules to complete all or part of the above described functions.

[0194] Each function unit and module in the above embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. In addition, the specific names of each function unit and module are only for mutual distinction, and are not used to limit the protection scope of the embodiments of the present application.

[0195] It should be noted that the information interaction, execution process and the like between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the brought technical effects can be referred to the method embodiments part, and will not be repeated here.

[0196] The embodiments of the present application also provide an electronic device, which includes one or more processors and a memory.

[0197] The memory is coupled with the one or more processors, and the memory is configured to store computer program codes including computer instructions. The one or more processors invoke the computer instructions to enable the electronic device to execute the above-mentioned optimization method for searching the enhanced generation model.

[0198] Figure 7 A structural schematic diagram of an electronic device provided by the embodiments of the present application is shown. The electronic device 700 can be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, a vehicle-mounted electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, or a server, a memory, a base station, or other communication devices, or a smart car. The embodiments of the present application do not make any limitation on the specific type of the electronic device.

[0199] The memory 701 can be used to store computer program 702 and modules, and the processor 703 executes various function applications and data processing of the electronic device by running the software programs and modules stored in the memory 701. The memory 701 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the memory 701 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0200] The processor 703 can include one or more of a central processor, an application processor (AP), a baseband processor, etc. The processor can be the nerve center and command center of the wireless router. The processor 703 can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions. The memory 701 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 703 executes various function applications and data processing of the network device by running the instructions stored in the memory. The memory 701 can include a program storage area and a data storage area, such as data of a sound signal to be played, etc. For example, the memory can be a double data rate synchronous dynamic random access memory (DDR) or a flash memory (Flash), etc.

[0201] The embodiments of the present application also provide a computer readable storage medium, wherein the computer readable storage medium stores computer instructions; when the computer readable storage medium runs on an electronic device, the electronic device executes the optimization method for searching an enhanced generation model.

[0202] The computer instructions can be stored in or transferred from one computer-readable storage medium to another computer-readable storage medium, such as from one website, computer, server, or data center to another website, computer, server, or data center, through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device, such as one or more servers, data centers, etc., integrated with one or more media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0203] The embodiments of the present application also provide a computer program product containing computer instructions, which, when the computer program product runs on an electronic device, enables the electronic device to execute the foregoing search enhancement generation model optimization method.

[0204] The computer storage medium and the computer program product provided by the embodiments of the present application are used to execute the method provided above, and thus the beneficial effects that can be achieved are referred to the beneficial effects of the method provided above, which will not be described herein again.

[0205] In the embodiments described above, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, such as from a website site, computer, server or data center to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server, data center, etc. integrated with one or more available media. The storage medium can be a magnetic disk, an optical disk, a read-only memory (Rom), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage medium can also include a combination of the above types of memory.

[0206] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0207] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments applied herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0208] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other manners. For example, the embodiments of the apparatus / network device described above are merely illustrative. For example, the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0209] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0210] The above-described embodiments are merely used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An optimization method for retrieving an enhanced generative model, characterized in that, The method comprises the following steps: retrieving a plurality of recall knowledge pieces related to the query request from a pre-constructed knowledge base based on the input query request; performing quality labeling of the plurality of retrieved recall knowledge pieces in multiple dimensions respectively to obtain quality labeling results of the plurality of recall knowledge pieces respectively; inputting the query request and the plurality of recall knowledge pieces into an initial retrieval enhancement generation model to be optimized to generate a plurality of initial scripts, wherein each of the initial scripts contains at least one of the recall knowledge pieces; determining a plurality of generation quality feedback results corresponding to the plurality of initial scripts respectively based on the quality labeling results of the plurality of recall knowledge pieces respectively and the plurality of initial scripts, wherein the generation quality feedback results are used to quantitatively evaluate the generation quality of the initial scripts; optimizing the initial retrieval enhancement generation model according to the generation quality feedback results corresponding to the plurality of initial scripts respectively to obtain a target retrieval enhancement generation model.

2. The method of claim 1, wherein, The plurality of dimensions include: a relevance dimension of the query request and the recall knowledge pieces, a factual accuracy dimension of the recall knowledge pieces, and a conflict dimension between different recall knowledge pieces; the quality labeling of the plurality of retrieved recall knowledge pieces in multiple dimensions respectively to obtain the quality labeling results of the plurality of recall knowledge pieces respectively comprises: for the relevance dimension, scoring the semantic similarity between each of the recall knowledge pieces and the query request respectively to obtain a first labeling result corresponding to each of the plurality of recall knowledge pieces respectively; for the factual accuracy dimension, performing binary labeling on the content authenticity of each of the recall knowledge pieces respectively to obtain a second labeling result corresponding to each of the plurality of recall knowledge pieces respectively; for the conflict dimension, judging whether there is a conflict between each of the recall knowledge pieces and other recall knowledge pieces respectively to obtain a third labeling result corresponding to each of the plurality of recall knowledge pieces respectively.

3. The method of claim 1, wherein, The determination of the plurality of generation quality feedback results corresponding to the plurality of initial scripts respectively based on the quality labeling results of the plurality of recall knowledge pieces respectively and the plurality of initial scripts comprises: determining a plurality of first feedback values corresponding to the plurality of initial scripts respectively based on the quality labeling results of the plurality of recall knowledge pieces respectively and the plurality of initial scripts, wherein the first feedback values include at least one of a relevance feedback value, a conflict resolution feedback value, a redundancy noise feedback value, and a factual error feedback value; determining a plurality of second feedback values corresponding to the plurality of initial scripts respectively, wherein the second feedback values include at least one of a fluency feedback value, a word count feedback value, and a language consistency feedback value; performing mathematical operations on the first feedback values and the second feedback values corresponding to the plurality of initial scripts respectively to obtain the generation quality feedback results corresponding to the plurality of initial scripts respectively, wherein the mathematical operations include: numerical addition, average taking, and normalization operation.

4. The method of claim 3, wherein, The first feedback value includes a relevance feedback value, the quality annotation result includes a first annotation result for a relevance dimension, the first annotation result at least includes a semantic similarity score, and determining the first feedback value corresponding to each of the initial scripts based on the quality annotation result of each of the recall knowledge segments and the initial scripts includes: determining a first relevant segment with a semantic similarity score higher than a first threshold from the recall knowledge segments according to the first annotation result corresponding to each of the recall knowledge segments; calculating a semantic knowledge overlap degree between each of the initial scripts and the first relevant segment respectively; determining a relevance feedback value corresponding to a first initial script from the initial scripts according to a comparison result of the semantic knowledge overlap degree corresponding to the first initial script and a preset overlap degree.

5. The method of claim 3, wherein, The first feedback value includes a conflict resolution feedback value, the quality annotation result includes a second annotation result for a fact accuracy dimension and a third annotation result for a conflict dimension, and determining the first feedback value corresponding to each of the initial scripts based on the quality annotation result of each of the recall knowledge segments and the initial scripts includes: determining whether a target recall knowledge segment with a second annotation result of content truth exists in a third initial script from the initial scripts, and whether a conflict exists between the target recall knowledge segment and other recall knowledge segments; determining a conflict resolution feedback value corresponding to the third initial script according to whether the target recall knowledge segment in the third initial script is content true and whether a conflict exists between the target recall knowledge segment and other recall knowledge segments.

6. The method of claim 3, wherein, The first feedback value includes a redundancy noise feedback value, the quality annotation result includes a first annotation result for a relevance dimension, and determining the first feedback value corresponding to each of the initial scripts based on the quality annotation result of each of the recall knowledge segments and the initial scripts includes: determining a second relevant segment with a semantic similarity score lower than a second threshold from the recall knowledge segments according to the first annotation result corresponding to each of the recall knowledge segments; determining a first judgment result by judging whether at least one second relevant segment exists in a fourth initial script from the initial scripts and / or whether a duplicate recall knowledge segment exists; determining a redundancy noise feedback value corresponding to the fourth initial script according to the first judgment result.

7. The method of claim 3, wherein, The first feedback value includes a fact error feedback value, the quality annotation result includes a second annotation result for a fact accuracy dimension, and determining the first feedback value corresponding to each of the initial scripts based on the quality annotation result of each of the recall knowledge segments and the initial scripts includes: determining a second judgment result by judging whether a recall knowledge segment with a second annotation result of content error exists in a fifth initial script from the initial scripts; determining a fact error feedback value corresponding to the fifth initial script according to the second judgment result.

8. The method according to any one of claims 3 to 7, characterized in that, The second feedback value includes a fluency feedback value, and the determining of the second feedback value corresponding to each of the initial scripts includes: checking the grammaticality, completeness and logical coherence of the sentences of the initial scripts respectively to obtain a fluency checking result corresponding to each of the initial scripts; determining a fluency feedback value corresponding to each of the initial scripts according to the fluency checking result corresponding to each of the initial scripts.

9. The method according to any one of claims 3 to 7, characterized in that, The second feedback value includes a word count feedback value, and the determining of the second feedback value corresponding to each of the initial scripts includes: if the actual word count of a sixth initial script in the initial scripts is greater than a preset script word count, calculating a word count difference between the actual word count of the sixth initial script and the preset script word count; multiplying the word count difference corresponding to the sixth initial script by a preset word count feedback coefficient to obtain a word count feedback value corresponding to the sixth initial script.

10. The method according to any one of claims 1 to 7, characterized in that, The optimization of the initial retrieval and enhancement generation model according to the generation quality feedback result corresponding to each of the initial scripts to obtain a target retrieval and enhancement generation model includes: constructing a plurality of training samples from the initial scripts and the generation quality feedback result corresponding to each of the initial scripts; dividing the training samples into a training data set and a validation data set; inputting the training data set into the initial retrieval and enhancement generation model to update each parameter in the initial retrieval and enhancement generation model to minimize a preset loss function, and obtaining a parameter-updated model, wherein the preset loss function includes a generalized relative policy optimization (GRPO) loss function; inputting the validation data set into the parameter-updated model to detect whether the parameter-updated model has converged; in the case that the parameter-updated model has converged, determining that the parameter-updated model has been trained to obtain the target retrieval and enhancement generation model.

Images