Search enhancement generation method, apparatus, device, storage medium, and program product

Abstract

The application provides a retrieval enhanced generation method, device, equipment, storage medium and program product. The method comprises the following steps: receiving the dialogue content of a user, and generating a query vector based on the dialogue content; determining an initial candidate segment set from a database based on the query vector; determining a target segment set based on the initial candidate segment set, the query vector, and a multi-dimensional index score; the multi-dimensional index score comprises a relevance score, a redundancy score and a coverage score; and generating response content corresponding to the dialogue content based on the target segment set. By optimizing the multi-dimensional index of the initial candidate segment set, the multi-dimensional index of the segment can be considered comprehensively, and low-quality or irrelevant content can be eliminated, thereby improving the overall quality of the target segment set. In addition, since the target segment set has been screened and optimized for multiple rounds, it can more accurately answer the user's questions and meet the user's needs, thereby reducing the time for the user to obtain effective information.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a retrieval enhancement generation method, apparatus, device, storage medium, and program product. Background Technology

[0002] Retrieval-Enhanced Generation (RAG) significantly enhances the capabilities of language models by combining large amounts of external data with the underlying model, making AI responses more realistic, personalized, and reliable. The core of RAG technology lies in combining two key technologies: retrieval and generation. The retrieval module finds the most relevant information to the query from a large amount of data, and then the generation module uses this retrieved information to generate a response or answer.

[0003] However, existing technologies use plaintext filtering schemes, simplify the filtering logic by sorting by relevance and deduplication by hashing, and only focus on the relevance between fragments and queries for filtering, resulting in poor filtering accuracy, which in turn leads to inaccurate subsequent responses or answers. Summary of the Invention

[0004] This application provides a retrieval enhancement generation method, apparatus, device, storage medium, and program product to improve screening accuracy and optimize response result quality.

[0005] In a first aspect, embodiments of this application provide a retrieval enhancement generation method, including:

[0006] Receive user dialogue content and generate query vectors based on the dialogue content;

[0007] Based on the query vector, an initial set of candidate fragments is determined from the database;

[0008] The target fragment set is determined based on the initial candidate fragment set and query vector, as well as multi-dimensional indicator scores; the multi-dimensional indicator scores include: relevance score, redundancy score, and coverage score.

[0009] Based on the target fragment set, generate response content corresponding to the dialogue content.

[0010] In one embodiment, the target fragment set is determined based on an initial candidate fragment set and a query vector, as well as multi-dimensional metric scores, specifically including:

[0011] For the segments in the initial candidate segment set, optimization processing is performed based on multi-dimensional index scores to determine the candidate segment set;

[0012] Based on the candidate fragment set, the target fragment set is determined.

[0013] In one embodiment, the segments in the initial candidate segment set are optimized based on multi-dimensional index scores to determine the candidate segment set, specifically including:

[0014] Select N segments from the initial candidate segment set as a subset of candidate segments;

[0015] Calculate the multi-dimensional index scores of candidate fragment subsets;

[0016] Based on multi-dimensional index scores, the fragments in the candidate fragment subset are adjusted and updated;

[0017] The multi-dimensional index scores of the adjusted candidate fragment subset are recalculated until the number of updates reaches the preset number or the preset stopping condition is met. The candidate fragment subset updated last is then used as the candidate fragment set.

[0018] In one embodiment, the segments in the candidate segment subset are adjusted and updated based on multi-dimensional index scores, specifically including:

[0019] Based on multi-dimensional index scores, adjust the segment weights of segments in the candidate segment subset;

[0020] Fragments with a weight lower than the first preset value are removed from the candidate fragment subset;

[0021] The remaining fragments in the candidate fragment set are added to the candidate fragment subset to form a new candidate fragment subset.

[0022] In one embodiment, the remaining fragments in the candidate fragment set are added to the candidate fragment subset, specifically including:

[0023] If the relevance score of the remaining segments is greater than or equal to the second preset value, and the redundancy score of the remaining segments is less than or equal to the third preset value, then

[0024] The remaining fragments are sorted according to their relevance scores and added to the candidate fragment subset.

[0025] In one embodiment, the segment weights of segments in the candidate segment subset are adjusted based on multi-dimensional index scores, specifically including:

[0026] For segments with relevance scores higher than the average relevance score, increase the weight of the segments;

[0027] For segments with redundancy scores higher than the average redundancy score, the weight of the segment is reduced;

[0028] Increase the weight of segments with higher coverage scores than the average coverage score.

[0029] In one embodiment, generating a query vector based on the dialogue content specifically includes:

[0030] Based on the dialogue content, obtain the plaintext vector of the dialogue content;

[0031] The plaintext vector is encrypted using a preset encryption algorithm to generate a query vector.

[0032] In one embodiment, the plaintext vector is encrypted using a preset encryption algorithm to generate a query vector, specifically including:

[0033] Generate a rotation matrix based on a preset key;

[0034] A query vector is generated by performing a linear transformation on the plaintext vector using a rotation matrix.

[0035] In one embodiment, an initial set of candidate fragments is determined from the database based on the query vector, specifically including:

[0036] Based on the query vector, calculate the similarity between all segments in the database and the query vector;

[0037] Segments with similarity greater than the fourth preset value are grouped into an initial candidate segment set; the initial candidate segment set includes: a secret state vector, secret state segment content, and a keyword hash set.

[0038] In one embodiment, based on the target fragment set, response content corresponding to the dialogue content is generated, specifically including:

[0039] The target fragment set is encrypted content;

[0040] After decrypting the target fragment set, the response content is generated and transmitted through a secure channel.

[0041] In one embodiment, the target fragment set is determined based on an initial candidate fragment set and a query vector, as well as multi-dimensional metric scores, specifically including:

[0042] Calculate multi-dimensional index scores based on the initial candidate fragment set and query vector;

[0043] Based on the multi-dimensional index scores, the contribution of each segment in the initial candidate segment set to the multi-dimensional index scores is obtained.

[0044] The fragments in the initial candidate fragment set are sorted in descending order according to their contribution, and the top K fragments are selected as the target fragment set.

[0045] In one embodiment,

[0046] Calculate scores for multi-dimensional metrics, including:

[0047] Calculate the cosine similarity between the query vector and each segment in the initial candidate segment set to generate a relevance score;

[0048] For each segment in the initial candidate segment set, the pairwise similarity of the keyword hash set is calculated to generate a redundancy score;

[0049] The keyword hash set of the query vector is matched with the global hash set of the initial candidate fragment set to generate a coverage score.

[0050] Multi-dimensional index scores are calculated based on relevance score, redundancy score, and coverage score.

[0051] In one embodiment, a multi-dimensional index score is calculated based on relevance score, redundancy score, and coverage score, specifically including:

[0052] The weight value of the relevance score is obtained by multiplying the relevance score by its weight coefficient.

[0053] The weight value of the redundancy score is obtained by multiplying the redundancy score by its weight coefficient.

[0054] The weight value of the coverage score is obtained by multiplying the coverage score by its weight coefficient.

[0055] The weighted value of the relevance score is subtracted from the weighted value of the redundancy score, and then the weighted value of the coverage score is added to obtain the multi-dimensional indicator score.

[0056] In one embodiment, the sum of the weight coefficients of the relevance score, the weight system of the redundancy score, and the weight coefficients of the coverage score is 1;

[0057] The weighting coefficient of the relevance score is greater than or equal to the fifth preset value.

[0058] In one embodiment, a cosine similarity score is calculated between the query vector and each segment in the initial candidate segment set to generate a relevance score, specifically including:

[0059] Calculate the cosine similarity between the fragment vector and the query vector;

[0060] The average cosine similarity of all candidate segments in the initial set is taken as the relevance score.

[0061] In one embodiment, pairwise similarity is calculated for the keyword hash set of each segment in the initial candidate segment set to generate a redundancy score, specifically including:

[0062] For all segments in the initial candidate segment set, calculate the similarity of the keyword hash sets between each pair of segments;

[0063] The average of all similarities is used as the redundancy score.

[0064] In one embodiment, the keyword hash set of the query vector is intersected with the global hash set of the initial candidate fragment set to generate a coverage score, specifically including:

[0065] Merge the keyword hash sets of all fragments in the initial candidate fragment set to obtain the global hash set;

[0066] Calculate the number of intersections between the global hash set and the keyword hash set of the query vector;

[0067] The coverage score is calculated based on the number of intersections.

[0068] In one embodiment, before receiving the user's query content, the method further includes:

[0069] Encryption operations are performed on the original document library to obtain the encrypted vector and encrypted fragment content;

[0070] Obtain the plaintext vector of each fragment in the original document library, extract keywords from each plaintext vector, and generate a keyword hash set using a hash function;

[0071] The keyword hash set, the secret vector, and the secret fragment content are stored to form a database.

[0072] Secondly, embodiments of this application provide a retrieval enhancement generation apparatus, comprising:

[0073] The receiving module is used to receive the user's dialogue content and generate query vectors based on the dialogue content;

[0074] The determination module is used to determine the initial set of candidate fragments from the database based on the query vector;

[0075] The processing module is used to determine the target fragment set based on the initial candidate fragment set and query vector, as well as multi-dimensional indicator scores; the multi-dimensional indicator scores include: relevance score, redundancy score and coverage score;

[0076] The generation module is used to generate response content corresponding to the dialogue content based on the target fragment set.

[0077] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0078] The memory stores the instructions that the computer executes;

[0079] The processor executes computer execution instructions stored in memory, causing the processor to perform any of the methods described above.

[0080] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the methods described above.

[0081] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements any of the methods described above.

[0082] This application provides a retrieval enhancement generation method, apparatus, device, storage medium, and program product. The method includes: receiving user dialogue content and generating a query vector based on the dialogue content; determining an initial candidate fragment set from a database based on the query vector; determining a target fragment set based on the initial candidate fragment set, the query vector, and multi-dimensional indicator scores; the multi-dimensional indicator scores include: relevance score, redundancy score, and coverage score; and generating response content corresponding to the dialogue content based on the target fragment set. By optimizing the initial candidate fragment set using multi-dimensional indicators, the multi-dimensional indicators of the fragments can be comprehensively considered, eliminating low-quality or irrelevant content, thereby improving the overall quality of the target fragment set. Secondly, the response content generated based on the target fragment set, which is related to the dialogue content, can more accurately answer the user's questions and meet the user's needs since the target fragment set has undergone multiple rounds of screening and optimization, reducing the time it takes for the user to obtain effective information. Attached Figure Description

[0083] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0084] Figure 1 A flowchart illustrating a retrieval enhancement generation method provided in an embodiment of this application;

[0085] Figure 2 A schematic flowchart of a search enhancement generation method provided in another embodiment of this application;

[0086] Figure 3 This is a schematic diagram of the structure of a search enhancement generation system provided in an embodiment of this application;

[0087] Figure 4 A schematic diagram of a search enhancement generation apparatus provided in an embodiment of this application;

[0088] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0089] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0090] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0091] First, let me explain the terms used in this application:

[0092] Rotation vector encryption: A linear transformation encryption technique based on orthogonal matrices that maintains the similarity of the encrypted vectors.

[0093] Encrypted semantic vector: An encrypted document or query vector used for encrypted similarity calculation.

[0094] Cosine similarity: measures the cosine of the angle between vectors and is used to quantify the degree of semantic matching.

[0095] In fields with extremely high data privacy requirements, such as healthcare, law, and finance, retrieval-enhanced generation (RAG)-based question-answering systems are widely used in knowledge-intensive scenarios. For example, in healthcare, sensitive information such as patient medical records and diagnostic records needs to be quickly retrieved through RAG systems to generate personalized medical advice; in legal scenarios, privacy protection and accurate matching of case documents and contract terms are core requirements; and in financial scenarios, the privacy and security of user transaction data and the efficient processing of risk assessment reports directly affect service compliance.

[0096] Existing technology: To achieve high-precision filtering and generation of massive documents while ensuring privacy, existing RAG systems typically employ plaintext filtering schemes, using relevance ranking and hash deduplication logic. They only focus on the relevance of fragments and queries, which can easily miss key information, resulting in insufficient filtering accuracy or omission of key information, affecting the filtering quality, and consequently leading to inaccurate response results that cannot accurately answer user questions.

[0097] To address the shortcomings of existing technologies, the inventors of this solution have creatively designed a new approach. This solution provides an enhanced retrieval generation method to solve the problems of poor filtering accuracy and inaccurate response results. This application is applicable to knowledge-intensive scenarios with extremely high data privacy requirements, such as healthcare, law, and finance. For example, in healthcare, sensitive information such as patient medical records and diagnostic records needs to be quickly retrieved through the RAG system to generate personalized medical recommendations; in legal scenarios, case documents and contract terms need to be accurately matched under privacy protection; and in financial scenarios, user transaction data and risk assessment reports need to be processed efficiently under compliance requirements.

[0098] This application provides a retrieval enhancement generation method that optimizes the initial candidate fragment set using multi-dimensional indicators. This comprehensively considers the multi-dimensional indicators of the fragments, eliminating low-quality or irrelevant content, thereby improving the overall quality of the target fragment set. It balances relevance, redundancy, and coverage requirements, ensuring the comprehensiveness and accuracy of the screening results. Secondly, it generates response content related to the dialogue content based on the target fragment set. Since the target fragment set has undergone multiple rounds of screening and optimization, it can more accurately answer the user's questions, meet the user's needs, and reduce the time users spend obtaining effective information.

[0099] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0100] like Figure 1 As shown, Figure 1 A flowchart illustrating a retrieval enhancement generation method according to an embodiment of this application. The dialog method includes the following steps:

[0101] Step S101: Receive the user's dialogue content and generate a query vector based on the dialogue content.

[0102] Specifically, it receives dialogue content input by the user, converts the dialogue content into a mathematical vector, and provides a basis for subsequent retrieval and matching.

[0103] Step S102: Determine the initial candidate fragment set from the database based on the query vector.

[0104] Specifically, in a pre-built database, content fragments similar to or related to the query vector are retrieved. The pre-built database stores a large number of text fragments, such as knowledge bases and historical dialogue records. The set of fragments that are closest to the query vector is found through vector similarity calculation and used as the initial candidate fragment set.

[0105] Step S103: Based on the initial candidate fragment set and query vector, as well as the multi-dimensional index scores, determine the target fragment set; the multi-dimensional index scores include relevance score, redundancy score and coverage score.

[0106] Specifically, the fragments in the initial candidate fragment set are further optimized from the perspective of multi-dimensional indicators to remove irrelevant or low-quality fragments. Through optimization and screening, the final target fragment set is determined. The multi-dimensional indicator scores are adopted, taking into account relevance, redundancy and coverage, which solves the problem that multiple indicators cannot be jointly optimized in the existing technology. This ensures that the screening results are highly relevant, low in redundancy and fully covered, which is especially suitable for complex scenarios that need to take into account multiple conflicting indicators.

[0107] Step S104: Based on the target fragment set, generate response content corresponding to the dialogue content.

[0108] Specifically, the content of the target fragment set is integrated and optimized to generate the final response content. Optionally, the response content is a polished, supplemented, or adjusted version of the target fragment set by the large language model to make the response content more in line with the expression habits of natural language and to answer the user's dialogue content.

[0109] This application optimizes the initial candidate fragment set using multi-dimensional metrics, comprehensively considering the multi-dimensional metrics of the fragments and eliminating low-quality or irrelevant content, thereby improving the overall quality of the target fragment set. Secondly, it generates response content related to the dialogue content based on the target fragment set. Since the target fragment set has undergone multiple rounds of screening and optimization, it can more accurately answer the user's questions, meet the user's needs, and reduce the time it takes for the user to obtain effective information.

[0110] In one embodiment, step S101 specifically includes the following:

[0111] Based on the dialogue content, obtain the plaintext vector of the dialogue content.

[0112] The plaintext vector is encrypted using a preset encryption algorithm to generate a query vector.

[0113] Specifically,

[0114] In one embodiment, the plaintext vector is encrypted using a preset encryption algorithm to generate a query vector, specifically including the following steps:

[0115] Generate a rotation matrix based on a preset key.

[0116] A query vector is generated by performing a linear transformation on the plaintext vector using a rotation matrix.

[0117] Specifically, the dialogue content is encrypted to generate a query vector. This is then processed by analyzing the plaintext vector. In this example, the default encryption algorithm is Lightweight Rotation Vector Encryption.

[0118] In one embodiment, plaintext vector Including fragment semantic vectors and query semantic vector Rotation matrix controlled by key K Perform a linear transformation to generate a query vector. At this point, the query vector is encrypted. It is a d×d orthogonal matrix, where d is the vector dimension, for example, 768 dimensions, adapted to the BERT pre-trained model, satisfying the following conditions. ( (where the identity matrix is ​​used) to ensure that the vector magnitude and the angle between vectors remain unchanged.

[0119] The encrypted query vector has the property of preserving the secret state similarity, that is, for any two plaintext vectors and Its dense state vector and satisfy: Where Sim is the cosine similarity, ensuring that the similarity can be directly calculated in the dense state without decryption.

[0120] This application introduces lightweight rotation vector encryption, which is based on linear transformation of orthogonal matrices. The encryption or decryption time is only slightly increased compared to plaintext. It supports vector similarity calculation in the encrypted state, and subsequent multi-dimensional index calculations do not require complex homomorphic operations, resulting in low overall screening time, ensuring privacy and security, and meeting the needs of real-time RAG applications. It balances user privacy and security with computational efficiency. Furthermore, based on the encrypted characteristics of rotation vector encryption and keyword hashing technology, it achieves the calculation of three-dimensional index scores of "relevance, redundancy, and coverage" without decrypting fragment content or vectors.

[0121] The retrieval enhancement generation method provided in this application can be calculated and applied in plaintext or in encrypted form, and the choice can be made based on the specific application scenario.

[0122] In one embodiment, prior to step S101, the following steps are also included:

[0123] Encryption operations are performed on the original document library to obtain the encrypted vector and encrypted fragment content.

[0124] Obtain the plaintext vector of each fragment in the original document library, extract keywords from each plaintext vector, and generate a keyword hash set using a hash function.

[0125] The keyword hash set, the secret vector, and the secret fragment content are stored to form a database.

[0126] Specifically, after encrypting the fragments in the original document library, an encrypted document library, or database, is obtained. This provides a cryptographic environment for subsequent computations, eliminating the need to decrypt the fragment content or vectors. End-to-end privacy control ensures that sensitive vectors and fragment content are decrypted only when necessary, avoiding the risk of vector or fragment content leakage. By using rotated vectors to encrypt the original document library, the vectors or fragment content in the encrypted database also possess cryptographic similarity preservation properties; that is, computations can be performed directly in the cryptographic environment without decryption.

[0127] In one embodiment, step S104 specifically includes the following:

[0128] The target fragment set is encrypted content.

[0129] After decrypting the target fragment set, the response content is generated and transmitted through a secure channel.

[0130] Specifically, the target fragment set is input into the Large Language Model (LLM) forward key management submodule to request a key, decrypt the target fragment set, adjust the fragment format of the target fragment set according to the LLM input window, and then transmit it through a secure channel. After the transmission is completed, the plaintext fragments and the key in the target fragment set are immediately erased.

[0131] This application employs a privacy control mechanism across the entire dialogue process to ensure that sensitive data is not leaked in plaintext during storage, filtering, calculation, and generation of the target fragment set. The database only stores keyword hash sets, encrypted vectors, and encrypted fragment content, without any plaintext information. In the initial retrieval stage, both the query vector and the encrypted vector in the database participate in encrypted form without decryption. In the three-index calculation stage, only the query vector and keyword hash are used, without decrypting the fragment content. A secure generation adaptation module is used, decrypting only before the target fragment set is input into the LLM. After decryption, the plaintext fragment is only temporarily stored in memory (not written to disk), and the plaintext and key are erased within 1 second after transmission. The session key K is only transmitted to the authorization module through a secure channel and is destroyed immediately after use, leaving no key residue.

[0132] Taking a privacy-preserving computing scenario-based intelligent question-answering assistant as an example, the original document library contains tens of thousands of content fragments related to privacy-preserving computing scenarios. For example: Business unit Shanghai branch, partner A, conducted a privacy-preserving computing dormant user activation campaign from May to August 2025, with a total of 1.646 million participants, 1.284 million overlapping users, and an activation rate of 8.93%. Vector extraction and encryption were performed on the original document library, and each fragment... Extracting semantic vectors Call the rotation vector encryption submodule and use the session key K to... Encryption generates dense state vectors And perform fragment encryption and hashing: use encryption algorithms to encrypt the fragments. Encrypted generation of encrypted fragment content Extract keywords "Shanghai branch, A, dormant user activation, activation rate 8.93%" to generate keyword hash set. The extracted content will be encrypted and stored, that is, { The fragment ID is stored in an encrypted database.

[0133] In one embodiment, step S103 specifically includes the following steps:

[0134] Based on the query vector, calculate the similarity between all segments in the database and the query vector.

[0135] Segments with similarity greater than the fourth preset value are grouped into an initial candidate segment set; the initial candidate segment set includes: a secret state vector, secret state segment content, and a keyword hash set.

[0136] Specifically, in a closed-state environment, based on the similarity between the query vector and all segments in the database, segments with similarity higher than a fourth preset value are selected to obtain an initial candidate segment set. For example: User input: A business person inputs "the effect of the dormant account activation activity between the Shanghai branch and Taobao". The system extracts the semantic vector from the user's dialogue content and encrypts it. It then calculates the similarity with the segment vectors in the database, that is, extracts the core demand keywords "Shanghai branch, Taobao, dormant account activation, activity effect, improvement suggestions", and generates a hash set. Extract semantic vectors Encryption generation ;calculate With all in the database The similarity is used to filter out 50 segments with a similarity ≥ a fourth preset value, forming a dense-state candidate segment set E(S). Optionally, the fourth preset value is 0.65, but it can also be other values; this application does not impose any restrictions here. Secondly, the data of the segment vectors in the initial candidate segment set is not limited and can be set according to actual needs. In this example, 50 segment vectors are selected.

[0137] Optionally, the cosine similarity between all segments in the database and the query vector can be calculated. The cosine value of the angle between the two segments can be used to quantify the degree of semantic matching.

[0138] This application uses rotation vector encryption technology to calculate encrypted similarity without decrypting the vector, thus reducing the risk of privacy leakage; it also combines cosine similarity to filter highly relevant segments, thereby improving the efficiency of initial retrieval.

[0139] In one embodiment, step S103 specifically includes the following steps, such as... Figure 2 As shown, Figure 2 A flowchart illustrating a dialogue method provided in another embodiment of this application:

[0140] Step S201: Select N segments from the initial candidate segment set as a subset of candidate segments.

[0141] Specifically, N segments are randomly selected from the initial candidate segment set or selected according to a certain strategy (such as similarity) to form a smaller subset of candidate segments. Here, N is a preset number, usually determined based on actual needs and computing resources. The value of N should be chosen appropriately to allow for adjustment.

[0142] Optionally, the weight of each segment in the candidate segment subset is initialized as follows: The initial weights are set within the range [0.3, 0.7]. This provides an initial evaluation benchmark for each segment. In the calculation of multi-dimensional metric scores, the weights affect the contribution of each segment to the overall performance. By setting initial weights, a basic estimate of the importance of each segment can be obtained at the beginning of the optimization process. For example, suppose we have 5 candidate segments with initial weights as follows: Segment 1: W10 = 0.5; Segment 2: W20 = 0.6; Segment 3: W30 = 0.4; Segment 4: W40 = 0.7; Segment 5: W50 = 0.3. These initial weights indicate that at the beginning of the optimization, segment 4 is considered relatively more important (highest weight), while segment 5 is relatively less important (lowest weight). Therefore, setting the initial weights within the range [0.3, 0.7] can avoid extreme cases. If the weights are initialized to 0 or 1, some segments may be over-ignored or over-emphasized during the optimization process, thus affecting the fairness and effectiveness of the optimization.

[0143] Step S202: Calculate the multi-dimensional index scores of the candidate fragment subset.

[0144] In one embodiment, the multi-dimensional indicator score includes: relevance score, redundancy score, and coverage score; calculating the multi-dimensional indicator score includes the following steps:

[0145] Cosine similarity is calculated between the query vector and each segment in the candidate segment subset to generate a relevance score. Pairwise similarity is calculated for the keyword hash sets of each segment in the candidate segment subset to generate a redundancy score. Intersection matching is performed between the keyword hash set of the query vector and the global hash set of the candidate segment subset to generate a coverage score. Multi-dimensional metric scores are calculated based on the relevance score, redundancy score, and coverage score.

[0146] Specifically, for example, if there are 10 segments in the candidate segment subset, the relevance score, redundancy score, and coverage score of each segment are calculated. For example, segment 1 has a relevance score of 0.8, a redundancy score of 0.3, and a coverage score of 0.6; segment 2 has a relevance score of 0.6, a redundancy score of 0.9, and a coverage score of 0.5, and so on. Based on the relevance score, redundancy score, and coverage score, a multi-dimensional index score is calculated to obtain the final score.

[0147] Step S203: Based on the multi-dimensional index scores, adjust and update the segments in the candidate segment subset.

[0148] Specifically, based on the example above, the relevance score of fragment 2 is low (0.6). Consider finding a sentence with higher relevance from the initial candidate fragment set to replace fragment 2. Fragment 11 is found from the initial candidate fragment set, with a relevance score of 0.9, a redundancy score of 0.2, and a coverage score of 0.8. Fragment 11 replaces fragment 2, and the candidate fragment subset is updated.

[0149] Step S204: Recalculate the multi-dimensional index scores of the adjusted candidate fragment subset until the number of updates reaches the preset number or the preset stopping condition is met, and take the last updated candidate fragment subset as the candidate fragment set.

[0150] Specifically, after each adjustment, the multi-dimensional index scores of the candidate fragment subset need to be recalculated to evaluate the effect of the adjustment. This process is repeated until a preset number of updates is reached or a preset stopping condition is met, at which point the last updated candidate fragment subset is taken as the final candidate fragment set. The preset number of updates and the preset stopping condition can be determined based on actual needs and computing resources. The preset stopping condition can be stopping when the relevance score reaches a certain threshold, or stopping when the relevance score, redundancy score, and coverage score all meet the preset stopping condition. This application does not impose a limitation based on the actual situation. For example, if the number of updates is 3, after 3 updates, the last updated candidate fragment subset is taken as the final candidate fragment set. This application uses an iterative optimization method to further optimize the initial candidate fragment set, gradually adjusting the candidate fragment subset to improve its comprehensive performance in multi-dimensional indicators; this is very effective when processing large amounts of data, ensuring that the final selected candidate fragment set achieves good results in multiple aspects.

[0151] In one embodiment, a multi-dimensional index score is calculated based on relevance score, redundancy score, and coverage score, specifically including the following steps:

[0152] The weight value of the relevance score is obtained by multiplying the relevance score by its weight coefficient.

[0153] The weight value of the redundancy score is obtained by multiplying the redundancy score by its weight coefficient.

[0154] The weight value of the coverage score is obtained by multiplying the coverage score by its weight coefficient.

[0155] The weighted value of the relevance score is subtracted from the weighted value of the redundancy score, and then the weighted value of the coverage score is added to obtain the multi-dimensional indicator score.

[0156] Specifically, by iteratively optimizing and dynamically eliminating low-contribution fragments and strengthening high-value fragments within the candidate fragment subset, the overall quality of the candidate fragment subset is ultimately optimized. For example, a fitness function is defined to quantify the "overall quality" of the candidate fragment subset, and the scores of the three indicators are dynamically adjusted through weight coefficients, i.e., multi-dimensional indicator scores, as shown in the formula:

[0157]

[0158] in, , , These are the relevance score, redundancy score, and coverage score, respectively. , , These are the weighting coefficients for relevance score, redundancy score, and coverage score, respectively, and they satisfy... .

[0159] In one embodiment, the sum of the weight coefficient of the relevance score, the weight system of the redundancy score, and the weight coefficient of the coverage score is 1; the weight coefficient of the relevance score is greater than or equal to a fifth preset value.

[0160] Optionally, the weighting coefficients of the relevance score Ensure that relevance score is the core indicator; for example, in the context of summary generation, set... , , Priority is given to coverage score and low redundancy. The higher the F-value of the three indicators, the higher the overall quality of the candidate fragment subset, i.e., high relevance, low redundancy, and high coverage.

[0161] For example, a multi-level weight adjustment mechanism is introduced to prioritize the three indicators (relevance, redundancy, and coverage) in a hierarchical manner. For instance, in a medical question-and-answer scenario, the priority order is relevance > coverage > redundancy, while in a legal contract scenario, it is adjusted to coverage > relevance > redundancy. Therefore, this application does not limit the optimization of model parameters through dynamic matching using a scenario-based weight rule base.

[0162] In one embodiment, step S203 specifically includes the following steps:

[0163] Based on multi-dimensional index scores, the fragment weights of fragments in the candidate fragment subset are adjusted.

[0164] Fragments with a weight lower than the first preset value are removed from the candidate fragment subset.

[0165] The remaining fragments in the candidate fragment set are added to the candidate fragment subset to form a new candidate fragment subset.

[0166] Specifically, update the subset of candidate fragments. In this example, the goal is to eliminate low-weight segments and add new ones. For example: weighting... (Threshold adjustable) segments from Removed from the middle.

[0167] In one embodiment, the segment weights of segments in the candidate segment subset are adjusted based on multi-dimensional index scores, specifically including:

[0168] For segments with a relevance score higher than the average relevance score, increase the weight of the segment.

[0169] For segments with redundancy scores higher than the average redundancy score, the weight of the segment is reduced.

[0170] Increase the weight of segments with higher coverage scores than the average coverage score.

[0171] Specifically, the weights are adjusted based on the "contribution" of each segment to the scores of the three indicators. This approach strengthens high-contribution segments and suppresses low-contribution segments. Segments with relevance scores higher than the subset's average relevance score are given increased weight; segments with redundancy scores higher than the subset's average redundancy score are given decreased weight; and segments containing the query's core hash (i.e., segments with high coverage scores) are given additional weight. This ensures high relevance, low redundancy, and high coverage.

[0172] In one embodiment, the remaining fragments in the candidate fragment set are added to the candidate fragment subset, specifically including:

[0173] If the relevance score of the remaining fragments is greater than or equal to the second preset value, and the redundancy score of the remaining fragments is less than or equal to the third preset value, then the remaining fragments that meet the conditions will be sorted according to their relevance scores and added to the candidate fragment subset.

[0174] Specifically, update the subset of candidate fragments. In this example, the goal is to eliminate low-weight segments and add new ones. For example: weighting... (Threshold adjustable) segments from Remove from the initial candidate fragment set and supplement with high-potential fragments: from the remaining fragments in the initial candidate fragment set, i.e., those that did not enter the candidate fragment subset. For each segment, segments with a relevance score greater than or equal to a second preset value and a redundancy score less than or equal to a third preset value are selected. These segments are then sorted according to their relevance scores, and segments with high relevance scores are added to the subset to form a new subset. The process continues iterating until the termination condition is met, resulting in the final set of candidate fragments. .

[0175] This application employs a three-indicator collaborative optimization model with dynamic weight adjustments to ensure the screening logic aligns with the core needs of different scenarios. For example, coverage is prioritized in medical scenarios, while redundancy is prioritized in financial scenarios, avoiding screening bias caused by a single indicator. Secondly, an iterative optimization mechanism gradually improves the overall quality of the candidate subset by strengthening high-contribution segments and suppressing low-contribution segments. This solves the problem of the inability to jointly optimize multiple indicators in existing technologies, ensuring that the screening results are highly relevant, low in redundancy, and fully coverive. It is particularly suitable for complex scenarios that require balancing multiple conflicting indicators. For example, matching legal contract clauses requires simultaneous satisfaction of relevance and coverage.

[0176] In one embodiment, step S103 specifically includes the following steps:

[0177] Based on the initial candidate fragment set and query vector, multi-dimensional index scores are calculated.

[0178] Based on the multi-dimensional index scores, the contribution of each segment in the initial candidate segment set to the multi-dimensional index scores is obtained.

[0179] Specifically, for the initial candidate fragment set Each segment Segment weights are calculated based on their contribution to the three indicators. :

[0180]

[0181] in, The value ranges from 0 to 1. The larger the value, the higher the contribution of the fragment to the three-index score of the initial candidate fragment set.

[0182] The fragments in the initial candidate fragment set are sorted in descending order according to their contribution, and the top K fragments are selected as the target fragment set.

[0183] Specifically, according to weight Descending order of initial candidate fragment set for dense state Sort the data and select the top K segments, where K = 8 to 12, to fit the LLM input window (e.g., GPT-4 supports 8 segments as input), forming the target segment set. ,in, .

[0184] In one embodiment, the multi-dimensional indicator score includes: relevance score, redundancy score, and coverage score; calculating the multi-dimensional indicator score includes the following steps:

[0185] Cosine similarity is calculated between the query vector and each segment in the initial candidate segment set to generate a relevance score. Pairwise similarity is calculated between the keyword hash sets of each segment in the initial candidate segment set to generate a redundancy score. Intersection matching is performed between the keyword hash set of the query vector and the global hash set of the initial candidate segment set to generate a coverage score. Based on the relevance score, redundancy score, and coverage score, a multi-dimensional metric score is calculated.

[0186] Specifically, based on the encrypted state characteristics of rotation vector encryption and keyword hashing technology, the quantitative calculation of three indicators, namely "relevance, redundancy, and coverage," can be achieved without decrypting the fragment content or vector.

[0187] In one embodiment, cosine similarity is calculated between the query vector and each segment in the initial candidate segment set to generate a relevance score, specifically including the following steps:

[0188] Each fragment in the initial candidate fragment set is associated with a fragment vector.

[0189] Calculate the cosine similarity between the fragment vector and the query vector.

[0190] The average cosine similarity of all candidate segments in the initial set is taken as the relevance score.

[0191] Specifically, let the initial candidate fragment set be... The number of N ranges from 30 to 80, which balances computational efficiency and coverage. Each segment is associated with a segment vector. , For each fragment vector Call the rotation vector encryption submodule to calculate and query the vector. cosine similarity The relevance score is calculated by averaging all similarities, using the following formula:

[0192]

[0193] Among them, the relevance score The value ranges from 0 to 1. The higher the relevance score, the higher the semantic matching degree between the fragment and the query vector.

[0194] In one embodiment, pairwise similarity is calculated for the keyword hash set of each segment in the initial candidate segment set to generate a redundancy score, specifically including the following steps:

[0195] For all segments in the initial candidate segment set, calculate the similarity of the keyword hash sets between each pair of segments.

[0196] The average of all similarities is used as the redundancy score.

[0197] Specifically, the redundancy score is used to characterize the degree of information overlap between fragments within the initial candidate fragment set. The dense state calculation is performed based on keyword hashing, and the steps are as follows: For the initial candidate fragment set... Calculate the overlap rate of keyword hash sets for all pairwise segments, and use the Jaccard similarity metric to measure the degree of overlap. The formula is as follows:

[0198]

[0199] in, The number of intersections between the two hash sets. Let be the number of the union.

[0200] The redundancy score is calculated by taking the average Jaccard similarity of all pairwise segments, using the following formula:

[0201]

[0202] Among them, redundancy score The value ranges from 0 to 1. The smaller the redundancy score, the less information overlaps between segments, i.e., the lower the redundancy.

[0203] In one embodiment, the keyword hash set of the query vector is intersected with the global hash set of the initial candidate fragment set to generate a coverage score, specifically including the following steps:

[0204] Merge the keyword hash sets of all fragments in the initial candidate fragment set to obtain the global hash set.

[0205] Calculate the number of intersections between the global hash set and the keyword hash set of the query vector.

[0206] The coverage score is calculated based on the number of intersections.

[0207] Specifically, the coverage score is used to characterize the degree to which the initial candidate fragment set covers the core requirements of the query vector. This is achieved based on keyword hash matching between the query vector and the fragments, with the following steps: In the user query preprocessing stage, the core requirement words of the dialogue content are extracted. For example, if the user's dialogue content is: "UnionPay privacy computing sleep user activation activity effect", the keywords "UnionPay, sleep user activation, privacy computing, effect" are extracted and a "query keyword hash set" is generated using the SHA-256 hash function. (m=4~6, ensuring no core requirements are omitted); for the initial candidate fragment set The keyword hash sets of all fragments are merged to obtain the "global hash set". (Remove duplicates to avoid double counting); Calculate and Number of intersections The formula for coverage score is: Among them, the coverage score The value ranges from 0 to 1. The closer the coverage score is to 1, the more comprehensive the coverage of the core query requirements by the subset.

[0208] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a retrieval enhancement generation system provided in an embodiment of this application. The retrieval enhancement generation system includes: a privacy protection module, a document preprocessing module, a confidential preliminary retrieval module, a three-index collaborative optimization module, and a security generation adaptation module.

[0209] Privacy Protection Module: Includes Rotation Vector Encryption Submodule and Key Management Submodule; among which, Rotation Vector Encryption Submodule is responsible for encrypting or decrypting fragment vectors of the original document library and user query vectors, as well as calculating vector similarity in encrypted state; Key Management Submodule generates session-level rotation encryption key (K) and transmission key (Kt), the key can only be called by authorized modules and is destroyed immediately after use.

[0210] Document preprocessing module: Performs "semantic vector extraction - vector encryption - fragment encryption - hash storage" operations on fragments in the original document library, and outputs the database.

[0211] The initial encrypted state retrieval module receives the encrypted state query vector output by the privacy protection module, filters highly relevant candidate segments based on encrypted state similarity calculation, and outputs a "set of encrypted state candidate segments" (including encrypted state segment vectors, encrypted state segment content, and segment keyword hashes).

[0212] The three-index collaborative optimization module calculates the three indices of "relevance, redundancy, and coverage" of the candidate fragment set in a dense environment, determines the fragment weights through a collaborative optimization model, and outputs the target fragment set by combining Top-K filtering.

[0213] Secure Generation Adaptor Module: Only requests a key from the LLM forward key management submodule when inputting the target fragment set, decrypts the target fragment set, adjusts the fragment format according to the LLM input window, and transmits it through a secure channel. After transmission is complete, immediately erases the plaintext fragments and key of the target fragment set.

[0214] Taking a privacy-preserving intelligent question-answering assistant as an example, the implementation process of this application is explained in detail: Privacy protection parameters: Rotation vector encryption: Vector dimension 768, orthogonal rotation moment Generated from a 128-bit session key K; Fragment encryption: AES-256-GCM; Retrieval and filtering parameters: cryptographic similarity threshold T=0.65, initial number of candidate fragments N=50, Top-K value: K=8; Collaborative optimization parameters: weight coefficients , , , Number of core requirement words for query: m=4, Number of fragment keywords: 5. (1) Document preprocessing and encrypted storage: The original document library contains tens of thousands of privacy computing scenario content fragments, such as "Business Shanghai Branch, partner Taobao, to carry out privacy computing dormant user activation activities, the activity period is from May 2025 to August 2025, the total number of users is 1.646 million, the intersection of users is 1.284 million, and the activation rate is 8.93%"; Vector extraction and encryption are performed: for each fragment Extracting semantic vectors Call the rotation vector encryption submodule and use the session key K to... Encryption generates dense state vectors Fragment Encryption and Hashing: Encrypting fragments using the AES-256 algorithm Encrypted generation of encrypted fragment content Extract keywords "Shanghai branch, Taobao, dormant account activation, activation rate 8.93%" to generate a hash set. Encrypted storage: { , , , fragment ID} is stored in the MySQL encrypted database. (2) Preliminary encrypted retrieval: User query: Business personnel input "Is the effect of the Shanghai branch and Taobao's dormant account activation activities objective, and what are the suggestions for improvement?"; Query preprocessing: Extract the core demand keywords "Shanghai branch, Taobao, dormant account activation, activity effect, improvement suggestions", and generate a hash set. Extract semantic vectors Encryption generation Dense-state retrieval: calculation With all in the encrypted document library The similarity was used to select 50 segments with a similarity ≥ 0.65 to form the initial candidate segment set E(S). (3) Three-index co-optimization and Top-K screening: Three-index co-optimization to determine the candidate segment subset: dense state correlation score (Average similarity between 50 segments and the query); Dense redundancy score (Mean overlap rate of pairwise hash segments); Dense state coverage score (Global hash set coverage) (4 keywords in the text). Segment weight calculation: such as segment weight Top-K filtering: Select the top 8 segments in descending order of weight to form a subset of the dense-state optimal segments. (Including core segments such as business parties, partners, and cooperation effects). (4) Secure decryption and generation adaptation: Decryption: The secure generation adaptation module requests key K and decrypts the target segment set to obtain plaintext segments. ; Adaptation and transmission: Organize into the format of "Fragment 1 (ID: 12345-01): Business Shanghai Branch...; Fragment 2 (ID: 12345-05): Cooperation with Taobao..." and transmit to LLM; Privacy destruction: After transmission, erase plaintext fragments and key K, and destroy session key in the key management submodule.

[0215] like Figure 4 As shown, Figure 4 This is a schematic diagram of a retrieval enhancement generation device provided in an embodiment of this application. The retrieval enhancement generation device 400 includes: a receiving module 401, used to receive user dialogue content and generate a query vector based on the dialogue content; a determining module 402, used to determine an initial candidate fragment set from a database based on the query vector; a processing module 403, used to determine a target fragment set based on the initial candidate fragment set, the query vector, and multi-dimensional index scores; the multi-dimensional index scores include: relevance score, redundancy score, and coverage score; and a generating module 404, used to generate response content corresponding to the dialogue content based on the target fragment set.

[0216] In one embodiment, the processing module 403 is used to optimize the segments in the initial candidate segment set based on multi-dimensional index scores to determine the candidate segment set; and to determine the target segment set based on the candidate segment set.

[0217] In one embodiment, the processing module 403 is used to select N segments from the initial candidate segment set as a subset of candidate segments; calculate the multi-dimensional index score of the subset of candidate segments; adjust and update the segments in the subset of candidate segments based on the multi-dimensional index score; calculate the multi-dimensional index score of the adjusted subset of candidate segments again until the number of updates reaches a preset number, and take the last updated subset of candidate segments as the candidate segment set.

[0218] In one embodiment, the processing module 403 is used to adjust the segment weights of segments in the candidate segment subset based on multi-dimensional index scores; remove segments with segment weights lower than a first preset value from the candidate segment subset; and add the remaining segments in the candidate segment subset to form a new candidate segment subset.

[0219] In one embodiment, the processing module 403 is configured to, if the relevance score of the remaining fragments is greater than or equal to a second preset value and the redundancy score of the remaining fragments is less than or equal to a third preset value, then sort the remaining fragments according to their relevance scores and add them to the candidate fragment subset.

[0220] In one embodiment, the processing module 403 is configured to increase the weight of segments with relevance scores higher than the average relevance score; decrease the weight of segments with redundancy scores higher than the average redundancy score; and increase the weight of segments with coverage scores higher than the average coverage score.

[0221] In one embodiment, the receiving module 401 is used to obtain the plaintext vector of the dialogue content based on the dialogue content; and to perform a linear transformation on the plaintext vector through a key-controlled rotation matrix to generate a query vector.

[0222] In one embodiment, the determining module 402 is used to calculate the similarity between all segments in the database and the query vector based on the query vector; segments with similarity greater than a fourth preset value are used to form an initial candidate segment set; the initial candidate segment set includes: a secret vector, secret segment content and a keyword hash set.

[0223] In one embodiment, the generation module 404 is used to generate response content after the target fragment set is encrypted, and transmits the response content through a secure channel after decrypting the target fragment set.

[0224] In one embodiment, the processing module 403 is used to calculate a multi-dimensional index score based on an initial candidate fragment set and a query vector; based on the multi-dimensional index score, obtain the contribution of each fragment in the initial candidate fragment set to the multi-dimensional index score; sort the fragments in the initial candidate fragment set in descending order according to the contribution, and obtain the top K fragments as the target fragment set.

[0225] In one embodiment, the processing module 403 is configured to calculate the cosine similarity between the query vector and each segment in the initial candidate segment set to generate a relevance score; calculate pairwise similarity between the keyword hash sets of each segment in the initial candidate segment set to generate a redundancy score; and perform intersection matching between the keyword hash set of the query vector and the global hash set of the initial candidate segment set to generate a coverage score. Based on the relevance score, redundancy score, and coverage score, a multi-dimensional index score is calculated.

[0226] In one embodiment, the processing module 403 is used to obtain the weight value of the relevance score based on the product of the relevance score and its weight coefficient; to obtain the weight value of the redundancy score based on the product of the redundancy score and its weight coefficient; to obtain the weight value of the coverage score based on the product of the coverage score and its weight coefficient; and to obtain the multi-dimensional index score by subtracting the weight value of the redundancy score from the weight value of the relevance score and adding the weight value of the coverage score.

[0227] In one embodiment, the sum of the weight coefficient of the relevance score, the weight system of the redundancy score, and the weight coefficient of the coverage score is 1; the weight coefficient of the relevance score is greater than or equal to a fifth preset value.

[0228] In one embodiment, the processing module 403 is used to associate each fragment in the initial candidate fragment set with a fragment vector; calculate the cosine similarity between the fragment vector and the query vector; and take the average of all cosine similarities in the initial candidate fragment set as the relevance score.

[0229] In one embodiment, the processing module 403 is used to calculate the similarity of the keyword hash sets between each pair of segments in the initial candidate segment set; and to use the average of all similarities as the redundancy score.

[0230] In one embodiment, the processing module 403 is used to merge the keyword hash sets of all fragments in the initial candidate fragment set to obtain a global hash set; calculate the number of intersections between the global hash set and the keyword hash set of the query vector; and calculate the coverage score based on the number of intersections.

[0231] In one embodiment, the processing module 403 is used to perform encryption operations on the original document library to obtain encrypted vectors and encrypted fragment content; obtain the plaintext vector of each fragment in the original document library, extract keywords from each plaintext vector, and generate a keyword hash set through a hash function; and store the keyword hash set, encrypted vectors, and encrypted fragment content to form a database.

[0232] The retrieval enhancement generation device 400 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0233] This application provides an electronic device, including: a memory and a processor;

[0234] The memory stores the instructions that the computer executes;

[0235] The processor executes computer execution instructions stored in memory, causing the processor to perform any of the methods described above.

[0236] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 500 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 500 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0237] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0238] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0239] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0240] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0241] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0242] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0243] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0244] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0245] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0246] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0247] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0249] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0250] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0251] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A retrieval enhancement generation method, characterized in that, include: Receive the user's dialogue content and generate a query vector based on the dialogue content; Based on the query vector, an initial set of candidate fragments is determined from the database; N segments are selected from the initial candidate segment set as a subset of candidate segments; Calculate the multi-dimensional index scores of the candidate fragment subset; Based on the scores of the multi-dimensional indicators, the segment weights of the segments in the candidate segment subset are adjusted; Fragments with a weight lower than a first preset value are removed from the candidate fragment subset; The remaining fragments in the candidate fragment set are added to the candidate fragment subset to form a new candidate fragment subset; The multi-dimensional index scores of the adjusted candidate fragment subset are recalculated until the number of updates reaches a preset number or a preset stopping condition is met. The candidate fragment subset updated last time is then used as the candidate fragment set. Based on the candidate fragment set, the target fragment set is determined; The multi-dimensional indicator scores include: relevance score, redundancy score, and coverage score; Based on the target fragment set, generate response content corresponding to the dialogue content.

2. The method according to claim 1, characterized in that, The step of adding the remaining fragments from the candidate fragment set to the candidate fragment subset specifically includes: If the relevance score of the remaining fragments is greater than or equal to a second preset value, and the redundancy score of the remaining fragments is less than or equal to a third preset value, then The remaining fragments are sorted according to their relevance scores and added to the candidate fragment subset.

3. The method according to claim 1, characterized in that, The calculation of the multi-dimensional index score of the candidate fragment subset includes: Calculate the cosine similarity between the query vector and each segment in the candidate segment subset to generate a relevance score; For each segment in the candidate segment subset, perform pairwise similarity calculations on the keyword hash set to generate a redundancy score; The keyword hash set of the query vector is matched with the global hash set of the candidate fragment subset to generate a coverage score. The multi-dimensional index score is calculated based on the relevance score, the redundancy score, and the coverage score.

4. The method according to claim 3, characterized in that, The calculation of the multi-dimensional indicator score based on the relevance score, the redundancy score, and the coverage score specifically includes: The weight value of the relevance score is obtained by multiplying the relevance score by its weight coefficient. The weight value of the redundancy score is obtained by multiplying the redundancy score by its weight coefficient. The weight value of the coverage score is obtained by multiplying the coverage score by its weight coefficient. The multi-dimensional indicator score is obtained by subtracting the weight value of the redundancy score from the weight value of the relevance score and adding the weight value of the coverage score.

5. The method according to claim 4, characterized in that, The sum of the weight coefficients of the relevance score, the weight system of the redundancy score, and the weight coefficient of the coverage score is 1; The weighting coefficient of the relevance score is greater than or equal to the fifth preset value.

6. The method according to any one of claims 1-5, characterized in that, The generation of query vectors based on the dialogue content specifically includes: Based on the dialogue content, obtain the plaintext vector of the dialogue content; The plaintext vector is encrypted using a preset encryption algorithm to generate a query vector.

7. The method according to claim 6, characterized in that, The step of encrypting the plaintext vector using a preset encryption algorithm to generate a query vector specifically includes: Generate a rotation matrix based on a preset key; The query vector is generated by performing a linear transformation on the plaintext vector using the rotation matrix.

8. The method according to claim 7, characterized in that, The step of determining the initial candidate fragment set from the database based on the query vector specifically includes: Based on the query vector, calculate the similarity between all segments in the database and the query vector; The segments with a similarity greater than a fourth preset value are used to form the initial candidate segment set.

9. The method according to claim 1, characterized in that, Before receiving the user's query content, the process also includes: Encryption operations are performed on the original document library to obtain the encrypted vector and encrypted fragment content; Obtain the plaintext vector of each fragment in the original document library, extract keywords from each plaintext vector, and generate a keyword hash set using a hash function; The database is formed by storing the keyword hash set, the encrypted vector, and the encrypted fragment content.

10. A retrieval enhancement generation device, characterized in that, include: The receiving module is used to receive the user's dialogue content and generate a query vector based on the dialogue content; The determination module is used to determine an initial set of candidate fragments from the database based on the query vector; The processing module is used to select N segments from the initial candidate segment set as a subset of candidate segments; and to calculate the multi-dimensional index score of the subset of candidate segments. Based on the multi-dimensional index scores, the segment weights of segments in the candidate segment subset are adjusted; segments with segment weights lower than a first preset value are removed from the candidate segment subset; the remaining segments in the candidate segment subset are added to the candidate segment subset to form a new candidate segment subset; the multi-dimensional index scores of the adjusted candidate segment subset are calculated again until the number of updates reaches a preset number or a preset stopping condition is met, and the last updated candidate segment subset is taken as the candidate segment set; Based on the candidate fragment set, the target fragment set is determined; The multi-dimensional indicator scores include: relevance score, redundancy score, and coverage score; The generation module is used to generate response content corresponding to the dialogue content based on the target fragment set.

11. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images