A retrieval question and answer method based on vector enhancement, a storage medium and an equipment

Abstract

The application relates to the technical field of AI question and answer, in particular to a retrieval question and answer method based on vector enhancement, a storage medium and equipment.A retrieval question and answer method based on vector enhancement comprises the following steps: S10, a source document is processed by using a multi-dimensional element enhancement technology and a preset large language model, a unified document enhancement vector is obtained, and a vector database is established; S20, a business request is acquired, the business request is coded into a query vector, retrieval is performed based on the vector database, and a context is generated; S30, the context is input into the preset large language model to generate a final answer; S40, feedback data of a user on the final answer is acquired; and S50, according to the feedback data, a to-be-adjusted semantic dimension causing an error recall is located, and a unified document enhancement vector of an affected original text block is regenerated to update the vector database.The application improves the information maintenance degree and retrieval accuracy of five types of key document semantics.

Description

Technical Field

[0001] This invention relates to the field of AI question answering technology, and in particular to a retrieval question answering method, storage medium, and device based on vector enhancement. Background Technology

[0002] Existing large language model question answering techniques, such as Retrieval Augmentation (RAG), face bottlenecks when handling complex queries and diverse data. Traditional methods typically perform single-dimensional vectorization of unstructured text, making it difficult to deeply understand the user's true intent. When faced with semantically similar but different intents, vector search retrieval accuracy is low.

[0003] Furthermore, existing methods typically encode text blocks directly into fixed-dimensional vectors, making it difficult to simultaneously retain summary information, keywords, specialized entities, key parameters, and semantic tags within a limited vector capacity. Existing vector compression methods usually apply a uniform compression rate to the entire vector, failing to determine compression ratios for different semantic dimensions of the document. Generating complete vectors for multiple dimensions separately would increase the vector library size and similarity calculation overhead. Therefore, there is an urgent need for an intelligent question-answering method that can deeply understand semantics and possess adaptive evolutionary capabilities. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a retrieval question answering method, storage medium and device based on vector enhancement.

[0005] The technical solution adopted by this invention to solve its technical problem is: a retrieval and question answering method based on vector enhancement, comprising the following steps: S10: The source document is segmented into blocks to generate a set of original text blocks. Five types of document-side enhancement elements corresponding to five semantic dimensions (summary, keywords, entities, parameters, and semantic tags) of the original text blocks are extracted. The five types of semantic compression units are used to convert the five types of document-side enhancement elements into five types of compressed semantic codes according to a preset total compression dimension. The five types of compressed semantic codes are concatenated in a fixed order, and a semantic increment with the same basic vector dimension as the original text blocks is generated through the recovery mapping module. The semantic increments corresponding to the five types of compressed semantic codes are merged into a unified document enhancement vector, and a vector database is established. S20, Obtain a business request, retrieve the business request based on the vector database, obtain the original text block corresponding to the business request, and generate context; S30, input the context into a preset large language model to generate the final answer; S40, Obtain user feedback data on the final answer; S50, based on the feedback data, determine the semantic dimension to be adjusted that caused the erroneous recall, and construct an incremental training sample containing the business request, positive sample text block, and negative sample text block; use the incremental training sample to update at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted, and regenerate the unified document augmentation vector of the original text block corresponding to the feedback data to update the vector database.

[0006] Optionally, step S10 includes: S11, using the preset large language model to extract the five types of document-side enhancement elements from the original text block; S12, input the original text block into the base vector model to obtain the base vector; use five types of semantic compression units to compress and encode the five types of document-side enhancement elements according to a preset total compression dimension, and concatenate the outputs of each semantic compression unit according to a preset order of the five semantic dimensions to obtain a joint compressed semantic code; convert the joint compressed semantic code into a semantic increment with the same dimension as the base vector through the recovery mapping module; fuse the semantic increment with the base vector to obtain the unified document enhancement vector; S13, store the unified document enhancement vector into the vector database.

[0007] Optionally, step S20 includes: S21, The business request is converted into a query vector using the basic vector model; S22, retrieve the original text block corresponding to the business request from the vector database according to the query vector; obtain the context according to the original text block corresponding to the business request.

[0008] Optionally, S22 includes: Based on the query vector, a similarity search is performed in the vector database to obtain a preset number of unified document augmentation vectors; Based on the preset number of unified document enhancement vectors, the corresponding original text blocks are obtained by backtracking. The original text blocks obtained from the backtracking are merged and deduplicated, and then sorted and integrated according to the retrieval scores of the similarity search to obtain the context.

[0009] Optionally, step S30 includes: S31, combine the service request and the context to obtain a prompt word; S32, input the prompt word into the preset large language model to obtain the final answer.

[0010] Optionally, step S40 includes: S41, Collect user satisfaction ratings and usage feedback for the final answer to obtain the feedback data.

[0011] Optionally, step S50 includes: S51, obtain the corresponding original text block according to the feedback data; divide the corresponding original text block into positive sample text block and negative sample text block; construct the incremental training sample according to the business request, the positive sample text block and the negative sample text block; determine the semantic dimension to be adjusted according to the error content in the feedback data; S52, using the incremental training samples, calculate the retrieval loss between the business request and the positive sample text block and the negative sample text block, and update at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted according to the retrieval loss; S53, using the updated semantic compression unit and recovery mapping module, regenerate the unified enhancement vector of the original text block corresponding to the feedback data, and update the vector database using the regenerated unified enhancement vector.

[0012] The present invention also provides a computer-readable storage medium storing a computer program adapted for loading by a processor to perform the steps of the vector-enhanced retrieval question-answering method as described in any of the preceding claims.

[0013] The present invention also includes a computer device, comprising a memory and a processor; The memory contains computer programs; The processor executes the steps of the vector-enhanced retrieval question-answering method as described above by invoking the computer program stored in the memory.

[0014] The implementation of this invention has the following beneficial effects: This invention sets up dynamic compression processes for five types of document-side semantics: abstracts, keywords, entities, parameters, and semantic tags. Under a fixed total compression dimensionality budget, each semantic compression unit is trained to adapt to the corresponding compression rate. Dimensionality preservation loss is used to reduce the loss of key semantics, and dimension decoupling loss is used to reduce information redundancy among the five types of compressed semantic codes. Ultimately, only a single unified document augmentation vector is formed, which reduces vector library storage overhead and similarity calculation volume, and improves retrieval accuracy. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of a vector-enhanced retrieval question-answering method in one embodiment. Detailed Implementation

[0016] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0018] This invention provides a vector-enhanced retrieval and question-answering method, such as... Figure 1 As shown, it includes the following steps: S10: The source document is segmented into blocks to generate a set of original text blocks. Five types of document-side enhancement elements corresponding to the five semantic dimensions of the original text blocks—summary, keywords, entities, parameters, and semantic tags—are extracted. Using five types of semantic compression units, the five types of document-side enhancement elements are converted into five types of compressed semantic codes according to a preset total compression dimension. The five types of compressed semantic codes are concatenated in a fixed order, and a semantic increment with the same basic vector dimension as the original text blocks is generated through the recovery mapping module. The semantic increments corresponding to the five types of compressed semantic codes are merged into a unified document enhancement vector, and a vector database is established.

[0019] It should be noted that the source documents come from various reports, articles, knowledge bases, etc.

[0020] The source document is segmented into blocks, and PaddleOCR is used to parse the document, extracting titles and paragraphs to generate a set of raw text blocks. The extracted titles and paragraphs constitute the raw text blocks. Semantic dimensions are categorized into five types: abstract, keywords, entities, parameters, and semantic tags. Document-side enhancement elements are extracted from the raw text blocks in the raw text block set based on these five semantic dimensions. These document-side enhancement elements are then further divided according to the five extracted semantic dimensions, resulting in five categories of document-side enhancement elements.

[0021] Five types of document-side enhancement elements are converted into corresponding enhanced text vectors. Under a preset total compression dimension budget, the enhanced text vectors are converted into five types of compressed semantic codes using five types of semantic compression units. The five types of compressed semantic codes are concatenated in a fixed order, and a semantic increment with the same basic vector dimension as the original text block is generated through the recovery mapping module.

[0022] The basic vector dimension of the original text block is consistent with the preset total compression dimension.

[0023] In some scenarios, the semantic increments corresponding to the five types of compressed semantic codes are fused with the base vector to obtain a unified document enhancement vector.

[0024] The preset total compression dimension is replaced by D dimensions. In D dimensions, semantic compression units corresponding to five semantic dimensions are used to compress five types of document-side enhancement elements, resulting in five types of compressed semantic codes. Each semantic compression unit is a vector encoding module with trainable parameters, used to convert document-side enhancement elements of the corresponding semantic dimension into D-dimensional vector representations.

[0025] The semantic increments corresponding to the five types of compressed semantic codes include restoring the five types of compressed semantic codes to their corresponding D-dimensional semantic increments. The D-dimensional semantic increments corresponding to the five types of compressed semantic codes are then fused with the D-dimensional base vectors of the original text corresponding to the five types of compressed semantic codes to obtain a unified document augmentation vector, and a vector database is established.

[0026] In some scenarios, the total compression dimension is preset to 768 dimensions.

[0027] The five types of enhancement elements are used only for document lateral vector enhancement. Among them, the summary represents the core content of the original text block; keywords represent search trigger words and subject terms; entities represent the names of devices, systems, people, or objects; parameters represent numerical values, thresholds, statuses, versions, times, or object characteristics; and semantic tags represent the category and business purpose of the text block.

[0028] In this embodiment, dynamic compression refers to changing the compression dimension used for each semantic dimension during training, enabling the same semantic compression unit to generate compressed semantic codes under multiple candidate compression dimensions. The five types of compressed semantic codes are not indexed separately, but are concatenated and uniformly restored as semantic increments. Each original text block ultimately corresponds to only one unified document augmentation vector.

[0029] This vector database is a unified augmented vector database that generates augmented elements only for the original text blocks.

[0030] S20: Obtain the business request, retrieve the business request based on the vector database, obtain the original text block corresponding to the business request, and generate the context.

[0031] Specifically, business requests do not generate summaries, keywords, entities, parameters, or semantic tags, nor do they pass through the five types of semantic compression units. The query side only calls the basic vector model to obtain the query vector encoding. Then, the query vector is matched with the segment representation vectors stored in the vector database for similarity. Target segments with similarity higher than a preset threshold are selected. Finally, the original text corresponding to all target segments is sorted and concatenated according to similarity from high to low to form a question-answering context that can be input into a large language model. In this embodiment, the parameters used for the basic vector model remain unchanged.

[0032] S30: Input the context into the preset large language model to generate the final answer.

[0033] The generated context is used as prompt words and input into a pre-defined large language model. The large language model then outputs the corresponding question-and-answer results based on the context information, thus completing the vector-enhanced retrieval and question-and-answer process.

[0034] S40: Obtain user feedback data on the final answer.

[0035] Feedback data is obtained through user evaluation portals and Q&A process logs, and may include at least one of the following: user satisfaction rating for the final answer submission, correct or incorrect markings, error content annotations, and user markings for relevant or irrelevant original text blocks.

[0036] S50: Based on the feedback data, determine the semantic dimension to be adjusted that caused the false recall, and construct incremental training samples containing business requests, positive sample text blocks, and negative sample text blocks; use the incremental training samples to update at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted, and re-feed back the unified document augmentation vector of the original text block corresponding to the feedback data to update the vector database.

[0037] A positive sample text block refers to an original text block that is confirmed by the user to be related to the business request, contains the information required to answer the business request, or can support the final answer; a negative sample text block refers to an original text block that is confirmed by the user to be unrelated to the business request, does not contain the information required to answer the business request, or leads to erroneous recall and incorrect answers.

[0038] Based on feedback data, identify the semantic dimensions that need adjustment that led to erroneous recall.

[0039] Based on the feedback data corresponding to the business request and the original text blocks, incremental training samples are constructed, containing the business request, positive sample text blocks, and negative sample text blocks. While keeping the basic vector model parameters unchanged, at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted is updated using the incremental training samples, and a unified document augmentation vector for the affected original text blocks is regenerated. The semantic dimension to be adjusted corresponds to five semantic dimensions.

[0040] Specifically, the semantic dimension to be adjusted belongs to five categories of semantic dimensions. The corresponding semantic compression unit is determined based on the semantic dimension to be adjusted. At least one of the following is used: updating the corresponding semantic compression unit using retrieval loss, and restoring the mapping parameters.

[0041] In some scenarios, feedback data is associated with the corresponding business request, the block identifier of the retrieved original text block, vector similarity, final answer, and session identifier to form a structured feedback record. The structured feedback record is used to determine the positive and negative sample text blocks corresponding to the query and to construct incremental training samples.

[0042] This invention employs a dynamic compression process for five document-side semantic categories: abstracts, keywords, entities, parameters, and semantic tags. Under a fixed total compression dimensionality budget, each semantic compression unit is trained to adapt to a corresponding compression rate, generating enhanced elements from multiple potential perspectives and constructing a semantically rich unified vector database. Secondly, when a user asks a question, the unified vector retrieval accurately matches the context. Then, a large language model generates answers and collects user feedback, continuously optimizing the preset large language model to enhance its ability to generate enhanced elements. This invention not only improves the information preservation and retrieval accuracy of the five key document semantic categories through multi-dimensional semantic modeling under limited vector dimensionality and storage overhead, but also significantly improves the intent understanding, retrieval accuracy, and adaptive evolutionary capabilities of the question-answering system through multi-dimensional enhancement, unified indexing, and closed-loop self-optimization.

[0043] In one embodiment, step S10 includes: S11 uses a pre-defined large language model to extract five types of document-side enhancement elements from the original text block.

[0044] For each original text block, five types of document-side enhancement elements are generated: summary, keywords, entities, parameters, and semantic tags. These five types of enhancement elements are used only for document-side vector construction.

[0045] In one embodiment, a preset large language model is invoked to extract five types of enhancement elements according to a fixed prompt template, as shown below.

[0046] "You are an expert in building enterprise knowledge bases."

[0047] Based on the input document chunk, extract information from multiple perspectives that can enhance knowledge retrieval and question answering.

[0048] Require: All content must be sourced directly from the original text and must not be fabricated.

[0049] The content is accurate, concise, and free of redundancy.

[0050] The requirement is that at least one dimension of the information must not be empty.

[0051] Please extract the following content: 1. summary Technical summary of 100-200 words.

[0052] 2. keywords 5 to 15 core keywords.

[0053] 3. entities Extract entities by category, including equipment, component, process, operation, fault, parameter, scenario, organization, document, technology, time, and other.

[0054] 4. parameters Extract parameters (name, value, unit) from text.

[0055] 5. semantic_tags Generate 5 to 10 semantic tags.

[0056] Output format: { "summary": "", "keywords": [], "entities": {}, "parameters": [ { "name": "", "value": "", "unit": "" } ], "workflow": [], "semantic_tags": [] } Document snippet: {{chunk}}.

[0057] In some embodiments, the method further includes: extracting five types of enhancement elements only from the original text block, without generating content not recorded in the original text block. When the original text block does not contain a certain type of enhancement element, that type of enhancement element is recorded as null.

[0058] Each type of non-empty augmenting element is appended to the corresponding original text block according to a fixed template, forming summary augmented text, keyword augmented text, entity augmented text, parameter augmented text, and semantic tag augmented text. For null value augmenting elements, no corresponding augmented text is generated.

[0059] Assign block identifiers to the original text blocks and record the block identifier and enhancement type identifier in each type of enhanced text, so that all five types of enhanced text can be traced back to the same original text block.

[0060] S12: Input the original text block into the base vector model to obtain the base vector; use five types of semantic compression units to compress and encode the five types of document-side enhancement elements according to the preset total compression dimension, and concatenate the outputs of each semantic compression unit according to the preset order of the five semantic dimensions to obtain the joint compressed semantic code; convert the joint compressed semantic code into a semantic increment with the same dimension as the base vector through the recovery mapping module; fuse the semantic increment with the base vector to obtain the unified document enhancement vector.

[0061] Understandably, by using the basic vector model and five types of semantic compression units to perform vector encoding and compression training on the original text blocks and the five types of augmented text, a unified document augmentation vector corresponding to the original text blocks can be obtained.

[0062] Unified document augmentation vectors can preserve richer semantic information and structured features, while controlling the overall dimensionality through compression encoding, thus reducing the computational burden of generating the final answer.

[0063] S13, store the unified document enhancement vector to the vector database.

[0064] The mapping relationship between the unified document enhancement vector, block identifier, and original text block is written into the vector database, and a single vector index is established for the unified document enhancement vector; no separate vector indexes are established for the five types of enhanced text vectors and the five types of compressed semantic codes.

[0065] In some embodiments, the source document may be divided into at least one independent raw text block, and a corresponding block identifier may be assigned to each raw text block.

[0066] When segmenting the source document, the segmentation rules can be automatically adapted based on the document's format and content structure: if the source document is a structured document with standardized hierarchical headings, it is prioritized for segmentation based on chapters and paragraphs as natural boundaries, ensuring the integrity and consistency of the content theme in each original text block; if the source document is a long text without obvious structural markers, it can be segmented according to a preset character length range, combined with semantic coherence, avoiding splitting content with the same complete semantic meaning into different original text blocks. The set of original text blocks is denoted as […]. Where N represents the number of original text blocks, This represents the k-th original text block.

[0067] In some embodiments, step S12 includes: S121, input the original text block into the basic vector model E0 to obtain the D-dimensional basic vector. In this implementation, the D-dimensional dimension is set to 768, and the basic vector... Inputting the five types of non-empty augmented text into the same base vector model E0 yields the corresponding D-dimensional augmented text vectors. The document side and the query side use the same basic vector model parameters.

[0068] S122, under the preset total compression dimension budget, determine the compression dimensions corresponding to the five semantic dimensions of abstract, keywords, entities, parameters and semantic tags respectively, and ensure that the sum of the five compression dimensions does not exceed the preset total compression dimension budget D; set the corresponding semantic compression units for the five semantic dimensions respectively.

[0069] S123, five types of semantic compression units are used to perform text encoding and dimensionality reduction mapping on five types of document-side enhancement elements: summary, keywords, entities, parameters, and semantic tags. The outputs of each semantic compression unit are concatenated in a fixed order of summary, keywords, entities, parameters, and semantic tags to obtain five types of compressed semantic codes. The sum of the output dimensions of each semantic compression unit does not exceed the preset total compression dimension budget, and the output of the semantic compression unit corresponding to the null value enhancement element is a zero vector of the preset dimension.

[0070] S124, input the five types of compressed semantic codes into the recovery mapping module to obtain a D-dimensional semantic increment with the same dimension as the basic vector: in, This represents the five classes of compressed semantic codes corresponding to the k-th original text block. This indicates the restoration of the mapping weights. This indicates the restoration of the mapping bias. Indicates the preset activation function. This represents the D-dimensional semantic increment.

[0071] S125, the residuals of the D-dimensional semantic increment and the D-dimensional fundamental vector of the original text block are added and normalized to obtain the unified document enhancement vector: in This represents the base vector of the k-th original text block. This represents the corresponding unified document augmentation vector, and Normalize represents L2 vector normalization.

[0072] S126, construct training samples using queries, positive sample text blocks, and negative sample text blocks; input the query into the basic vector model to obtain the query vector, and generate unified document augmentation vectors corresponding to positive sample text blocks and negative sample text blocks according to S121 to S135 respectively.

[0073] S127, calculate the retrieval loss based on the similarity between the query vector and the positive sample document augmentation vector and the negative sample document augmentation vector; backpropagate the gradient of the retrieval loss to the recovery mapping module and the five types of semantic compression units, and update the parameters of the five types of semantic compression units and the recovery mapping parameters while keeping the basic vector model parameters unchanged.

[0074] If you need to retain the retrieval loss formula, it can be written as: in, For query vector, Augmentation vectors for positive sample documents Augmentation vectors for negative sample documents, The preset sorting interval is used to limit the similarity difference between the query vector and the positive sample document augmentation vector, and is set to 0.2.

[0075] S128. After training is completed, the five types of semantic compression units and the recovery mapping module that have been trained are used to generate unified document enhancement vectors for each original text block to be added to the database, according to S121 to S125.

[0076] In one embodiment, step S20 includes: S21 uses the basic vector model to convert business requests into query vectors.

[0077] It should be noted that no semantic extraction or dynamic compression of the five types are performed on business requests.

[0078] First, the business requests are preprocessed by cleaning, denoising, sentence segmentation, and semantic normalization. Then, the preprocessed business requests are globally semantically encoded using a pre-trained vector model, converting the natural language business requests into query vectors of fixed dimensions in the vector space.

[0079] S22, retrieve the original text block corresponding to the business request from the vector database according to the query vector; obtain the context based on the original text block corresponding to the business request.

[0080] Based on the query vector, an approximate nearest neighbor semantic matching retrieval is performed in the vector database. The semantic similarity between the query vector and each text augmentation vector in the vector database is calculated. At least one target vector whose similarity meets a preset threshold is selected, and the text data associated with the target vector is obtained as the matching context. This adapts to business requests with synonyms and different sentence structures, effectively solving the problems of semantic ambiguity and expression differences.

[0081] In one embodiment, S22 includes: Based on the query vector, a similarity search is performed in the vector database to obtain a preset number of unified document augmentation vectors.

[0082] Furthermore, the query vectors are used to perform similarity retrieval (cosine similarity calculation) to find the top-30 most similar unified document augmentation vectors as relevant vectors.

[0083] Based on a preset number of unified document augmentation vectors, the corresponding original text blocks are obtained by backtracking.

[0084] In some embodiments, the corresponding original text block is found from the vector database by using block identifiers corresponding to a preset number of unified document enhancement vectors.

[0085] The original text blocks obtained from the backtracking are merged and deduplicated, and then sorted and integrated according to the retrieval scores based on similarity to obtain the context.

[0086] It's important to note that this step doesn't directly build a multi-indexed knowledge base with "question index, summary index, answer index, and inverted index" based on the source document. Instead, it generates enhanced elements for the original text blocks, forming a unified enhanced vector database. The retrieval object is not a fusion of multiple parallel index results, but rather a backtracking of the corresponding original text blocks after a unified document enhanced vector retrieval, followed by deduplication and integration.

[0087] For the original text blocks matched by the retrieved unified document augmentation vectors, content redundancy cleaning and duplicate segment removal are first performed. Then, the text is concatenated and integrated according to semantic logic and word order, paragraph relationships are clarified, and sentence connection logic is supplemented to finally generate a coherent, complete, and logically sound overall contextual content. The unified document augmentation vectors in the vector database integrate multi-dimensional semantic features, knowledge expansion information, and contextual features. Compared with ordinary vectors, they can more accurately identify original text blocks that are deeply related to the query intent, while reducing interference from literal similar but semantically unrelated content.

[0088] In one embodiment, step S30 includes: S31 combines the business request and context to obtain the prompt word.

[0089] By combining the original text retrieved from the vector database with the business request, prompt words are generated. This can overcome the knowledge deadline limitations of large models, significantly reduce the occurrence of hallucinations, and make up for the shortcomings of private professional knowledge.

[0090] S32, input the prompt words into the preset large language model to get the final answer.

[0091] Generating the final answer using these prompts can comprehensively improve the accuracy, consistency, and detail of the response, support the fusion and output of multi-source heterogeneous knowledge, significantly reduce model fine-tuning and inference costs, improve system response speed, and enable traceable verification of the answer, ensuring business data security, facilitating content review and compliance management, and meeting the management requirements of nuclear power plants.

[0092] In one embodiment, step S40 includes: S41, Collect user satisfaction ratings and usage feedback for the final answer to obtain feedback data.

[0093] Feedback data is obtained through user rating portals and Q&A process logs. Feedback data includes at least one of the following: satisfaction rating, correct or incorrect answer markers, error content annotations, relevant or irrelevant markers for recalling the original text block, and user supplementary comments.

[0094] After users receive the final answer output by the system, a standardized evaluation entry point is set up, allowing users to rate and evaluate the answer based on dimensions such as accuracy, completeness, semantic relevance, and logical coherence. A text feedback entry point can also be provided, allowing users to fill in the reasons why the question remains unresolved, existing defects, and optimization suggestions. By linking user question content, retrieving text sources, generating answers, conversation information, and user actions through associated logs, structured scoring data, choice-based feedback, and custom text suggestions are collected uniformly. After data anonymization, filtering of invalid information, and consolidation, standardized and statistically analyzable user feedback data is generated.

[0095] In one embodiment, step S50 includes: S51, obtain the corresponding original text block based on the feedback data; divide the corresponding original text block into positive sample text block and negative sample text block; construct incremental training samples based on business requests, positive sample text block and negative sample text block; determine the semantic dimension to be adjusted based on the error content in the feedback data.

[0096] Specifically, based on the feedback data, original text blocks that are confirmed by users to be relevant to the business request, contain the information needed to answer the business request, or can support the final answer are identified as positive sample text blocks. Original text blocks that are confirmed by users to be irrelevant to the business request, do not contain the information needed to answer the business request, or lead to false recall or incorrect answers are identified as negative sample text blocks. Based on the errors in the feedback data, semantic dimensions to be adjusted are determined, such as numerical values, thresholds, status, or unit errors, and these are identified as parameter dimensions. Based on the errors in the feedback data, at least one semantic dimension to be adjusted is determined from the summary, keywords, entities, parameters, and semantic tags.

[0097] Among them, errors in topic summary correspond to the abstract dimension, errors in core terms correspond to the keyword dimension, errors in object name correspond to the entity dimension, errors in numerical values, thresholds, status, version, or time conditions correspond to the parameter dimension, and errors in text category or business purpose correspond to the semantic tag dimension; when the specific dimension cannot be determined, all five semantic dimensions are recorded as semantic dimensions to be adjusted.

[0098] S52, calculate the retrieval loss between the business request and the positive sample text block and the negative sample text block using incremental training samples, and update at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted according to the retrieval loss.

[0099] The business request is input into the basic vector model to obtain the query vector. According to S121 to S125, the unified document augmentation vectors corresponding to the positive sample text block and the negative sample text block are generated respectively. The retrieval loss is calculated based on the similarity difference between the query vector and the positive sample document augmentation vector and the negative sample document augmentation vector. Based on the retrieval loss, at least one of the semantic compression unit parameters and the recovery mapping parameters corresponding to the semantic dimension to be adjusted is updated.

[0100] In some scenarios, the parameters of the base vector model should remain unchanged.

[0101] S53, using the updated semantic compression unit and recovery mapping module, regenerates the unified augmented vector of the original text block corresponding to the feedback data, and updates the vector database using the regenerated unified augmented vector.

[0102] Using the updated semantic compression unit and recovery mapping module, the unified augmentation vector of the affected original text block is regenerated, and the vector database is updated using the regenerated unified augmentation vector.

[0103] The updated semantic compression unit and recovery mapping module are used to regenerate the unified document enhancement vector for the affected original text blocks and replace the corresponding historical vector in the vector database.

[0104] When only the semantic compression unit and its corresponding reconstruction mapping are updated, the original text blocks that participated in this incremental training or are associated with the semantic dimension to be adjusted are regenerated; when the recovery mapping parameters of the recovery mapping module are updated... or When an update occurs, since the restoration mapping parameters participate in the generation of all unified document augmentation vectors, the unified document augmentation vectors of all original text blocks in the vector database are regenerated. Through the above feedback update, the system uses newly added positive and negative samples to adjust the compressed semantic representation on the document side, so that subsequent queries are closer to the confirmed positive sample text blocks and farther away from the confirmed negative sample text blocks. In this embodiment, five types of augmentation information are compressed into the joint semantic code, and only one D-dimensional unified document augmentation vector is established for each original text block, which not only retains the information of summary, keywords, entities, parametric and semantic tags, but also avoids establishing vector indexes for the five types of augmentation elements separately.

[0105] The present invention also provides a computer-readable storage medium storing a computer program adapted for loading by a processor to perform the steps of the vector-enhanced retrieval question-answering method as described in any of the preceding claims.

[0106] The present invention also includes a computer device, comprising a memory and a processor; The memory contains computer programs; The processor executes the steps of the vector-based question-answering method, as described above, by calling computer programs stored in memory.

[0107] The above embodiments only illustrate preferred embodiments of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. Therefore, all equivalent transformations and modifications made with respect to the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A retrieval and question answering method based on vector augmentation, characterized in that, Includes the following steps: S10, the source document is divided into blocks to generate a set of original text blocks, and five types of document-side enhancement elements corresponding to the five semantic dimensions of the original text blocks, namely, summary, keywords, entities, parameters and semantic tags; the five types of document-side enhancement elements are converted into five types of compressed semantic codes according to the preset total compression dimension using five types of semantic compression units. The five types of compressed semantic codes are concatenated in a fixed order, and a semantic increment with the same basic vector dimension as the original text block is generated through the recovery mapping module. The semantic increments corresponding to the five types of compressed semantic codes are merged into a unified document enhancement vector, and a vector database is established. S20, Obtain a business request, retrieve the business request based on the vector database, obtain the original text block corresponding to the business request, and generate context; S30, input the context into a preset large language model to generate the final answer; S40, Obtain user feedback data on the final answer; S50, based on the feedback data, determine the semantic dimension to be adjusted that caused the erroneous recall, and construct an incremental training sample containing the business request, positive sample text block, and negative sample text block; use the incremental training sample to update at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted, and regenerate the unified document augmentation vector of the original text block corresponding to the feedback data to update the vector database.

2. The retrieval and question answering method based on vector augmentation according to claim 1, characterized in that, Step S10 includes: S11, using the preset large language model to extract the five types of document-side enhancement elements from the original text block; S12, input the original text block into the base vector model to obtain the base vector; use five types of semantic compression units to compress and encode the five types of document-side enhancement elements according to a preset total compression dimension, and concatenate the outputs of each semantic compression unit according to a preset order of the five semantic dimensions to obtain a joint compressed semantic code; convert the joint compressed semantic code into a semantic increment with the same dimension as the base vector through the recovery mapping module; fuse the semantic increment with the base vector to obtain the unified document enhancement vector; S13, store the unified document enhancement vector into the vector database.

3. The retrieval and question answering method based on vector augmentation according to claim 1, characterized in that, Step S20 includes: S21, The business request is converted into a query vector using the basic vector model; S22, retrieve the original text block corresponding to the business request from the vector database according to the query vector; obtain the context according to the original text block corresponding to the business request.

4. The retrieval and question answering method based on vector augmentation according to claim 3, characterized in that, S22 includes: Based on the query vector, a similarity search is performed in the vector database to obtain a preset number of unified document enhancement vectors; Based on the preset number of unified document augmentation vectors, the corresponding original text blocks are obtained by backtracking. The original text blocks obtained from the backtracking are merged and deduplicated, and then sorted and integrated according to the retrieval scores of the similarity search to obtain the context.

5. The retrieval and question answering method based on vector augmentation according to claim 1, characterized in that, Step S30 includes: S31, combine the service request and the context to obtain a prompt word; S32, input the prompt word into the preset large language model to obtain the final answer.

6. The retrieval and question answering method based on vector augmentation according to claim 1, characterized in that, Step S40 includes: S41, Collect user satisfaction ratings and usage feedback for the final answer to obtain the feedback data.

7. The retrieval and question answering method based on vector augmentation according to claim 1, characterized in that, Step S50 includes: S51, obtain the corresponding original text block according to the feedback data; divide the corresponding original text block into positive sample text block and negative sample text block; construct the incremental training sample according to the business request, the positive sample text block and the negative sample text block; determine the semantic dimension to be adjusted according to the error content in the feedback data; S52, using the incremental training samples, calculate the retrieval loss between the business request and the positive sample text block and the negative sample text block, and update at least one of the semantic compression unit parameters and recovery mapping parameters corresponding to the semantic dimension to be adjusted according to the retrieval loss; S53, using the updated semantic compression unit and recovery mapping module, regenerate the unified enhancement vector of the original text block corresponding to the feedback data, and update the vector database using the regenerated unified enhancement vector.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the steps of the vector-enhanced retrieval question-answering method as described in any one of claims 1 to 7.

9. A computer device, characterized in that, Including memory and processor; The memory contains computer programs; The processor executes the steps of the vector-enhanced retrieval question-answering method as described in any one of claims 1 to 7 by invoking the computer program stored in the memory.

Images