Retrieval enhancement generation method for recalling in combination with article abstract

Abstract

The invention provides a retrieval enhancement generation method for recalling in combination with an article abstract, and relates to the field of information retrieval. The method comprises the following steps: dividing an article to obtain a plurality of semantic block paragraphs; generating an article abstract of the article and paragraph abstracts of all semantic block paragraphs in the article; obtaining a retrieval input text; based on a first similarity value between the retrieval input text and the article abstract, recalling an article corresponding to the article abstract; based on a second similarity value between the retrieval input text and a paragraph abstract in the article, recalling a semantic block paragraph corresponding to the paragraph abstract; and inputting the retrieval input text, the article and the semantic chunk paragraph into a retrieval result generation module to generate an enhanced retrieval result.

Description

Technical Field

[0001] The present invention relates to the field of information retrieval, and in particular to a retrieval enhancement generation method for recalling articles in combination with article abstracts. Background Art

[0002] The existing RAG (Retrieval-Augmented Generation) technology combines the retrieval module with the generation model, making full use of the external knowledge base and generating the model by retrieving relevant articles as context input, thereby improving the question-answering performance.

[0003] However, the recall module in the existing RAG system still has the following deficiencies:

[0004] 1. Redundant information interference. Existing recall methods often directly retrieve the full text of the article, which leads to the introduction of a large amount of irrelevant information interference and reduces the effectiveness of the context.

[0005] 2. Insufficient processing capabilities for long articles. For long articles, existing retrieval methods have difficulty accurately locating key information and may overlook content that is crucial to answering questions, thereby reducing effectiveness due to the dispersion of key content or excessively long context.

[0006] 3. It's difficult to balance computational efficiency and effectiveness. In large-scale article databases, it's often difficult to achieve both efficiency and effectiveness in recall. Dense retrieval methods that directly search the full text, while capable of high semantic matching, are computationally expensive. Sparse retrieval methods, while efficient, lack recall accuracy.

[0007] 4. Insufficient utilization of summary information. Although the technology for generating summaries is relatively mature, summary content is mostly used to assist reading or quick understanding, and has not been fully integrated into the recall strategy of the RAG system, thus failing to realize its potential in information concentration and relevance improvement.

[0008] In the prior art, such as the Chinese invention patent with publication number CN117573843A, a medical auxiliary question-answering method and system based on knowledge calibration and retrieval enhancement is disclosed, including: creating a knowledge base by combining medical expertise and hospital information, creating keywords for the knowledge base, and dividing the documents in the knowledge base into fragments, extracting summaries of each document fragment, and then using the extracted key information as nodes to create a tree-like index for each document in the knowledge base in a bottom-up manner; however, the above-mentioned comparative document does not disclose the use of article summaries for preliminary recall to condense the core information in the article. Summary of the Invention

[0009] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a retrieval enhancement generation method for recalling articles in combination with article abstracts, which can improve the recall accuracy and reduce the impact of article length on system efficiency.

[0010] To achieve the above-mentioned purpose, the present invention provides a retrieval enhancement generation method for recalling articles in combination with article abstracts, the steps of which are as follows:

[0011] Divide the article into multiple semantic block paragraphs;

[0012] Generate an article summary of the article and paragraph summaries of all semantic block paragraphs in the article;

[0013] Get the search input text;

[0014] Recalling an article corresponding to the article abstract based on a first similarity value between the search input text and the article abstract;

[0015] Recalling a semantic block paragraph corresponding to the paragraph abstract based on a second similarity value between the retrieval input text and the paragraph abstract in the article;

[0016] The search input text, the article and the semantic block paragraph are all input into a search result generation module to generate enhanced search results.

[0017] According to a technical solution of the present invention, the process of dividing an article into multiple semantic block paragraphs is as follows:

[0018] Based on the sliding window method, the sentences in the article are traversed by sliding with a preset window size and sliding step size to obtain multiple text windows;

[0019] Calculating semantic similarity between adjacent text windows; and when the semantic similarity is greater than or equal to a preset semantic similarity threshold, merging the contents of the adjacent text windows into the same semantic block paragraph and merging the adjacent text windows into one text window; or when the semantic similarity is less than a preset similarity threshold, using the contents of the latter text window as the beginning of a new semantic block paragraph;

[0020] All semantic block paragraphs are converted into vector form and stored.

[0021] According to a technical solution of the present invention, it also includes:

[0022] The length of the merged semantic block paragraph is calculated. When the length is greater than a length threshold, the merge is cancelled and the content of the next text window is used as the start of a new semantic block paragraph.

[0023] According to a technical solution of the present invention, the generation of the article summary and the paragraph summaries of all semantic block paragraphs in the article is performed in the preprocessing stage, and the specific process is as follows:

[0024] Scan the article at a set scanning cycle to determine whether the article and the semantic block paragraphs within the article have summaries;

[0025] If the article and / or the semantic block paragraph in the article does not have a summary, generating an article summary of the article and / or a paragraph summary of the semantic block paragraph in the article;

[0026] The article abstract and paragraph abstract are converted into vector form and stored.

[0027] According to a technical solution of the present invention, based on a first similarity value between a search input text and an article abstract, a process of recalling an article corresponding to the article abstract is as follows:

[0028] Converting the search input text into a vector form;

[0029] Calculating a first similarity value between the search input text and the article abstract by using a hybrid similarity algorithm combining cosine similarity and Manhattan distance exponential decay;

[0030] Recall the articles corresponding to the abstracts of the articles to be recalled;

[0031] The article abstract to be recalled is an article abstract having a first similarity value with the search input text that is greater than a first similarity threshold.

[0032] According to a technical solution of the present invention, based on the second similarity value between the retrieved input text and the paragraph summary in the article, the process of recalling the semantic block paragraph corresponding to the paragraph summary is as follows:

[0033] Calculating a second similarity value between the retrieved input text and the paragraph summary by using a hybrid similarity algorithm combining cosine similarity and Manhattan distance exponential decay;

[0034] Recall the semantic block paragraph corresponding to the summary of the paragraph to be recalled;

[0035] The paragraph summary to be recalled is a paragraph summary whose second similarity value with the search input text is greater than a second similarity threshold.

[0036] According to a technical solution of the present invention, it also includes:

[0037] Based on the maximum parallel number allowed by the hardware for retrieval enhancement generation, the service response time threshold, the single processing time coefficient, and the current system load, the number of semantic block paragraphs allowed by the hardware is calculated;

[0038] Comparing the number of semantic block paragraphs allowed by the hardware with the number of semantic block paragraphs to be recalled, and taking the smaller value as the number of candidate semantic block paragraphs;

[0039] The number of semantic block paragraphs to be recalled is: the number of semantic block paragraphs corresponding to the summary of the paragraph to be recalled;

[0040] According to the descending sorting of the second similarities, semantic block paragraphs that meet the number of candidate semantic block paragraphs are selected as candidate semantic block paragraphs.

[0041] According to a technical solution of the present invention, the step of rearranging the candidate semantic block paragraphs is also included:

[0042] Based on the cross encoder, the similarity scores between the search input text and each candidate semantic block paragraph are calculated based on the search input text, the candidate semantic block paragraph, and all the semantic block paragraphs in the article where the candidate semantic block paragraph is located. The candidate semantic block paragraphs are then sorted in descending order according to the similarity scores to obtain a candidate semantic block paragraph list.

[0043] Calculating the mean μ and standard deviation σ of all candidate semantic block paragraphs in the candidate semantic block paragraph list;

[0044] Retain the candidate semantic block paragraphs with scores greater than μ+γσ in the candidate semantic block paragraph list;

[0045] Where γ∈[0.5,1.5].

[0046] The present invention also provides an electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the above-mentioned retrieval enhancement generation method for recalling in combination with article abstracts.

[0047] The present invention also provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the above-mentioned retrieval enhancement generation method for recalling in combination with article abstracts is implemented.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] This paper proposes a retrieval enhancement generation method that combines article abstracts with recall. By incorporating high-quality article abstracts into the recall process, this method leverages the information condensation provided by article abstracts and efficiently matches abstracts with questions. This significantly improves recall accuracy, enhances the relevance and effectiveness of recall results, and reduces the impact of article length on system efficiency.

[0050] In addition, by dynamically combining full-text and summary information, this method achieves a balance between retrieval efficiency and answer quality, providing more accurate contextual input for the generative question-answering system, thereby improving the overall performance of the system.

[0051] Specifically, the present invention has achieved remarkable results in the following aspects:

[0052] 1. Recall relevance is significantly improved. By using article summaries for preliminary recall, the core information in the article is condensed, making the recall results more focused on content relevant to the question. This effectively reduces the interference of irrelevant information and significantly improves the accuracy of recall and the effectiveness of context.

[0053] 2. Enhanced processing capabilities for long articles. This invention extracts key information from long articles through summaries, ensuring that the core content is accurately retrieved and delivered, thereby improving the performance of the question-answering system in long article scenarios.

[0054] 3. Optimize retrieval efficiency and computational cost. By prioritizing abstract matching, computational complexity is significantly reduced. In addition, by dynamically integrating abstracts and full-text information, a balance between efficiency and effectiveness is achieved, enabling the system to operate efficiently in large-scale article libraries.

[0055] 4. Adaptable to diverse application scenarios, the method of the present invention performs well in scenarios such as intelligent question answering, knowledge retrieval, and customer service. It has wide applicability, especially in complex tasks that require processing large-scale article libraries or long articles. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0057] Figure 1 The figure schematically shows the principle diagram of the retrieval enhancement generation method for recalling article abstracts according to one embodiment of the present invention. DETAILED DESCRIPTION

[0058] The description of the embodiments in this specification should be combined with the corresponding drawings, which should be considered a complete part of this specification. In the drawings, the shapes and thicknesses of the embodiments may be exaggerated and indicated for simplicity or convenience. Furthermore, the various structural components in the drawings will be described separately. It is worth noting that components not shown in the drawings or not described in words are known to those of ordinary skill in the art.

[0059] The description of the embodiments herein and any references to directions and orientations are for ease of description only and are not to be construed as limiting the scope of the present invention. The following description of the preferred embodiments may involve combinations of features, which may exist independently or in combination. The present invention is not specifically limited to the preferred embodiments. The scope of the present invention is defined by the claims.

[0060] like Figure 1 As shown, the present invention provides a retrieval enhancement generation method for recalling article abstracts, and the steps are as follows:

[0061] S1. Divide the article into multiple semantic block paragraphs;

[0062] S2. Generate an article summary and paragraph summaries of all semantic block paragraphs in the article; and associate the article summary with the paragraph summaries of the semantic block paragraphs in the article;

[0063] S3. Obtain search input text;

[0064] S4. Recalling articles corresponding to the article abstract based on a first similarity value between the retrieval input text and the article abstract;

[0065] S5. Recalling a semantic block paragraph corresponding to the paragraph summary based on a second similarity value between the retrieved input text and the paragraph summary in the article;

[0066] S6. Input the search input text, article and semantic block paragraphs into the search result generation module to generate enhanced search results.

[0067] In this embodiment, the article is first divided into paragraphs based on the text semantics, and related sentences are aggregated using a sliding window method to obtain multiple semantic block paragraphs.

[0068] Then, in the offline data preprocessing stage, that is, before providing online services, for articles without summaries or whose summary quality does not meet the requirements (such as insufficient word count or incoherent sentences, etc.), structured summaries can be generated by aggregating semantic blocks layer by layer, paragraph -> chapter -> article.

[0069] The search input text can be a question raised by the user, or other forms of relevant search information, such as keywords, query statements, etc.

[0070] Based on the first similarity value between the search input text and the article abstract, it usually means sorting the first similarity values ​​in descending order and selecting multiple article abstracts with higher order, or multiple article abstracts whose first similarity values ​​with the search input text are greater than a first similarity threshold.

[0071] Based on the second similarity value between the retrieval input text and the paragraph abstracts in the article, it usually refers to sorting the second similarity values ​​in descending order and selecting multiple paragraph abstracts with higher rankings, or multiple paragraph abstracts whose second similarity values ​​with the retrieval input text are greater than a second similarity threshold.

[0072] The retrieval result generation module can be an answer generation module, a content generation module, or a knowledge graph construction module. Based on the retrieval input text, articles, and semantic block paragraphs, the corresponding retrieval results are generated and output to the user, which is the result of retrieval enhancement generation.

[0073] This implementation utilizes a retrieval-enhanced generation method for recalling articles using abstracts. By innovatively incorporating article abstracts, this method significantly improves the performance of the recall module and optimizes the overall performance of the generative question-answering system. By combining abstract information with article paragraphs, this method enhances the recall quality and efficiency of intelligent question-answering systems, supplementing some paragraphs that conventional recall methods fail to capture. This provides efficient and reliable technical support for the application of intelligent question-answering systems in complex task scenarios.

[0074] In S1, the process of dividing the article into multiple semantic block paragraphs is as follows:

[0075] Based on the sliding window method, the sentences in the article are traversed with a preset window size and sliding step size to obtain multiple text windows;

[0076] Calculating the semantic similarity between adjacent text windows; and merging the contents of adjacent text windows into the same semantic block paragraph when the semantic similarity value is greater than or equal to a preset semantic similarity threshold, and merging the adjacent text windows into one text window; or, when the semantic similarity value is less than a preset similarity threshold, using the content of the latter text window as the start of a new semantic block paragraph;

[0077] All semantic block paragraphs are converted into vector form and stored.

[0078] In this implementation, the article is segmented based on text semantics, and related sentences are aggregated using a sliding window approach. A fixed-size text window (e.g., 5 sentences) and sliding step size (e.g., 2 sentences) are preset. The semantic similarity of adjacent text windows is calculated to determine whether to merge them. If the similarity between adjacent windows exceeds a threshold (e.g., 0.85), they are merged into the same semantic block paragraph; otherwise, a new semantic block paragraph is started.

[0079] The above method can effectively control the granularity of semantic block paragraphs while preserving contextual coherence, adapt to subsequent vectorization processing, and ensure the semantic integrity of each semantic block paragraph.

[0080] For paragraphs containing subheadings, the subheadings are combined to supplement the paragraph content. Subsequently, an appropriate vector model (such as BGE or BAAI General Embedding) is used to convert the semantic block paragraphs into embedded vectors or multi-dimensional vectors. These are stored in a vector library and associated with metadata such as the article title and paragraph location to support efficient retrieval.

[0081] In the above process, the rule matching method is used to quickly filter low-quality texts (such as incoherent sentences, etc.), and finally output a structured vectorized article collection.

[0082] S1 also includes:

[0083] The length of the merged semantic block paragraph is calculated. When the length is greater than a length threshold, the merge is canceled, and the content of the next text window is used as the start of a new semantic block paragraph.

[0084] In this embodiment, the article paragraphs are divided based on the text semantics, and related sentences are aggregated in a sliding window manner. At the same time, the length of the semantic block paragraphs can be limited by combining the maximum length limit (length threshold, such as 512 tokens) to avoid generating overly long semantic block paragraphs.

[0085] In S2, the article summary and the paragraph summaries of all semantic block paragraphs in the article are generated in the preprocessing stage. The specific process is as follows:

[0086] Scan the article at a set scanning cycle to determine whether the article and the semantic block paragraphs within the article have summaries;

[0087] If the article and / or the semantic block paragraph in the article does not have a summary, generating an article summary of the article and / or a paragraph summary of the semantic block paragraph in the article;

[0088] Convert article summaries and paragraph summaries into vector form and store them.

[0089] In this embodiment, during the offline data preprocessing stage, the (RAG) system automatically scans each unprocessed article in the article library. For articles without abstracts or whose abstracts do not meet the quality requirements (such as insufficient word count or incoherent sentences), a hierarchical abstract generation strategy is used to process them:

[0090] That is, commercial large models are used in general scenarios, and fine-tuned models in related fields are called in professional fields. Structured summaries are generated in a layer-by-layer aggregation manner of paragraph->chapter->article. All generated summaries must pass rule verification to ensure quality. Articles with existing compliant summaries directly enter the subsequent process, and then use vectorization models such as BGE to convert the summaries into multi-dimensional vectors / embedded vectors, which are stored together with the article metadata in vector databases such as Milvus, and optimized indexes are established to support efficient retrieval. This process is executed periodically through offline batch processing, which is completely isolated from online services, ensuring the high performance and stability of online retrieval services. At the same time, it supports incremental update mechanisms to handle new or modified articles.

[0091] In S4, based on the first similarity value between the retrieval input text and the article abstract, the process of recalling the article corresponding to the article abstract is as follows:

[0092] Convert the search input text into vector form;

[0093] The first similarity value between the retrieval input text and the article abstract is calculated by combining the cosine similarity and the Manhattan distance exponential decay hybrid similarity algorithm;

[0094] Recall the articles corresponding to the abstracts of the articles to be recalled;

[0095] The article abstract to be recalled is an article abstract having a first similarity value with the search input text greater than a first similarity threshold.

[0096] In this embodiment, in the question-answering task, the article abstract is matched with the question (retrieval input text).

[0097] First, the search input text is converted into a multi-dimensional vector through a vectorization model (such as BGE), and then a hybrid similarity algorithm is used for recall. The above algorithm combines cosine similarity with an exponential decay term based on Manhattan distance through linear weighting, using the formula:

[0098] S=α·cos(A,B)+(1-α)·exp(-β||AB||1)

[0099] Among them, α∈[0.6,0.8] controls the semantic matching weight, β∈[0.1,0.3] adjusts the sensitivity of local features, A and B are the vector representations of the retrieved input text and article abstract respectively), which not only retains the ability of cosine similarity to capture global semantics, but also enhances the sensitivity to key term differences through the L1 norm.

[0100] In S5, based on the second similarity value between the retrieved input text and the paragraph summary in the article, the process of recalling the semantic block paragraph corresponding to the paragraph summary is as follows:

[0101] The second similarity value between the retrieved input text and the paragraph summary is calculated by combining a hybrid similarity algorithm of cosine similarity and Manhattan distance exponential decay;

[0102] Recall the semantic block paragraph corresponding to the summary of the paragraph to be recalled;

[0103] The paragraph summary to be recalled is a paragraph summary whose second similarity value with the retrieved input text is greater than a second similarity threshold.

[0104] In this embodiment, based on the candidate articles condensed by summarization in the previous step, paragraphs in the candidate articles are recalled based on vectors. The vector matching formula can also use the hybrid similarity calculation method in the previous embodiment to select the semantic block paragraphs or information fragments most relevant to the question from the article, ensuring the completeness and accuracy of the recall results.

[0105] S5 also includes:

[0106] Based on the maximum parallel number allowed by the hardware for retrieval enhancement generation, the service response time threshold, the single processing time coefficient, and the current system load, the number of semantic block paragraphs allowed by the hardware is calculated;

[0107] Compare the number of semantic block paragraphs allowed by the hardware with the number of semantic block paragraphs to be recalled, and use the smaller value as the candidate semantic block paragraph number;

[0108] The number of semantic block paragraphs to be recalled is: the number of semantic block paragraphs corresponding to the summary of the paragraph to be recalled;

[0109] According to the descending sorting of the second similarities, semantic block paragraphs that meet the number of candidate semantic block paragraphs are selected as candidate semantic block paragraphs.

[0110] In this implementation, the semantic block paragraphs or information fragments most relevant to the question are selected from the article. At the same time, combined with the actual hardware limitations and the QPS requirements of the service, the number of retrieved paragraphs (the number of candidate semantic block paragraphs) is calculated as follows: N = min(N_max, (QPS·T) / k), where N_max is the maximum parallel number allowed by the hardware (such as the GPU memory limitation), T is the service response time threshold, k is the single processing time coefficient (measured k≈0.2ms / vector), and QPS is the current system load.

[0111] Before S5 ends and S6 is executed, the following step is also included:

[0112] Based on the cross encoder, the similarity score between the search input text and each candidate semantic block paragraph is calculated based on the search input text, the candidate semantic block paragraph, and all the semantic block paragraphs in the article where the candidate semantic block paragraph is located. The candidate semantic block paragraphs are then sorted in descending order according to the similarity score to obtain a candidate semantic block paragraph list.

[0113] Calculate the mean μ and standard deviation σ of all candidate semantic block paragraphs in the candidate semantic block paragraph list;

[0114] Retain the candidate semantic block paragraphs whose scores are greater than μ+γσ in the candidate semantic block paragraph list;

[0115] Where γ∈[0.5,1.5].

[0116] In this implementation, the recalled semantic block paragraphs are re-arranged by relevance through a cross encoder (such as DeBERTa-v3), and the candidate semantic block paragraphs are combined with the full-text fragments of the article in which the candidate semantic block paragraphs are located (the corresponding article abstract and paragraph abstract can also be added). They are then sorted by relevance with the retrieval input text (such as questions), and the candidate semantic block paragraphs related to the retrieval input text are retained.

[0117] Then, an intelligent truncation strategy based on statistical distribution is adopted: that is, the mean μ and standard deviation σ of the scores of all candidate semantic block paragraphs are calculated, and paragraphs with scores greater than μ+γσ are retained (γ∈[0.5,1.5] is configurable, and the default value is γ=1). At the same time, at least min(3,N)(N≥3) candidate semantic block paragraphs are retained, and finally a quality-controlled article set is formed and sent to the subsequent answer generation module.

[0118] The solution in this embodiment improves the recall accuracy by 12.7% on the test set compared to the existing top-K method, while keeping the increase in computational time at no more than 15%.

[0119] The present invention discloses a retrieval enhancement generation method for recalling articles in combination with article abstracts, the method comprising: dividing an article to obtain a plurality of semantic block paragraphs; generating an article abstract of the article and paragraph abstracts of all semantic block paragraphs in the article; obtaining a retrieval input text; based on a first similarity value between the retrieval input text and the article abstract, recalling the article corresponding to the article abstract; based on a second similarity value between the retrieval input text and the paragraph abstracts in the article, recalling the semantic block paragraphs corresponding to the paragraph abstract; and inputting the retrieval input text, the article, and the semantic block paragraphs into a retrieval result generation module to generate enhanced retrieval results.

[0120] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0121] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0123] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0124] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A retrieval enhancement generation method for recalling article abstracts, characterized in that: Here are the steps: Divide the article into multiple semantic block paragraphs; Generate an article summary of the article and paragraph summaries of all semantic block paragraphs in the article; Get the search input text; Recalling an article corresponding to the article abstract based on a first similarity value between the search input text and the article abstract; Recalling a semantic block paragraph corresponding to the paragraph abstract based on a second similarity value between the retrieval input text and the paragraph abstract in the article; The search input text, the article and the semantic block paragraph are all input into a search result generation module to generate enhanced search results.

2. The retrieval enhancement generation method for recalling combined with article abstracts according to claim 1 is characterized in that: The process of dividing an article into multiple semantic block paragraphs is as follows: Based on the sliding window method, the sentences in the article are traversed by sliding with a preset window size and sliding step size to obtain multiple text windows; Calculating semantic similarity between adjacent text windows; and when the semantic similarity is greater than or equal to a preset semantic similarity threshold, merging the contents of the adjacent text windows into the same semantic block paragraph and merging the adjacent text windows into one text window; or when the semantic similarity is less than a preset similarity threshold, using the contents of the latter text window as the beginning of a new semantic block paragraph; All semantic block paragraphs are converted into vector form and stored.

3. The retrieval enhancement generation method for recalling articles in combination with article abstracts according to claim 2 is characterized in that: Also includes: The length of the merged semantic block paragraph is calculated. When the length is greater than a length threshold, the merge is cancelled and the content of the next text window is used as the start of a new semantic block paragraph.

4. The retrieval enhancement generation method for recalling articles in combination with article abstracts according to any one of claims 1 to 3, characterized in that: Generating the article summary of the article and the paragraph summaries of all semantic block paragraphs in the article is performed in the preprocessing stage. The specific process is as follows: Scan the article at a set scanning cycle to determine whether the article and the semantic block paragraphs within the article have summaries; If the article and / or the semantic block paragraph in the article does not have a summary, generating an article summary of the article and / or a paragraph summary of the semantic block paragraph in the article; The article abstract and paragraph abstract are converted into vector form and stored.

5. The retrieval enhancement generation method for recalling combined with article abstracts according to claim 4 is characterized in that: Based on the first similarity value between the retrieval input text and the article abstract, the process of recalling the article corresponding to the article abstract is as follows: Converting the search input text into a vector form; Calculating a first similarity value between the search input text and the article abstract by using a hybrid similarity algorithm combining cosine similarity and Manhattan distance exponential decay; Recall the articles corresponding to the abstracts of the articles to be recalled; The article abstract to be recalled is an article abstract having a first similarity value with the search input text that is greater than a first similarity threshold.

6. The retrieval enhancement generation method for recalling articles in combination with article abstracts according to claim 5 is characterized in that: Based on the second similarity value between the retrieved input text and the paragraph summary in the article, the process of recalling the semantic block paragraph corresponding to the paragraph summary is as follows: Calculating a second similarity value between the retrieved input text and the paragraph summary by using a hybrid similarity algorithm combining cosine similarity and Manhattan distance exponential decay; Recall the semantic block paragraph corresponding to the summary of the paragraph to be recalled; The paragraph summary to be recalled is a paragraph summary whose second similarity value with the search input text is greater than a second similarity threshold.

7. The retrieval enhancement generation method for recalling combined with article abstracts according to claim 6 is characterized in that: Also includes: Based on the maximum parallel number allowed by the hardware for retrieval enhancement generation, the service response time threshold, the single processing time coefficient, and the current system load, the number of semantic block paragraphs allowed by the hardware is calculated; Comparing the number of semantic block paragraphs allowed by the hardware with the number of semantic block paragraphs to be recalled, and taking the smaller value as the number of candidate semantic block paragraphs; The number of semantic block paragraphs to be recalled is: the number of semantic block paragraphs corresponding to the summary of the paragraph to be recalled; According to the descending sorting of the second similarities, semantic block paragraphs that meet the number of candidate semantic block paragraphs are selected as candidate semantic block paragraphs.

8. The retrieval enhancement generation method for recalling combined with article abstracts according to claim 7 is characterized in that: It also includes the step of rearranging the candidate semantic block paragraphs: Based on the cross encoder, the similarity scores between the search input text and each candidate semantic block paragraph are calculated based on the search input text, the candidate semantic block paragraph, and all the semantic block paragraphs in the article where the candidate semantic block paragraph is located. The candidate semantic block paragraphs are then sorted in descending order according to the similarity scores to obtain a candidate semantic block paragraph list. Calculating the mean μ and standard deviation σ of all candidate semantic block paragraphs in the candidate semantic block paragraph list; Retain the candidate semantic block paragraphs with scores greater than μ+γσ in the candidate semantic block paragraph list; Where γ∈[0.5,1.5].

9. An electronic device, characterized in that: include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to enable the electronic device to perform the retrieval enhancement generation method for recalling in combination with article abstracts as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, implement the retrieval enhancement generation method for recalling in combination with article abstracts as described in any one of claims 1 to 8.

Images