An enterprise data asset retrieval method based on multi-layer hybrid retrieval enhancement
Patent Information
- Application Number
- CN202610989107.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-03
- Publication Date
- 2026-10-16
AI Technical Summary
[0005]本发明的主要目的在于提供一种基于多层混合检索增强的企业数据资产检索方法，以解决现有单一检索方式难以在同一检索过程中同时兼顾检索结果的覆盖度与相关性精度，从而导致检索结果数量不足或相关性偏低的问题
本发明通过对查询文本依次在文章标题、文章摘要以及文章切片三个层级上展开检索，并以各层级检索所得文章的数量是否达到预设数量作为是否继续向下一层级递进的判断依据，使得在较粗粒度的检索即可满足结果数量要求时，无需对全部文档进行细粒度的向量计算，从而在保证检索响应效率的同时控制了计算开销。随着检索层级由文章标题向文章摘要、文章切片逐层推进，文档的匹配依据由标题扩展至经语言模型提炼的摘要语义以及文章内部的局部语义切片，并在最细粒度的切片层级上将关键字检索与语义检索的结果加以融合，使得在标题字面难以命中或文档篇幅较长的情形下，仍能够召回与查询真正相关的文档，提高了检索结果的覆盖度。在每一层级的递进过程中，对各层级的检索结果按相关性降序合并，并依据文章的唯一标识符去除重复，避免了结果的重复与冗余，使得最终输出的文档在数量得到补足的同时仍保持相关性由高至低的有序排列，从而在同一检索过程中兼顾了检索结果的覆盖度与相关性精度。
Smart Images
Figure CN122925614A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data retrieval technology, and in particular to a method for retrieving enterprise data assets based on multi-layered hybrid retrieval enhancement. Background Technology
[0002] With the continuous advancement of enterprise informatization, the data assets accumulated within enterprises, such as documents, reports, and knowledge entries, are showing a trend of large-scale and diversified growth. Users expect to quickly and accurately obtain content relevant to their needs from massive amounts of documents. As a bridge connecting user needs and enterprise data assets, the document retrieval system directly affects the efficiency and quality of information acquisition for users through the coverage and relevance accuracy of its search results.
[0003] Existing document retrieval systems typically employ a single retrieval method to find documents. One approach involves literal matching based on keywords in the article title or text. This method is computationally simple and fast, but it often fails to find truly relevant documents when faced with queries that are varied in expression, contain numerous technical terms, or have semantic variations, resulting in insufficient results. Another approach uses vectorization techniques for semantic matching. While this method can understand the query intent to some extent, performing fine-grained semantic calculations on all documents is costly. Furthermore, when documents are lengthy, single-granularity matching struggles to simultaneously capture both the overall theme and local details, thus limiting the relevance and accuracy of the search results.
[0004] It is evident that, whether relying solely on literal keyword matching or semantic matching, a single retrieval method cannot simultaneously guarantee sufficient coverage and maintain high relevance accuracy in the same retrieval process. This results in insufficient number of retrieval results or low relevance, affecting the effectiveness of users obtaining the information they need from the enterprise's data assets. Summary of the Invention
[0005] The main objective of this invention is to provide a multi-layered hybrid retrieval enhancement method for enterprise data assets, in order to solve the problem that existing single retrieval methods cannot simultaneously take into account the coverage and relevance accuracy of retrieval results in the same retrieval process, resulting in insufficient number of retrieval results or low relevance.
[0006] To achieve the above objectives, this invention provides a multi-level hybrid retrieval enhancement method for enterprise data assets. The method includes: receiving query text input by a user; performing word segmentation on the query text to obtain a keyword sequence; performing keyword retrieval based on article titles in a preset article library; filtering articles whose relevance meets a preset threshold; when the number of retrieved articles reaches a preset number N, outputting the top N articles ranked from highest to lowest relevance; when the number of retrieved articles is less than the preset number N, proceeding to the next level of retrieval; generating a summary for each article in the preset article library using a language model; performing keyword retrieval on the summaries; filtering articles whose relevance meets a preset threshold; merging the keyword retrieval results based on article titles and the keyword retrieval results based on the summaries, ranked from highest to lowest relevance, and removing duplicates; when the number of merged articles reaches a preset number N, outputting the top N articles ranked from highest to lowest relevance. If the number of merged articles is less than a preset number N, the search proceeds to the next level. Each article in the preset article database is sliced to obtain several slice units. High-dimensional vector representations are generated for each slice unit, and an index is constructed and stored. Keyword and semantic searches based on slice units are performed on the query text. The keyword search results and semantic search results based on slice units are merged, and articles whose comprehensive relevance meets a preset threshold are selected. The keyword search results based on the article title, the keyword search results based on the abstract, and the search results based on the slice units are sorted from high to low relevance, merged, and duplicates are removed. When the number of merged articles reaches a preset number N, the top N articles are output in descending order of relevance. When the number of merged articles is less than the preset number N, all articles are output. The structured information of the finally determined articles is output, so that the matching granularity can be dynamically adjusted according to the difficulty of the query.
[0007] As a preferred technical solution of the present invention, the step of segmenting the query text to obtain a keyword sequence, and performing keyword retrieval based on the article title in a preset article library to filter out articles whose relevance meets a preset threshold includes: segmenting the query text using a Chinese word segmentation tool to generate a keyword sequence; constructing a text vector space model using the TF-IDF algorithm to calculate the vector similarity between the query text and the article title; and filtering out articles with similarity meeting the preset similarity threshold as keyword retrieval results based on the article title.
[0008] As a preferred embodiment of the present invention, when the number of retrieved articles reaches a preset number N, the top N articles are output in descending order of relevance. When the number of retrieved articles is less than the preset number N, the search proceeds to the next level. This includes: determining the number of articles retrieved based on keywords in the article title search, where the preset number N is 10; when the number of articles reaches 10, outputting the relevant information of the top 10 articles, including the article title, abstract, author name, date, and tags; and when the number of articles is less than 10, proceeding to keyword search based on the abstract.
[0009] As a preferred technical solution of the present invention, the step of generating a summary for each article in the preset article library using a language model, performing keyword retrieval on the summary, and filtering out articles whose relevance meets a preset threshold includes: using a language model to perform content understanding and analysis on each article in the preset article library, extracting the core semantic information of the article, generating a text summary and storing it; performing word segmentation on each generated summary to generate a keyword sequence for the summary; receiving the query text, performing word segmentation on the query text to generate a keyword sequence; using the TF-IDF algorithm to construct a text vector space model, calculating the vector similarity between the query text and the summary, and filtering out articles with similarity meeting the preset similarity threshold as keyword retrieval results based on the summary.
[0010] As a preferred technical solution of the present invention, the step of merging the keyword search results based on the article title and the keyword search results based on the abstract according to their relevance from high to low, removing duplicates, and when the number of merged articles reaches a preset number N, outputting the top N articles in order of relevance from high to low, and when the number of merged articles is less than the preset number N, proceeding to the next level of retrieval, includes: arranging the keyword search results based on the article title and the keyword search results based on the abstract according to their relevance from high to low; merging the two levels of search results and removing duplicate articles based on the unique identifier of the articles; when the number of merged articles reaches 10, outputting the relevant information of the top 10 articles; when the number of merged articles is less than 10, proceeding to the retrieval based on slice units.
[0011] As a preferred embodiment of the present invention, the step of slicing each article in the preset article library to obtain several slice units, generating high-dimensional vector representations for the slice units, and constructing an index for storage includes: based on document structure features and semantic integrity, using a preset slicing strategy to cut each article in the preset article library into multiple slice units with relatively independent semantics; using a vector embedding model to encode each slice unit to generate a corresponding high-dimensional dense vector representation; storing the high-dimensional dense vector representations in a vector database, and constructing a nearest neighbor search index for the high-dimensional dense vector representations.
[0012] As a preferred technical solution of the present invention, the step of performing keyword retrieval and semantic retrieval based on slice units on the query text, and then filtering out articles whose comprehensive relevance meets a preset threshold after fusing the keyword retrieval results and semantic retrieval results based on slice units, includes: performing word segmentation on the query text and vectorizing the query text using the vector embedding model to generate a query vector; performing keyword retrieval and calculating similarity based on the query text and the slice units, and retrieving slices whose similarity meets a preset threshold as first candidate slices; calculating the semantic similarity between the query vector and the vector representation of the slice units, and filtering out slices whose semantic similarity meets a preset threshold from the slice units ranked first in semantic similarity as second candidate slices; using a weighted summation fusion strategy to calculate a comprehensive relevance score for each slice unit in the first candidate slice and the second candidate slice; sorting the slice units in descending order according to the comprehensive relevance score, and using the articles corresponding to slice units whose comprehensive relevance scores are higher than the preset threshold as the retrieval results based on slice units.
[0013] As a preferred embodiment of the present invention, the step of employing a weighted summation fusion strategy to calculate a comprehensive relevance score for each slice unit in the first candidate slice and the second candidate slice includes: configuring corresponding weights for keyword retrieval and semantic retrieval based on slice units, wherein the sum of the weights of the keyword retrieval and the semantic retrieval is 1; and performing a weighted summation of the keyword retrieval similarity and semantic similarity of each slice unit according to the configured weights to obtain the comprehensive relevance score of the slice unit; wherein the similarity is cosine similarity.
[0014] As a preferred embodiment of the present invention, the steps of arranging the keyword search results based on the article title, the keyword search results based on the abstract, and the search results based on the slice unit in descending order of relevance, merging them, and removing duplicates, and outputting the top N articles in descending order of relevance when the number of merged articles reaches a preset number N, and outputting all articles when the number of merged articles is less than the preset number N, include: arranging the keyword search results based on the article title, the keyword search results based on the abstract, and the search results based on the slice unit in descending order of relevance; merging the three levels of search results and removing duplicate articles based on the unique identifier of the articles; outputting the relevant information of the top 10 articles when the number of merged articles reaches 10, and outputting the relevant information of all articles when the number of merged articles is less than 10.
[0015] As a preferred embodiment of the present invention, the output of the structured information of the final determined article includes: outputting relevant information of the final determined article, wherein the relevant information includes the article title, abstract, author name, date, and tags.
[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention searches the query text sequentially at three levels: article title, article summary, and article slice. The determination to proceed to the next level is based on whether the number of articles retrieved at each level reaches a preset threshold. This allows for efficient retrieval while controlling computational overhead, as coarse-grained searches can meet the result quantity requirements without requiring fine-grained vector calculations on all documents. As the search levels progress from article title to article summary and then to article slice, the document matching criteria expand from the title to the semantics of the summary extracted by the language model and local semantic slices within the article. At the finest-grained slice level, the results of keyword retrieval and semantic retrieval are integrated. This ensures that even when the title is difficult to match or the document is lengthy, documents truly relevant to the query can still be retrieved, improving the coverage of the search results. In each level of the progressive process, the search results of each level are merged in descending order of relevance, and duplicates are removed based on the unique identifier of the article, avoiding duplication and redundancy of results. This ensures that the final output documents are not only sufficient in number but also maintain an orderly arrangement of relevance from high to low, thus balancing the coverage and relevance accuracy of search results in the same search process. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the main flow of the retrieval method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the title keyword-based retrieval process provided in an embodiment of the present invention; Figure 3 A schematic diagram of the process for abstract keyword retrieval provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the slice-based hybrid retrieval process provided in an embodiment of the present invention. Detailed Implementation
[0018] To make the technical problem to be solved by the present invention, the technical solution adopted, and the technical effects that can be achieved clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention. Without conflicting with each other, the following embodiments and the technical features in the embodiments can be combined with each other.
[0019] See Figure 1 The present invention provides a multi-layered hybrid retrieval enhancement method for enterprise data assets. In its overall structure, it consists of multiple retrieval levels sequentially unfolding around the same query text. The method uses whether the number of articles retrieved at each level reaches a preset number as the criterion for whether to proceed to a finer-grained level, allowing the matching granularity to be dynamically adjusted according to the difficulty of the query. The method includes steps S1 to S5.
[0020] Step S1: Receive the query text input by the user. The query text can be one or more keywords entered by the user, or it can be a search request in natural language form. After obtaining the query text, it is used as the input object shared by subsequent search levels.
[0021] Step S2: The query text is segmented into words, and a keyword search is performed on the article titles in a preset article database to filter articles whose relevance meets a preset threshold. When the number of retrieved articles reaches a preset number N, the top N articles are output in descending order of relevance. When the number of retrieved articles is less than the preset number N, the process proceeds to the next level of the search. (See also...) Figure 2 The query text was segmented using the jieba segmentation tool. Relying on its built-in dictionary and segmentation rules, the continuous query text was divided into several keywords with actual semantic meaning. Function words and auxiliary words with limited retrieval value were filtered out, resulting in a keyword sequence. Based on this, a text vector space model was constructed using the TF-IDF algorithm. By statistically analyzing the word frequency of each keyword in the article titles and its inverse document frequency in the entire article database title set, corresponding weights were assigned to each keyword. The weight can be expressed as ,in The frequency of the keyword in the text. The number of documents containing this keyword. The total number of documents in the document library is used as the reference. Based on this, the query text is mapped to a vector for each article title, and the degree of matching is measured by the vector similarity between the two. This vector similarity can be expressed as cosine similarity, i.e., for each query vector... With title vector Their similarity Articles with a similarity threshold of 10 or higher are selected as keywords for retrieval based on their article titles. In one implementation, the preset number N is set to 10. When the number of articles meeting the criteria reaches or exceeds 10, they are sorted from highest to lowest TF-IDF score, and the top 10 articles and their related information are output. The related information may include the article title, abstract, author name, publication date, and tags. When the number of articles meeting the criteria is less than 10, the search does not end there. Instead, the search scope is further expanded from the article title, and the search automatically transitions to the keyword retrieval level based on the abstract.
[0022] Step S3: Generate a summary for each article in the preset article library using a language model. Perform keyword retrieval on the generated summaries and filter out articles whose relevance meets a preset threshold. Sort the keyword retrieval results based on article titles and summaries separately according to relevance from highest to lowest, then merge them and remove duplicates. When the number of merged articles reaches a preset number N, output the top N articles in descending order of relevance. When the number of merged articles is less than the preset number N, proceed to the next level of the retrieval process. (See also...) Figure 3Before receiving user queries, this layer preprocesses a pre-defined article database. Using a language model, it performs content understanding and analysis on each article, extracting core semantic information such as core arguments, key evidence, and main conclusions. Based on this, a concise and information-dense text summary is generated and stored in the database as an efficient information carrier for subsequent retrieval. Simultaneously, each generated summary undergoes word segmentation preprocessing, using the jieba word segmentation tool to break it down into several words, resulting in a keyword sequence reflecting the core content of the summary. Upon receiving a user's search request, the query text is segmented in the same way, removing meaningless function words and auxiliary words to obtain a keyword sequence for the query text. Then, a text vector space model is constructed using the TF-IDF algorithm, representing the keyword sequence of the query text and the keyword sequences of each summary as vectors. Each dimension of the vector corresponds to a keyword, the value of which is determined by the TF-IDF value of that keyword. The cosine similarity between the query text vector and each summary vector is calculated to quantify their correlation. Articles meeting the similarity threshold are selected based on a pre-defined similarity threshold and used as the keyword retrieval results based on the summary. The search results from the two levels are then integrated. The keyword search results based on article titles and those based on abstracts are sorted in descending order of their relevance scores and then merged. Duplicate articles are identified and removed using the unique identifier of each article to avoid redundancy. The number of articles in the merged results is then assessed. If the number reaches or exceeds 10, the top 10 articles and their related information are output. If the number is less than 10, the search scope is expanded from the article abstract to the full text of the article, and the search automatically switches to a hybrid search level based on article slices.
[0023] Step S4: Slice each article in the preset article database, generate high-dimensional vector representations for the resulting slice units, and construct an index for storage. Use a combination of slice-based keyword retrieval and semantic retrieval to select articles whose relevance meets a preset threshold. Sort the keyword retrieval results based on article title, keyword retrieval results based on abstract, and mixed retrieval results based on slices from highest to lowest relevance, then merge them and remove duplicates. When the number of merged articles reaches a preset number N, output the top N articles sorted from highest to lowest relevance. When the number of merged articles is less than the preset number N, output all articles. (See also...) Figure 4This layer first slices each article in the article library. Based on the document structure features and semantic integrity requirements of the articles, each article is divided into multiple relatively independent semantic slice units according to a preset slicing strategy. This avoids the decrease in retrieval accuracy caused by excessively long articles, enabling subsequent searches to more accurately locate and query relevant local content. For example, the slicing strategy can be a pre-set segmentation rule or a user-defined segmentation rule. For instance, articles can be segmented by setting parameters such as segment separators, segment interval lengths, or heading levels to balance the appropriate length of the slices with semantic coherence. In another implementation, a certain amount of content overlap can be set between adjacent slice units to reduce semantic fragmentation caused by improper segmentation. After slicing, each slice unit is encoded using a vector embedding model, such as bge, to transform the text information in the slice into a corresponding high-dimensional dense vector representation. This vector characterizes the semantic information of the slice and is stored in a dedicated vector database, Milvus. Milvus then builds an efficient nearest neighbor search index for these vectors, supporting subsequent fast vector similarity retrieval. When the user inputs query text, it is segmented to extract key information, and the query text is vectorized using the same vector embedding model as the slice encoding, resulting in a query vector with the same dimension as the slice vector, placing them in a comparable semantic space. Based on this, a hybrid slice-based retrieval is performed: on one hand, keyword retrieval and similarity calculation are performed between the query text and the article slices, and slices with similarity meeting a preset threshold are selected as first candidate slices; on the other hand, the semantic similarity between the query vector and the slice vector is calculated, and slices with semantic similarity exceeding a preset threshold are selected from the top-ranked slices as second candidate slices. Both methods select relevant slices from the perspectives of keyword matching and semantic association. The two sets of candidate slices are then fused using a weighted summation strategy. Weights are assigned to keyword retrieval and semantic retrieval based on their respective importance. For each candidate slice, a comprehensive relevance score is obtained by weighting and summing its keyword similarity and semantic similarity. This comprehensive relevance score can be expressed as: ,in and These are the keyword retrieval similarity and semantic similarity of the slice, respectively. and The corresponding weights and satisfying For example, take Subsequently, all candidate slices are sorted in descending order based on their comprehensive relevance scores, and articles corresponding to slices with comprehensive relevance scores higher than a preset threshold are selected to form a slice-based hybrid search result. Finally, the multi-level search results are integrated. Keyword search results based on article titles, keyword search results based on abstracts, and hybrid search results based on slices are sorted in descending order according to their respective relevance scores and then merged. Duplicate articles are removed using the unique identifier of each article. The cumulative number of merged articles is counted. When the cumulative number reaches a preset 10, the top 10 articles are selected and their relevant information is output. If the cumulative number is less than 10, the relevant information of all articles is output.
[0024] Step S5: Output the structured information of the article. For the finalized article, output its relevant information, which may include structured information such as article title, abstract, author name, date, and tags, so that users can quickly identify and obtain the required document based on the above structured information.
[0025] It should be noted that the preset quantity N is not limited to 10. It can be set to other values based on the actual needs of the system and the user's search preferences in different application scenarios. The similarity thresholds used at each level can also be set independently and adjusted according to the search results. In addition to cosine similarity, other similarity measures such as inner product or similarity calculated based on Euclidean distance can also be used in other implementations for measuring text relevance at each level. The language model used to generate article summaries, the vector embedding model used for slice encoding, and the vector database used to store vectors and build a nearest neighbor search index can all be replaced with other models or components with corresponding functions without departing from the concept of this invention. Such replacements do not affect the implementation of the method described in this invention. In the above-mentioned weighted summation-based fusion strategy, the specific values of the keyword matching weight and the semantic similarity weight can be flexibly configured according to the relative importance of keyword matching and semantic association in a specific scenario, and are not limited to a value of 0.5 for both.
[0026] Through the above steps S1 to S5, the enterprise data asset retrieval method based on multi-level hybrid retrieval enhancement provided by this embodiment of the invention takes the progressive retrieval of query text at three levels—article title, article summary, and article slice—as the main line, combines keyword matching with semantic similarity calculation, and dynamically adjusts the retrieval granularity based on whether the number of retrieval results at each level reaches a preset number, thereby taking into account both the coverage and relevance accuracy of retrieval results in the retrieval process for enterprise data assets.
[0027] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for retrieving enterprise data assets based on multi-layer hybrid retrieval enhancement, characterized in that, The method includes: Receive query text input by the user; The query text is segmented to obtain a keyword sequence. The keyword is searched in the preset article library based on the article title. Articles whose relevance meets the preset threshold are selected. When the number of retrieved articles reaches the preset number N, the top N articles are output in order of relevance from high to low. When the number of retrieved articles is less than the preset number N, the search proceeds to the next level. A language model is used to generate a summary for each article in the preset article library. Keyword retrieval is performed on the summary to filter out articles whose relevance meets a preset threshold. The keyword retrieval results based on the article title and the keyword retrieval results based on the summary are sorted from high to low relevance and then merged and duplicates are removed. When the number of merged articles reaches a preset number N, the top N articles are output in order of relevance from high to low. When the number of merged articles is less than the preset number N, the search proceeds to the next level. Each article in the preset article library is sliced to obtain several slice units. A high-dimensional vector representation is generated for the slice units and an index is constructed and stored. Keyword retrieval and semantic retrieval based on slice units are performed on the query text. The keyword retrieval results and semantic retrieval results based on slice units are merged and articles whose comprehensive relevance meets the preset threshold are selected. The keyword retrieval results based on the article title, the keyword retrieval results based on the abstract, and the retrieval results based on the slice units are sorted from high to low relevance and then merged and duplicates are removed. When the number of merged articles reaches the preset number N, the top N articles are output in order of relevance from high to low. When the number of merged articles is less than the preset number N, all articles are output. Output the final structured information of the article.
2. The method according to claim 1, characterized in that, The process involves segmenting the query text to obtain a keyword sequence, then performing a keyword search based on the article title in a pre-defined article library to filter out articles whose relevance meets a pre-defined threshold, including: The query text is segmented using a Chinese word segmentation tool to generate a keyword sequence; A text vector space model is constructed using the TF-IDF algorithm, and the vector similarity between the query text and the article title is calculated. Articles that meet the preset similarity threshold are selected as keyword search results based on their article titles.
3. The method according to claim 1, characterized in that, When the number of retrieved articles reaches a preset number N, the top N articles are output in descending order of relevance. When the number of retrieved articles is less than the preset number N, the search proceeds to the next level, including: The number of articles retrieved based on keywords in the article title is determined, and the preset number N is 10; When the number of articles reaches 10, output the relevant information of the top 10 articles, including article title, abstract, author name, date, and tags; When the number of articles is less than 10, the search will switch to keyword retrieval based on the abstract.
4. The method according to claim 1, characterized in that, The process of generating a summary for each article in the preset article library using a language model, performing keyword retrieval on the summaries, and filtering out articles whose relevance meets a preset threshold includes: The language model is used to perform content understanding and analysis on each article in the preset article library, extract the core semantic information of the article, generate a text summary and store it; Each generated abstract is segmented into words to generate a sequence of keywords for the abstract; Upon receiving the query text, the query text is segmented to generate a keyword sequence; A text vector space model is constructed using the TF-IDF algorithm. The vector similarity between the query text and the summary is calculated. Articles that meet the similarity threshold are selected as keyword search results based on the summary.
5. The method according to claim 1, characterized in that, The process involves merging the keyword search results based on article titles and keyword search results based on abstracts, sorting them by relevance from highest to lowest, and removing duplicates. When the number of merged articles reaches a preset number N, the top N articles are output in descending order of relevance. When the number of merged articles is less than the preset number N, the process proceeds to the next level of retrieval, including: The keyword search results based on the article title and the keyword search results based on the abstract are sorted from high to low relevance. The two-level search results are merged, and duplicate articles are removed based on the article's unique identifier. When the number of merged articles reaches 10, output the relevant information of the top 10 articles. When the number of merged articles is less than 10, switch to retrieval based on slice units.
6. The method according to claim 1, characterized in that, The step of slicing each article in the preset article library to obtain several slice units, generating high-dimensional vector representations for the slice units, and constructing an index for storage includes: Based on document structure features and semantic integrity, a preset slicing strategy is used to cut each article in the preset article library into multiple slice units with relatively independent semantics. Each slice unit is encoded using a vector embedding model to generate a corresponding high-dimensional dense vector representation; The high-dimensional dense vector representation is stored in a vector database, and a nearest neighbor search index is constructed for the high-dimensional dense vector representation.
7. The method according to claim 6, characterized in that, The step of performing keyword and semantic retrieval based on slice units on the query text, and then fusing the keyword retrieval results and semantic retrieval results based on slice units to filter out articles whose comprehensive relevance meets a preset threshold, includes: The query text is segmented into words, and the vector embedding model is used to vectorize the query text to generate a query vector. Based on the query text and the slice unit, keyword retrieval is performed and similarity is calculated. Slices whose similarity meets a preset threshold are retrieved as first candidate slices. Calculate the semantic similarity between the query vector and the vector representation of the slice unit, and select slices whose semantic similarity meets a preset threshold from the slice units ranked first in semantic similarity as second candidate slices; A weighted summation fusion strategy is adopted to calculate a comprehensive correlation score for each slice unit in the first candidate slice and the second candidate slice; The slice units are sorted in descending order based on the comprehensive relevance score, and the articles corresponding to the slice units with a comprehensive relevance score higher than a preset threshold are used as the search results based on the slice units.
8. The method according to claim 7, characterized in that, The weighted summation fusion strategy calculates a comprehensive relevance score for each slice unit in the first and second candidate slices, including: Assign corresponding weights to keyword retrieval and semantic retrieval based on slice units, and the sum of the weights of keyword retrieval and semantic retrieval is 1; Based on the configured weights, the keyword retrieval similarity and semantic similarity of each slice unit are weighted and summed to obtain the comprehensive relevance score of the slice unit; The similarity mentioned above uses cosine similarity.
9. The method according to claim 1, characterized in that, The process involves sorting the keyword search results based on article titles, keywords based on abstracts, and search results based on slice units according to their relevance from highest to lowest, then merging them and removing duplicates. When the number of merged articles reaches a preset number N, the top N articles are output in descending order of relevance. When the number of merged articles is less than the preset number N, all articles are output, including: The keyword search results based on the article title, the keyword search results based on the abstract, and the search results based on the slice unit are sorted from high to low relevance. The three-level search results are merged, and duplicate articles are removed based on their unique identifiers. When the number of merged articles reaches 10, output the relevant information of the top 10 articles; when the number of merged articles is less than 10, output the relevant information of all articles.
10. The method according to claim 1, characterized in that, The structured information of the final output article includes: Output the relevant information of the finalized article, including the article title, abstract, author name, date, and tags.