Intelligent text retrieval method and system, storage medium and program product

Abstract

The invention provides an intelligent text retrieval method and system, a storage medium and a program product, and relates to the field of intelligent text retrieval, the method comprises the following steps: segmenting each original document into a plurality of semantic segments according to semantic similarity; extracting a text feature of each semantic fragment to generate a text fragment vector; selecting a related fragment from each text fragment vector, wherein the first similarity between the related fragment and the retrieval vector is greater than a preset first threshold value; calculating a second similarity between each related fragment and the adjacent fragment; combining the adjacent segments with the second similarity greater than a preset second threshold value with the corresponding related segments to obtain stitching segments; based on a retrieval result corresponding to a historical retrieval request of a user, determining a retrieval granularity which is firstly displayed; and according to the retrieval granularity, adjusting the display content of the stitching fragment, and returning a text retrieval result. By implementing the method, the semantic integrity and the result relevancy of the text retrieval result can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent text retrieval, and in particular to an intelligent text retrieval method, system, storage medium and program product. Background Art

[0002] With the rapid development of information technology, human society has entered an era of data explosion. Whether it's academic papers in scientific research, case files in the legal industry, or internal technical documents and market reports, massive and complex unstructured text data has become the primary carrier of information. Against this backdrop, how to quickly and accurately retrieve the required information from this vast sea of ​​documents has become a critical bottleneck for improving decision-making efficiency and promoting innovation across all industries. Traditional text retrieval technology based on keyword matching, due to its inability to deeply understand the inherent semantics of text, has been unable to meet users' increasingly high standards for information accuracy and completeness.

[0003] In related technologies, each long document is generally divided into multiple independent, standardized text blocks according to preset rules, such as fixed character length, paragraph marks or chapter structure. Subsequently, the system uses advanced text vectorization models (such as language models based on the Transformer architecture) to calculate each text block, abstract the rich semantic information contained therein and convert it into a high-dimensional mathematical vector. When the user makes a search request in natural language, the system also converts the request into a query vector. Finally, by calculating the semantic similarity (such as cosine similarity) between the query vector and all text block vectors in the document library in the vector space, the system selects one or more text blocks that are closest to the user's intention, and presents these text blocks to the user as the final search results.

[0004] However, the underlying operating logic of the relevant technology is to regard the original document as a collection of unrelated text blocks, and its retrieval process is essentially to select those individual text blocks in this collection that are most directly relevant to the user's query. When the content of a complete discussion, a complex operational process, or a conclusion with preconditions happens to be divided and distributed in two or more consecutive text blocks at physical locations in the original document, the relevant technology will show its limitations. For example, the system may successfully detect the first text block because its semantic match with the query is extremely high, but for the second text block that follows and is used to supplement explanations, limit conditions, or explain cause and effect, the system may discard it because its direct relevance to the query is slightly lower. This processing method directly leads to the destruction of the inherent integrity of the retrieval results. Summary of the Invention

[0005] The present application provides an intelligent text retrieval method, system, storage medium and program product for addressing the problem that multiple related text fragments returned by a related retrieval system often have logical incoherence or incomplete information after being combined.

[0006] In a first aspect, the present application provides an intelligent text retrieval method, which is applied to an intelligent text retrieval system. The method comprises: Each original document is divided into multiple semantic segments according to the semantic similarity of the semantic content; Extracting text features of each semantic segment and generating a text segment vector corresponding to each semantic segment; selecting one or more relevant segments from each text segment vector whose first similarity to a search vector is greater than a preset first threshold, the search vector being converted according to a user's search request; calculating a second similarity between each of the relevant segments and an adjacent segment, wherein the adjacent segment and the relevant segment are physically continuous in the original document; Merging adjacent segments whose second similarity is greater than a preset second threshold with corresponding related segments to obtain a stitched segment; Determine the search granularity to be displayed first based on the search results corresponding to the user's historical search requests, wherein the search granularity includes summary level, paragraph level and chapter level; The display content of the stitched segments is adjusted according to the search granularity, and a text search result is returned.

[0007] Through the above embodiment, the system subdivides the original document into multiple semantic segments based on semantic similarity, and extracts text features from each semantic segment to generate a vector, thus achieving fine-grained, semantic-level modeling of the document content. On this basis, relevant segments are screened by the similarity between the retrieval vector and the segment vector, and further stitched together based on the semantic continuity of adjacent segments, ensuring the contextual integrity and semantic coherence of the retrieval results. Finally, the retrieval granularity is dynamically adjusted according to the user's historical behavior, optimizing the adaptability of the results and making the returned content both accurate and comprehensive. This solution improves the semantic integrity and result relevance of text retrieval.

[0008] In some embodiments, the step of dividing each original document into a plurality of semantic segments according to the semantic similarity of the semantic content specifically includes: Performing preliminary sentence-level segmentation on the original document to obtain a plurality of preliminary sentence units, wherein the preliminary sentence units are the smallest grammatical units obtained by dividing the original document according to sentence boundaries; Calculate the semantic similarity between each pair of adjacent preliminary sentence units to obtain the adjacent sentence similarity; The preliminary sentence units whose adjacent sentences have similarities greater than a preset threshold are merged to obtain multiple semantic segments.

[0009] Through the above embodiment, the system performs preliminary sentence-level segmentation on the original document and merges it into semantic segments based on the semantic similarity of adjacent sentences. It can take into account both syntactic boundaries and semantic continuity, effectively avoids important semantics from being divided into scattered fragments, and improves the accuracy of semantic retrieval and the integrity of the context.

[0010] In some embodiments, after the step of adjusting the display content of the stitched segment according to the search granularity and returning the text search result, the method further includes: In response to a user clicking a preset first control, constructing a document structure graph of the original document corresponding to each of the related segments, wherein each semantic segment in the original document serves as a node in the document structure graph; In each of the document structure graphs, the logical relationship between the nodes is identified based on a preset logical rule to obtain one or more logical relationship edges; Taking the node corresponding to each of the related fragments as a starting point, traversing the one or more logical relationship edges in the document structure graph corresponding to the starting point to obtain one or more logical paths; Aggregating the semantic segments corresponding to all nodes on the one or more logical paths to obtain stitched segments; Regenerate display content based on the stitched fragments and return text retrieval results.

[0011] Through the above embodiment, after the system generates text retrieval results, if the user needs to conduct a further in-depth search, the system can regenerate new stitched fragments by clicking the first control of the system interface, that is, based on the selected related fragments with high similarity to the retrieval vector, further construct the structural diagram of the original document, and form a logical path based on the logical relationship traversal, and gather the fragments on the path to form a stitched fragment. This feature not only takes into account semantic relevance through structured modeling and logical relationship mining, but also combines the logical hierarchy and content association within the document, which can reduce the omission of semantic fragments caused by physical proximity alone to a certain extent. Ultimately, the retrieval results can cover the complete logical chain, improving the systematicity and coherence of information.

[0012] In some embodiments, after the step of adjusting the display content of the stitched segments according to the search granularity and returning the text search results, the method further includes: In response to the user clicking the preset second control, for each relevant segment, searching for candidate segments in other original documents whose semantic similarity with the relevant segment is greater than a preset third threshold; Calculate the context coherence score for each pair of relevant segments and candidate segments; Merging the relevant segments and candidate segments whose contextual coherence scores are greater than a preset coherence threshold to obtain a cross-document stitched segment; Regenerate display content based on the cross-document stitched fragments and return text retrieval results.

[0013] Through the above embodiment, after the system generates text retrieval results, if the user needs to conduct a further breadth search, they can regenerate cross-document stitched segments by clicking the second control on the system interface. That is, during the segment stitching process, the system is not limited to the original document, but also introduces candidate segments across documents. By calculating the contextual coherence score, multiple related segments and candidate segments are merged. Utilizing the two dimensions of semantic similarity and contextual coherence, semantic stitching across documents is achieved, effectively breaking the limitations of document boundaries. Ultimately, the retrieval results are improved in terms of information coverage and semantic continuity, avoiding information fragmentation.

[0014] In some embodiments, the step of determining the search granularity to be displayed first based on the search results corresponding to the user's historical search requests specifically includes: Count the average reading dwell time of each search granularity under the user's historical search requests to obtain the granular dwell time data; Obtain the similarity between the current search request and the historical search request in terms of topic type and text structure complexity, and obtain the request similarity score; The historical search granularity corresponding to the historical search request with the highest request similarity score is determined as the search granularity to be displayed first.

[0015] Through the above-described embodiment, the system calculates the dwell time of different search granularities under the user's historical search requests and calculates similarity scores based on the topic and structural complexity to intelligently determine the search granularity to display first. This dynamic matching based on historical behavior and request characteristics can adapt to the user's reading habits and preferences. Ultimately, the display granularity of search results better meets user needs, improving the search experience and content utilization efficiency.

[0016] In some embodiments, the step of converting the search vector according to the user's search request includes: Perform word segmentation on the query text corresponding to the user's search request to obtain the word segmentation result; Removing stop words from the word segmentation result based on a stop word list to obtain a valid word segmentation set; Based on the valid word segmentation set and all semantic fragments in the vector database, the keyword score of each valid word segmentation is calculated using the TF-IDF algorithm to obtain a word segmentation score set, wherein the word segmentation score set is a list of scores corresponding to each valid word segmentation; Sorting the word segmentation score set and removing low-scoring words with scores ranking below a preset ratio to obtain a key word segmentation set; Generate a search vector corresponding to each valid word in the key word set.

[0017] Through the above-described embodiment, the system incorporates steps such as word segmentation, stop word removal, TF-IDF keyword scoring, score sorting, and filtering when converting user search requests into search vectors, ensuring that the final search vector accurately reflects user intent. The key lies in the use of score filtering to eliminate noisy words and retain high-weighted word segmentations, making the converted search vector more semantically accurate. Ultimately, this improves the match and relevance between the search request and the document fragment.

[0018] In some embodiments, the step of adjusting the displayed content of the stitched segments according to the search granularity and returning the text search results specifically includes: Comparing the keywords extracted from the search request with the text content of the stitched segments to obtain a keyword matching degree for each stitched segment, where the keyword matching degree is the coverage rate of the search keywords in the stitched segments; The display order of the different stitched segments is determined according to the order of the keyword matching from large to small, and is returned as a text search result.

[0019] Through the above-described embodiment, the system compares search keywords with the stitched fragment content, calculates the keyword matching degree, and uses this to sort and display the search results. This not only ensures the high relevance of the results content, but also improves the intuitiveness of the search results and the efficiency of information utilization. The introduction of keyword matching effectively prevents irrelevant content from being mixed into the search results. Ultimately, the search results are more targeted and readable, meeting the demand for high-quality information screening.

[0020] In a second aspect, the present application provides an intelligent text retrieval system, the intelligent text retrieval system comprising: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions. The one or more processors call the computer instructions so that the intelligent text retrieval system can implement an intelligent text retrieval method provided in the above embodiment, which will not be repeated here.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on an intelligent text retrieval system, the intelligent text retrieval system can implement an intelligent text retrieval method provided in the above embodiment, which will not be described in detail here.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on an intelligent text retrieval system, the intelligent text retrieval system can implement an intelligent text retrieval method provided in the above embodiment, which will not be described in detail here.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By segmenting documents based on semantic similarity, combining the semantic continuity of adjacent segments with the logical relationships within the document structure, this approach dynamically stitches together previously fragmented and disjointed related segments into semantically complete ones. In particular, the introduction of logical path traversal and cross-document contextual coherence calculation ensures that the final search results not only include directly relevant information but also automatically piece together complete arguments or causal chains, thus addressing the technical challenges of logical discontinuity and information loss associated with segment combination in traditional searches.

[0024] 2. To address the diverse needs of different users for information completeness in different search scenarios, the system intelligently determines the optimal display granularity by analyzing the actual reading dwell time and request similarity across users' historical search granularities. This ensures optimal information coverage while maintaining semantic coherence, regardless of whether the user requires summary, paragraph, or chapter-level content. This effectively mitigates the issue of incomplete information caused by overly fine or coarse search results.

[0025] 3. This solution utilizes multi-dimensional semantic analysis methods, including word segmentation, TF-IDF weighting, keyword screening, and coverage matching, during search vector generation and result ranking to ensure that each stitched or combined fragment is closely related to user intent. Sorting results based on keyword matching further enhances the targeted and practical nature of the combined content, preventing the mixing of irrelevant fragments. Ultimately, the search output is both logically coherent and comprehensive. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flowchart of an intelligent text retrieval method according to an embodiment of the present application; Figure 2 This is another flowchart of an intelligent text retrieval method according to an embodiment of the present application; Figure 3 It is a schematic diagram of the physical device structure of the intelligent text retrieval system in the embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "said," "above," "the," and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to any or all possible combinations comprising one or more of the listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, the following describes the process of the method provided by this implementation. Figure 1 , which is a flow chart of an intelligent text retrieval method in an embodiment of the present application.

[0030] S101. Each original document is divided into a plurality of semantic segments according to the semantic similarity of the semantic contents.

[0031] Among them, original documents refer to the initial text files that have not been systematically processed and serve as the retrieval data source, such as academic papers, case files, technical documents, etc.; semantic content refers to the intrinsic meaning, ideas or information expressed by the text fragment; semantic similarity refers to the degree of proximity or correlation between the semantic content of two text fragments, which is usually obtained through quantitative calculation; semantic fragments refer to text units that are relatively complete and coherent in semantics, formed after segmentation based on semantic similarity.

[0032] This step is performed in the initial stage of the intelligent text retrieval system's preprocessing of the original document and is completed before the user initiates a retrieval request. Specifically, the system obtains all the original documents to be retrieved. For each document, it no longer relies on surface formats such as fixed character lengths and paragraph marks for segmentation, but instead deeply analyzes the semantic logic within the document. By calculating the semantic similarity between different text parts in the document, parts with close semantic connections are merged, and parts with weaker semantic connections are split, ultimately obtaining multiple semantic fragments. Each semantic fragment is semantically coherent and can fully express a relatively independent viewpoint, description, or logical unit.

[0033] Optionally, the original document is segmented into independent sentence units; a pre-trained language model (such as Word2Vec) is used to generate a sentence vector for each sentence; the cosine similarity of the sentence vectors of adjacent sentences is calculated as the semantic similarity; a semantic similarity threshold is set, and consecutive adjacent sentences with a similarity greater than the threshold are merged into a semantic segment until the entire document is traversed.

[0034] Optionally, perform preliminary paragraph-level division on the original document to obtain initial paragraph units; extract the core keywords and semantic topics of each paragraph; calculate the matching degree (such as Jaccard coefficient) of the core topics of adjacent paragraphs as semantic similarity; based on a preset threshold, merge adjacent paragraphs with high topic matching into semantic segments, and use paragraphs with large topic differences as the starting point of new segments.

[0035] It is understandable that other methods may be used to implement semantic segmentation, such as identifying core semantic components in combination with semantic role labeling and then performing segmentation, which is not limited here.

[0036] S102: Extract text features of each semantic segment and generate a text segment vector corresponding to each semantic segment.

[0037] Among them, text features refer to the attributes or elements that can represent the core information of semantic fragments, including keywords, parts of speech, syntactic structures, semantic themes, etc.; text fragment vectors refer to the high-dimensional vectors obtained by mathematically transforming the text features of semantic fragments, which are used to quantify the semantic information of the fragments so that semantic similarity can be calculated through vector operations.

[0038] Specifically, the system uses a feature extraction algorithm to mine the underlying text features of each semantic segment obtained in step S101. These features include not only surface-level keyword frequency but also abstract information such as semantic associations and contextual dependencies. Subsequently, a vector transformation model (such as a Transformer-based language model) is used to map these features into high-dimensional text segment vectors. The position of each vector in mathematical space reflects the semantic connotation of its corresponding semantic segment, and the distance or angle between vectors can be used to measure semantic similarity.

[0039] Optionally, perform word segmentation and stop word removal on the semantic fragment, retaining valid vocabulary; use the BERT pre-trained model to encode the processed text to obtain the context-related features of the fragment; extract the [CLS]token vector of the last layer of the model as the text fragment vector of the semantic fragment.

[0040] Optionally, calculate the TF-IDF weight of each word in the semantic segment and filter high-weight keywords; generate a word vector for each keyword based on the Word2Vec model; and perform weighted averaging of the keyword word vectors using the TF-IDF weight as the weight to obtain the text segment vector of the semantic segment.

[0041] It is understandable that other methods can also be used to generate text fragment vectors, such as directly generating sentence-level vectors through Sentence-BERT, which is not limited here.

[0042] S103: Select one or more relevant segments from each text segment vector, the first similarity between which and the search vector is greater than a preset first threshold.

[0043] The search vector refers to the vector obtained by converting the user's search request in the same way as the text fragment vector, which is used to quantitatively represent the user's search intention.

[0044] Specifically, the system first receives a user's search request (e.g., a natural language query) and converts it into a search vector using the same logic used to generate text segment vectors. Subsequently, the system traverses all text segment vectors generated in S102 and calculates a first similarity between each text segment vector and the search vector (commonly calculated using methods such as cosine similarity and Euclidean distance). The calculated first similarity is compared with a preset first threshold, and semantic segments with similarities exceeding the threshold are selected. These segments are then identified as relevant segments directly related to the user's search request.

[0045] Optionally, the user search request is segmented and stop words are removed to generate a query text; a search vector is generated using the same BERT model as the text fragment vector; the cosine similarity between each text fragment vector and the search vector is calculated as the first similarity; a preset first threshold (such as 0.7) is set, and fragments with a similarity greater than 0.7 are marked as related fragments.

[0046] Optionally, the weights of keywords in the search request are calculated based on the TF-IDF algorithm to generate a search vector; the weighted Euclidean distance between the text segment vector and the search vector is calculated, and the smaller the distance, the higher the first similarity; a distance threshold (such as 0.3) is set, and segments with a distance less than 0.3 are marked as related segments; and the related segments are preliminarily sorted from high to low according to similarity.

[0047] It is understandable that other methods may be used to implement relevant segment screening, such as combining keyword exact matching and semantic similarity comprehensive judgment, which is not limited here.

[0048] In addition, the system can also receive query text input by the user, split the continuous text into discrete words or sub-words through an adapted word segmentation tool (such as Jieba for Chinese and WordPiece for English), obtain the word segmentation result, and provide basic semantic units for subsequent processing. Then, based on a general and domain-specific stop word list, meaningless words (such as "de", "zai") in the word segmentation result are filtered, and an effective word segmentation set that can represent the core intention is retained to reduce noise interference. Subsequently, using the TF-IDF algorithm, combined with the term frequency (TF) of the effective word segmentation in the query and the inverse document frequency (IDF) in all semantic segments of the vector database, the keyword score of each effective word segmentation is calculated to generate a word segmentation score set, quantifying the importance of words. Then, the word segmentation score set is sorted in descending order of scores, and low-score words are removed according to a preset ratio (such as the first 60%) to screen out the core key word segmentation set, focusing on the retrieval core intention. Finally, a model consistent with the generation of semantic segment vectors (such as GloVe, BERT) is called to generate word vectors for the key word segmentations, and the final retrieval vector is generated through weighted fusion (according to the keyword score weight) or context encoding, realizing the accurate quantification of the retrieval intention and laying a foundation for subsequent similarity matching.

[0049] S104. Calculate the second similarity between each relevant segment and its adjacent segment.

[0050] Specifically, for each relevant segment screened in step S103, the system traces back to its corresponding original document and locates the semantic segment that is directly adjacent to this relevant segment in the physical position (i.e., the adjacent segment). Subsequently, using the same logic as calculating semantic similarity (such as vector cosine similarity, topic matching degree, etc.), the semantic association degree between the relevant segment and each adjacent segment is calculated, that is, the second similarity. The level of the second similarity reflects the semantic coherence between the relevant segment and the adjacent segment.

[0051] S105. Merge the adjacent segments with a second similarity greater than a preset second threshold with the corresponding relevant segments to obtain stitched segments.

[0052] Specifically, for each relevant segment, the system compares the second similarity between it and the adjacent segment with the preset second threshold. If the second similarity exceeds the threshold, it means that the two are highly semantically associated and logically coherent, and the system will merge this adjacent segment with the relevant segment; if it does not exceed the threshold, it will not be merged. Through this process, the originally possibly scattered relevant segments are "stitched" into stitched segments with more complete semantics, which not only retains the core relevant content but also supplements the necessary context information, avoiding information loss caused by segment segmentation.

[0053] Optionally, traverse each relevant segment and its corresponding preceding and following adjacent segments; extract adjacent segments whose second similarity is greater than a preset second threshold (such as 0.65); splice the relevant segment with its qualified preceding and following adjacent segments in sequence to form a stitched segment containing context; remove redundant information (such as repeated expressions) from the merged stitched segment to retain the core semantics.

[0054] Optionally, for each relevant segment, all adjacent segments in the same document that are physically continuous with the segment are retrieved; the adjacent segments are sorted from high to low according to the second similarity; the adjacent segments with higher sorting and a second similarity greater than a preset second threshold are merged with the relevant segment in sequence until the second similarity of the newly added adjacent segments is lower than the threshold; a merged stitched segment is generated and its source position is marked.

[0055] It is understandable that other methods may be used to achieve segment merging, such as identifying core semantic components in combination with semantic role annotations and then merging, which is not limited here.

[0056] S106: Determine the search granularity to be displayed first based on the search results corresponding to the user's historical search requests.

[0057] Among them, historical search requests refer to the search query text and related parameters initiated by users in the past; search results refer to the system return content corresponding to historical search requests; search granularity refers to the level of detail of the search results display, which is used to adapt to the user's needs for information completeness; summary level refers to the display granularity in the form of brief summary and core idea extraction, with concise and refined content; paragraph level refers to the granularity displayed with paragraphs as the basic unit, retaining the paragraph structure and key details of the text; chapter level refers to the granularity displayed with complete chapters as units, including the complete logical chain and all details of the text.

[0058] Specifically, the system retrieves the user's historical search data, including the topics and text structure of previous search requests, as well as user feedback on different granularities in the corresponding search results (such as reading dwell time). By analyzing this historical data, the system calculates the similarity between the current search request and previous search requests in dimensions such as topic type and text complexity, and finds the most similar historical requests. The user's preferred search granularity in that historical request (such as the granularity with the longest dwell time) is then used as the initial granularity for the current search, ensuring that the display format matches user preferences.

[0059] Optionally, extract all historical search requests and their corresponding search granularities (summary level / paragraph level / chapter level) and user reading dwell time from the user's historical data; count the average dwell time of each granularity in the historical requests, and select the granularity with the longest dwell time as the "preferred granularity" of the historical request; calculate the similarity between the current search request and each historical request in terms of subject keywords, text length, and structural complexity to obtain a request similarity score; determine the "preferred granularity" of the historical request with the highest similarity score as the search granularity currently displayed first.

[0060] S107: Adjust the display content of the stitched segments according to the search granularity, and return the text search results.

[0061] Specifically, the system makes targeted adjustments to the stitched fragments based on the determined search granularity: if it's at the summary level, it extracts the core ideas and key data from the stitched fragment for a concise summary; if it's at the paragraph level, it retains the fragment's original paragraph structure, highlighting keywords and key sentences; if it's at the chapter level, it presents the entire content of the stitched fragment in its entirety, supplementing it with necessary contextual explanations. Simultaneously, the system calculates the degree of match between each stitched fragment and the search query keywords, sorting them from highest to lowest match, and ultimately returning the adjusted content to the user as a text search result.

[0062] Optionally, the stitched fragments are trimmed according to the search granularity: at the summary level, the core sentences (no more than 3 sentences) in the fragment that are directly related to the search keywords are retained; at the paragraph level, the complete paragraph structure is retained and the keywords are highlighted; at the chapter level, all the content is retained and logical connecting words are added; the keywords in the search request are extracted, and the keyword coverage of each stitched fragment (including the number of keywords / total number of keywords) is calculated as the matching degree; the stitched fragments are sorted from high to low by the matching degree, and the top 5 are displayed as the priority results; the sorted results are organized in a granular format, and the source identifier is added before returning to the user.

[0063] Optionally, call corresponding content generation modules for different granularities: the summary-level module generates a structured summary of the fragment (including background, core conclusions, and key evidence); the paragraph-level module retains the original paragraph layout and marks the paragraph topic sentences; the chapter-level module generates the content directory and core idea index of the fragment; calculates the keyword matching degree and semantic similarity comprehensive score of each stitched fragment; sorts by comprehensive score and filters the top 10 results; adds a "switch granularity" option to the results to allow users to dynamically adjust the display format, and finally returns the sorted results.

[0064] It is understandable that other methods can also be used to adjust the displayed content and return results, such as optimizing the typesetting style based on the user's historical browsing habits, which is not limited here.

[0065] In addition, after the system returns the search results based on the stitched fragments determined based on the second similarity between each relevant fragment and the adjacent fragments in step S104 and step S105, if the user is not satisfied with the current results, he or she can perform further search by clicking the selection control (including the preset first control and the preset second control) on the system interface: the first control is used to further deeply mine the content of each original document, and to deeply mine the relevant fragments from the original document corresponding to each relevant fragment by constructing a document structure diagram and the logical relationship between different nodes in the document structure diagram, so that the logic of the returned search results is more comprehensive and complete; the second control is used to further breadth mine each relevant fragment, and to splice the relevant fragments of different documents by calculating the semantic similarity and coherence score with the relevant fragments of other documents, thereby expanding the search results and improving the "breadth" of the returned search results.

[0066] The execution process of clicking the preset first control is specifically referred to step S108 to step S111.

[0067] S108: In response to the user clicking on the preset first control, construct a document structure diagram of the original document corresponding to each relevant segment.

[0068] Specifically, for each relevant segment, the system locates the original document to which it belongs. It then extracts all semantic segments from that original document (including relevant segments and other non-relevant semantic segments) as nodes in the document structure diagram. Node attributes include the segment's text content, its physical location in the original document (such as the start / end page numbers and paragraph numbers), and its core topic tags. The system also preliminarily defines basic connectivity relationships between nodes (such as physical proximity), forming a preliminary document structure diagram framework.

[0069] Optionally, for the original documents corresponding to the relevant fragments, extract all semantic fragments and assign a unique node ID to each fragment; add attribute information to each node: text content, physical location index, core keyword set; based on the physical order of the fragments in the original document, use "adjacent" edges to connect nodes with consecutive positions; save the document structure diagram containing node attributes and adjacent edges to the graph database.

[0070] Optionally, call a graph building tool (such as Neo4j) to create a graph space based on the original document; convert all semantic fragments of the document into nodes in the graph, and bind the node ID to the unique identifier of the fragment in the document; use the semantic similarity calculation results to add "semantic association" edges for non-adjacent fragments whose semantic similarity is higher than a basic threshold (such as 0.3); generate a document structure diagram containing nodes, adjacent edges and semantic association edges and preview it visually.

[0071] S109: In each document structure diagram, the logical relationship between nodes is identified based on preset logical rules to obtain one or more logical relationship edges.

[0072] Specifically, for each document structure diagram, the system calls upon a preset logical rule library to analyze the text content of all nodes (semantic fragments) in the diagram. Logical rules can include: rules based on logical connectives (such as "because...so..." corresponding to causal relationships), rules based on syntactic structure (such as "if...then..." corresponding to conditional relationships), rules based on topic progression (such as "first...second...finally..." corresponding to progressive relationships), and so on. By matching these rules, the system identifies the specific logical relationship types between nodes and adds logical relationship edges with relationship type labels (such as "causal edge" and "progressive edge") between the corresponding nodes. These edges can accurately reflect the logical coherence between fragments and enhance the semantic expression capabilities of the document structure diagram.

[0073] S110 , taking the node corresponding to each relevant segment as a starting point, traverse one or more logical relationship edges in the document structure graph corresponding to the starting point to obtain one or more logical paths.

[0074] Specifically, the system initiates a traversal process in the document structure diagram corresponding to each relevant fragment (the starting node). Traversal methods include depth-first traversal (prioritizing in-depth exploration of a single logical branch) and breadth-first traversal (prioritizing coverage of multiple logical branches). The system adjusts the traversal priority based on the importance of the logical relationship type (e.g., causal relationships take precedence over parallel relationships). During the traversal process, the system visits the associated nodes along the logical relationship edge in sequence, recording the node ID and the logical relationship type of the edge until it encounters a node with no subsequent logical relationship edge or a loop node. Ultimately, one or more logical paths extending from the starting node are formed, and each path contains a continuous, logically related sequence of nodes.

[0075] Optionally, determine the starting point node corresponding to the relevant fragment, initialize the path list and add the starting point node to the current path; identify all logical relationship edges associated with the starting node and sort them by relationship importance (such as causal > progressive > parallel); visit the next node along the logical relationship edge that ranks first and add the node to the current path; repeat the sorting steps until there are no untraversed logical relationship edges at the node, save the current path and backtrack to the nearest branch point, continue to traverse other branches, and finally obtain multiple logical paths.

[0076] Optionally, set weights for the logical relationship edges in the document structure diagram (e.g., 0.8 for causal relationships and 0.5 for parallel relationships); use the Dijkstra algorithm to calculate the shortest weighted paths to other nodes with the starting node as the center (the lower the weight, the higher the path priority); select paths with weighted path lengths less than a preset threshold (e.g., weight sum < 2.0) as valid logical paths; sort the paths from smallest to largest by path weight sum, and retain the top N (e.g., top 3) logical paths.

[0077] S111 , gathering semantic segments corresponding to all nodes on one or more logical paths to obtain stitched segments.

[0078] Specifically, for each logical path, the system extracts the semantic fragment text content corresponding to all nodes on the path. The fragments are sorted according to the physical location order of the nodes in the original document (or the traversal order in the logical path) to ensure the coherence of the text flow. After sorting, the system optimizes the connection between the fragments, such as removing repeated expressions and adding logical transition words (such as "therefore" and "in addition") to enhance coherence. For multiple logical paths, the system can select the fragments corresponding to the optimal path according to the completeness of the path (such as the number of nodes included) and the relevance to the search request (such as keyword coverage), or merge the core fragments of multiple paths to finally form a stitched fragment.

[0079] Finally, the system regenerates the display content based on the final stitched segments and returns the search results. The process of regenerating the display content based on the final stitched segments is basically the same as the data processing process in step S107 and will not be repeated here.

[0080] In addition, the execution process of clicking the preset first control is as follows: For each relevant fragment screened out from the original document, the system breaks through the limitation of a single document and expands the search scope to all other original documents. By calculating the semantic similarity between the relevant fragment and the semantic fragments in other documents, the system screens out candidate fragments with a similarity greater than a preset third threshold, and mines potential supplementary information from the global document library; then, for each pair of relevant fragments and candidate fragments, the system analyzes them from multiple dimensions such as topic consistency, logical relationship, and expression style, quantifies the results of each dimension and weightedly integrates them to obtain a quantitative score that measures the contextual coherence of the two, ensuring that the screened candidate fragments are not only semantically similar, but also logically coherent; finally, the system compares the contextual coherence score with the preset coherence threshold, splices the fragment pairs with scores exceeding the threshold in a logical order, adds transition words and removes repeated expressions when necessary, and finally forms a cross-document stitched fragment that integrates relevant information from multiple documents, is semantically coherent, and has complete information.

[0081] Finally, the system regenerates the display content based on the cross-document stitched fragments and returns the search results. The process of regenerating the display content based on the cross-document stitched fragments is basically the same as the data processing process in step S107 and will not be repeated here.

[0082] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of an intelligent text retrieval method in an embodiment of the present application.

[0083] S201 : Perform preliminary sentence-level segmentation on the original document to obtain a plurality of preliminary sentence units.

[0084] This step is performed during the initial phase of the intelligent text retrieval system's semantic segmentation of the original document. Specifically, the system obtains the original document to be processed and uses natural language processing techniques to identify sentence boundaries (such as punctuation marks like periods, question marks, exclamation points, or language-specific syntactic markers). It then breaks the continuous text stream into independent preliminary sentence units based on these boundaries. Each preliminary sentence unit retains its complete grammatical structure and semantic information, ensuring the accuracy of subsequent semantic similarity calculations.

[0085] Optionally, read the text content of the original document and remove redundant spaces, line breaks, and other formatting symbols from the text; locate sentence delimiters based on preset sentence boundary rules (such as recognizing the Chinese "。?!" and English ".", "?", "!"); segment the text into independent sentence strings based on the delimiters, with each string serving as a preliminary sentence unit; verify the segmented sentence units and merge short fragments caused by incorrect punctuation (such as periods after abbreviations).

[0086] Optionally, call the sentence segmentation function in a natural language processing tool (such as NLTK or spaCy); preprocess the original document text and mark punctuation and syntactic pauses; use the tool's built-in sentence boundary detection model to identify complete sentences and generate a preliminary list of sentence units; manually or automatically spot-check the segmentation results and correct special sentence patterns that were incorrectly segmented (such as sentences within quotation marks).

[0087] S202: Calculate the semantic similarity between each pair of adjacent preliminary sentence units to obtain adjacent sentence similarity.

[0088] Specifically, the system traverses all preliminary sentence units and selects each pair of adjacent sentence units (e.g., sentence 1 and sentence 2, sentence 2 and sentence 3, etc.) in the order in which they appear in the original document. Using semantic similarity calculation algorithms (such as vector cosine similarity and topic matching), the system analyzes the degree of correlation between adjacent sentences in terms of semantic content, core themes, and logical relationships, generating a corresponding similarity score. A higher score indicates greater semantic coherence between adjacent sentences, making them more suitable for merging into a single semantic segment.

[0089] Optionally, extract the core keywords of each preliminary sentence unit (using the TF-IDF or TextRank algorithm); calculate the Jaccard coefficient (number of intersection keywords / number of union keywords) of the keyword sets of adjacent sentences as the basic similarity; adjust the similarity score based on the results of the sentence syntactic dependency analysis (such as the subject-verb-object association); and generate the final adjacent sentence similarity score.

[0090] It is understandable that other methods can also be used to implement similarity calculation, such as calculating similarity after generating sentence embeddings through the BERT model, which is not limited here.

[0091] S203: Merge adjacent sentences to obtain multiple semantic segments, and select one or more relevant segments from them according to the search vector.

[0092] Specifically, the system compares the similarity scores of each pair of adjacent preliminary sentence units with a preset threshold. If the similarity is greater than the threshold, indicating a close semantic connection and logical coherence between the two, the system will merge the two sentence units; if it is less than or equal to the threshold, the two sentence units will not be merged, and the system will use this as the starting point for a new semantic segment. By traversing and merging sentence by sentence, the originally independent sentence units are integrated into multiple semantic segments. Each segment contains continuous and semantically coherent sentences that can fully express a relatively independent point of view, description, or logical unit.

[0093] Next, the system executes steps S102 and S103 based on the integrated semantic segments, and selects one or more relevant segments according to the search vector, which will not be described in detail here.

[0094] Optionally, initialize the semantic fragment list and use the first preliminary sentence unit as the starting sentence of the first fragment; traverse adjacent sentence pairs in sequence, and if the similarity is greater than a preset threshold (such as 0.6), merge the subsequent sentence unit into the current fragment; if the similarity is less than or equal to the threshold, end the current fragment and use the subsequent sentence unit as the starting sentence of the new fragment; after the traversal is completed, generate a semantic fragment list containing all merged fragments and mark the number of sentences in each fragment.

[0095] Optionally, sort the similarity scores of all adjacent sentences and filter out sentence pairs with similarity greater than a preset threshold; construct a sentence association graph based on the screening results, and connect adjacent sentence pairs that meet the similarity criteria with edges; identify connected components in the association graph, and merge the continuous sentence units contained in each connected component into a semantic segment; perform semantic integrity check on the merged segments, and split segments that are too long or have topic deviations.

[0096] S204: Construct a stitched segment based on the second similarity between each related segment and the adjacent segment.

[0097] S205 : Count the average reading dwell time of each search granularity under the user's historical search requests to obtain granularity dwell time data.

[0098] Specifically, the system extracts all historical search request records from the user behavior log, including the display content of different granularities (summary level, paragraph level, chapter level) in the search results corresponding to each request and the user's reading behavior data (such as page dwell time, scrolling depth, etc.). For each search granularity, the system filters out abnormal dwell time (such as too short false touch time) and calculates the average dwell time at that granularity. For example, if a user stays for an average of 5 minutes in the paragraph-level results of a historical request, but only stays for 1 minute at the summary level, it means that the user prefers the paragraph-level granularity. Finally, granular dwell time data containing the correspondence between "search granularity-average dwell time" is generated.

[0099] S206: Obtain the similarity between the current search request and the historical search request in terms of topic type and text structure complexity to obtain a request similarity score.

[0100] Specifically, the system labels the current search request by topic type (e.g., through keyword matching or topic model classification) and analyzes the text structure complexity (e.g., calculating average sentence length and term proportion). It then traverses historical search requests, extracting the topic type and structural complexity features of each historical request. By calculating the matching degree between the current request and historical requests in terms of topic type (e.g., 1 point for the same topic, 0.6 points for related topics) and the Euclidean distance in structural complexity (the smaller the distance, the higher the similarity), the results of the two dimensions are weighted and fused (e.g., a topic weight of 0.7 and a structure weight of 0.3) to obtain the final request similarity score. The higher the score, the more similar the two search scenarios are.

[0101] Optionally, extract topic keywords for the current search request and historical search requests respectively (such as using the TF-IDF algorithm); calculate the Jaccard similarity of the topic keyword set as the topic matching degree; analyze the text structure complexity, calculate the average sentence length difference and term density difference between the current and historical requests, and obtain the structural similarity after normalization; and obtain the request similarity score by weighted summation of the topic matching degree (weight 0.6) and structural similarity (weight 0.4).

[0102] S207: Determine the historical search granularity corresponding to the historical search request with the highest request similarity score as the search granularity to be displayed first.

[0103] Specifically, the system sorts all historical search requests by request similarity score from high to low, selecting the historical search request with the highest score (i.e., the historical sample most similar to the current request). The system then extracts the historical search granularity corresponding to that historical request (i.e., the granularity at which the user spent the longest time for that historical request, as counted in step S205), and determines this granularity as the first granularity to be displayed in the current search. For example, if the historical request most similar to the current request had the longest user stay at the paragraph-level granularity, the current search will prioritize displaying paragraph-level content.

[0104] S208: Compare the keywords extracted from the search request with the text content of the stitched segments to obtain a keyword matching degree for each stitched segment.

[0105] Specifically, the system extracts keywords from the current search request (e.g., by filtering high-weight words using the TF-IDF or TextRank algorithms) to form a keyword set. It then traverses the text content of each stitched segment and counts the number of keywords contained in the segment (including exact matches and synonym matches). The number of matched keywords is divided by the total number of keywords in the search request to obtain the keyword matching degree for the stitched segment (ranging from 0 to 1). For example, if the search keyword is "artificial intelligence + search method," and a stitched segment contains both keywords, the matching degree is 1; if it only contains "artificial intelligence," the matching degree is 0.5.

[0106] Optionally, extract N core keywords (e.g., N=5) from the search request to form a keyword list; perform word segmentation and stop word removal on each stitched segment to obtain a segment vocabulary set; count the number of words M in the segment vocabulary set that completely match the keyword list; calculate the matching degree = M / N, and round to two decimal places as the final result.

[0107] Optionally, perform synonym expansion on the search request keywords (such as through WordNet or a domain vocabulary) to form an extended keyword set; traverse the stitched fragment text and identify matching keywords through string matching and semantic similarity calculation (such as word vector cosine similarity > 0.8); count the number of matched extended keywords M and remove duplicate matches; calculate the matching degree = M / the number of original keywords, where the number of original keywords is the total number of keywords before expansion.

[0108] S209: Determine the display order of different stitched segments according to the order of keyword matching from large to small, and return it as a text search result.

[0109] Specifically, the system sorts all stitched segments by keyword matching from highest to lowest. Segments with equal matching scores are sorted by their physical location in the original document (or sorted again by semantic similarity). Then, based on the search granularity determined for initial presentation in step S207, the sorted segments are adjusted (e.g., extracting core sentences at the summary level and retaining the original structure at the paragraph level). Finally, the adjusted segments are displayed in sorted order on the results page, with supplementary information such as source document identification and keyword highlighting added to form a complete text search result, which is then returned to the user.

[0110] Optionally, all stitched fragments are sorted in descending order according to keyword matching to generate a preliminary sorted list; fragments with the same matching degree are sorted again according to their corresponding logical path lengths (the longer the better); based on the retrieval granularity displayed first (such as the summary level), display content of corresponding granularity is generated for each fragment; the fragments are displayed in sorted order on the result page, with the matching score and source document marked, and returned to the user.

[0111] The intelligent text retrieval system of the embodiment of the present invention is applied to electronic devices. Figure 3 A schematic diagram of the architecture of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0112] It should be noted that Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0113] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be accomplished by instructions (computer programs) or by instructions (computer programs) controlling related hardware. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. The electronic device of this embodiment includes a storage medium and a processor, wherein the storage medium stores multiple instructions that can be loaded by the processor to execute any step of the method provided in the embodiments of the present invention.

[0114] Specifically, the storage medium and the processor are electrically connected, directly or indirectly, to enable data transmission or interaction. For example, these elements may be electrically connected to each other via one or more signal lines. The storage medium stores computer-executable instructions for implementing the data access control method, including at least one software functional module that can be stored in the storage medium in the form of software or firmware. The processor executes various functional applications and data processing by running the software programs and modules stored in the storage medium. The storage medium may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The storage medium is used to store programs, and the processor executes the programs after receiving execution instructions.

[0115] Furthermore, the software programs and modules in the above-mentioned storage medium may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components. The processor may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., which may implement or execute the various methods, steps, and logic flow diagrams disclosed in this embodiment. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0116] Since the instructions stored in the storage medium can execute the steps of any method provided in the embodiments of the present invention, the beneficial effects of any method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0117] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An intelligent text retrieval method, applied to an intelligent text retrieval system, characterized in that: The method comprises: Each original document is divided into multiple semantic segments according to the semantic similarity of the semantic content; Extracting text features of each semantic segment and generating a text segment vector corresponding to each semantic segment; selecting one or more relevant segments from each text segment vector whose first similarity to a search vector is greater than a preset first threshold, the search vector being converted according to a user's search request; calculating a second similarity between each of the relevant segments and an adjacent segment, wherein the adjacent segment and the relevant segment are physically continuous in the original document; Merging adjacent segments whose second similarity is greater than a preset second threshold with corresponding related segments to obtain a stitched segment; Determine the search granularity to be displayed first based on the search results corresponding to the user's historical search requests, wherein the search granularity includes summary level, paragraph level and chapter level; The display content of the stitched segments is adjusted according to the search granularity, and a text search result is returned.

2. The method according to claim 1, characterized in that The step of dividing each original document into a plurality of semantic segments according to the semantic similarity of the semantic content specifically includes: Performing preliminary sentence-level segmentation on the original document to obtain a plurality of preliminary sentence units, wherein the preliminary sentence units are the smallest grammatical units obtained by dividing the original document according to sentence boundaries; Calculate the semantic similarity between each pair of adjacent preliminary sentence units to obtain the adjacent sentence similarity; The preliminary sentence units whose adjacent sentences have similarities greater than a preset threshold are merged to obtain multiple semantic segments.

3. The method according to claim 1, characterized in that After the step of adjusting the display content of the stitched segments according to the search granularity and returning the text search results, the method further includes: In response to a user clicking a preset first control, constructing a document structure graph of the original document corresponding to each of the related segments, wherein each semantic segment in the original document serves as a node in the document structure graph; In each of the document structure graphs, the logical relationship between the nodes is identified based on a preset logical rule to obtain one or more logical relationship edges; Taking the node corresponding to each of the related fragments as a starting point, traversing the one or more logical relationship edges in the document structure graph corresponding to the starting point to obtain one or more logical paths; Aggregating the semantic segments corresponding to all nodes on the one or more logical paths to obtain stitched segments; Regenerate display content based on the stitched fragments and return text retrieval results.

4. The method according to claim 1, wherein After the step of adjusting the display content of the stitched segments according to the search granularity and returning the text search results, the method further includes: In response to the user clicking the preset second control, for each relevant segment, searching for candidate segments in other original documents whose semantic similarity with the relevant segment is greater than a preset third threshold; Calculate the context coherence score for each pair of relevant segments and candidate segments; Merging the relevant segments and candidate segments whose contextual coherence scores are greater than a preset coherence threshold to obtain a cross-document stitched segment; Regenerate display content based on the cross-document stitched fragments and return text retrieval results.

5. The method according to claim 1, wherein The step of determining the search granularity to be displayed first based on the search results corresponding to the user's historical search requests specifically includes: Count the average reading dwell time of each search granularity under the user's historical search requests to obtain the granular dwell time data; Obtain the similarity between the current search request and the historical search request in terms of topic type and text structure complexity, and obtain the request similarity score; The historical search granularity corresponding to the historical search request with the highest request similarity score is determined as the search granularity to be displayed first.

6. The method according to claim 1, characterized in that The step of converting the search vector according to the user's search request includes: Perform word segmentation on the query text corresponding to the user's search request to obtain the word segmentation result; Removing stop words from the word segmentation result based on a stop word list to obtain a valid word segmentation set; Based on the valid word segmentation set and all semantic fragments in the vector database, the keyword score of each valid word segmentation is calculated using the TF-IDF algorithm to obtain a word segmentation score set, wherein the word segmentation score set is a list of scores corresponding to each valid word segmentation; Sorting the word segmentation score set and removing low-scoring words with scores ranking below a preset ratio to obtain a key word segmentation set; Generate a search vector corresponding to each valid word in the key word set.

7. The method according to claim 1, characterized in that The step of adjusting the display content of the stitched segments according to the search granularity and returning the text search results specifically includes: Comparing the keywords extracted from the search request with the text content of the stitched segments to obtain a keyword matching degree for each stitched segment, where the keyword matching degree is the coverage rate of the search keywords in the stitched segments; The display order of the different stitched segments is determined according to the order of the keyword matching from large to small, and is returned as a text search result.

8. An intelligent text retrieval system, characterized in that: The intelligent text retrieval system includes: one or more processors and memories; The memory is coupled to the one or more processors, and is configured to store computer program codes, where the computer program codes include computer instructions. The one or more processors call the computer instructions to enable the intelligent text retrieval system to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on an intelligent text retrieval system, the intelligent text retrieval system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on an intelligent text retrieval system, the intelligent text retrieval system is enabled to perform the method according to any one of claims 1 to 7.

Images