Long text retrieval method, device and computer equipment based on graph structure

Abstract

The present application relates to a long text retrieval method, apparatus, and computer device based on a graph structure. The method comprises: dividing an input long text into blocks, guiding a large language model to analyze the sub-texts after the blocks are divided, obtaining text information of each sub-text, connecting the nodes of the sub-texts with similar core elements to obtain a first structure graph, processing the first structure graph using a community clustering algorithm to obtain a second structure graph with a hierarchical structure of a graph community, receiving a question input by a user, processing and optimizing the question based on a large language model, confirming whether the question is a specific question or an abstract question, if the question is a specific question, retrieving the question from the first structure graph to obtain a reading queue, generating notebook content based on the reading queue, if the question is an abstract question, retrieving the question from the second structure graph to obtain notebook content containing answers and scores, and summarizing and outputting the notebook content through the large language model.

Description

Technical Field

[0001] The present application relates to the field of text processing and retrieval technology, and in particular to a long text retrieval method, apparatus, and computer equipment based on a graph structure. Background Art

[0002] Long text processing refers to a model's ability to process and analyze information spanning large distances within a text. It is a crucial feature of large language models when understanding and generating natural language. For example, information points within a text may be scattered across different paragraphs or chapters. This information is crucial for understanding the entire text or answering specific questions. The model needs to be able to track and connect these scattered points of information. This involves not only understanding the surface meaning of the text but also extracting deeper semantic information, relationships, and structure.

[0003] There are two existing methods for processing long texts: model-level methods and agent-level methods. Model-level methods focus on improving the internal structure of large language models so that they can process long texts more effectively. This method generally includes: 1) enhancing the model's understanding of the order of words in the text by adjusting position embeddings; 2) using a modified attention mechanism to improve the model's ability to handle long-distance dependencies; 3) maintaining the relationship between different parts of a long text through specific alignment techniques, so that the model can better understand and process long texts, etc. Agent-level methods focus on using retrieval-enhanced large language models or agents to process long texts, for example: 1) retrieving task-related information after segmenting the long text to assist in response generation; 2) using LLM as an agent to leverage its powerful planning and reflection capabilities to handle complex problems; 3) organizing documents into a graph structure and exploring the nodes and edges in the graph to collect necessary information, etc.

[0004] Both approaches have their pros and cons. Model-level approaches typically require training for texts of a specific length, which leads to high training costs and difficulty in constructing datasets. Furthermore, large language models with long contexts optimized using model-level approaches often overlook key details within these long contexts, limiting their ability to handle complex tasks such as multi-hop problems. While agent-level approaches can handle long texts, they are limited in capturing multi-hop reasoning, global issues, and long-range information dependencies. When texts are too long, they can lead to problems such as forgetting early information and missing details. Summary of the Invention

[0005] Based on this, it is necessary to provide a long text retrieval method, device and computer equipment based on graph structure to address the above technical problems.

[0006] A long text retrieval method based on a graph structure, the method comprising:

[0007] The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0008] Connect the nodes of sub-texts with similar core elements to obtain the first structure graph;

[0009] Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities;

[0010] Receive a question input by a user, process and optimize the question based on the large language model, and confirm whether the question is specific or abstract;

[0011] When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores.

[0012] The notebook content is summarized and outputted through a large language model.

[0013] In one embodiment, the method further includes: dividing the input long text into blocks using predefined rules to obtain multiple semantically complete sub-texts.

[0014] In one embodiment, the method further includes guiding a large language model to analyze the segmented sub-texts to obtain core elements corresponding to words or phrases in the sub-texts and atomic facts corresponding to sentence structures; the core elements include named entities, keywords, subject terms, and relationships between named entities; the atomic facts represent atomic fact triples that reflect sentence structures and sub-text semantics.

[0015] In one embodiment, the method further includes: recursively performing a community clustering algorithm on the first structure graph to generate a hierarchical structure of graph communities until the community size reaches a preset value, thereby obtaining a second structure graph.

[0016] In one embodiment, the method further includes: using a large language model to summarize the atomic facts of each node in the second structural graph to generate the atomic facts of the community.

[0017] In one embodiment, the method further includes: receiving a question input by a user, rewriting and simplifying the question based on a large language model, and obtaining an exploration plan for the optimized question using a thinking chain guidance method; and determining whether the question is a specific question or an abstract question based on the exploration plan.

[0018] In one embodiment, it also includes: when the question is a specific question, guiding the large language model to determine the starting node to be retrieved in the first structure diagram according to the semantics of the question, and adding the starting node to the reading queue; marking the node with an ID, and providing the node ID and the corresponding atomic fact to the agent, so that the agent reads the atomic fact to obtain overview information of the node; in the agent, using the pre-trained BERT small model to determine the neighboring nodes of the starting node containing useful information according to the question and the exploration plan, adding the ID of the neighboring node containing useful information to the reading queue, traversing the original sub-text of each node according to the reading queue, reading the nodes in the queue that meet the question and the exploration plan through the BERT small model, and adding them to the notebook content; when the question is an abstract question, guiding the large language model to answer the question in parallel according to the summary text of each community, and generating a value score for each answer; arranging the answers in descending order according to the value score, and adding them to the notebook content.

[0019] A long text retrieval device based on a graph structure, comprising:

[0020] The input module is used to divide the input long text into blocks and guide the large language model to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to words or phrases and the atomic facts corresponding to sentence structures;

[0021] A first graph structure generation module is used to connect nodes of sub-texts with similar core elements to obtain a first structure graph;

[0022] A second graph structure generating module, configured to process the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities;

[0023] A question processing module is used to receive questions input by users, process and optimize the questions based on the large language model, and confirm whether the questions are specific or abstract;

[0024] The retrieval output module is used to retrieve the question in the first structural diagram to obtain a reading queue when the question is a specific question, and generate notebook content based on the reading queue; when the question is an abstract question, retrieve the question in the second structural diagram to obtain notebook content containing answers and scores; and summarize and output the notebook content through a large language model.

[0025] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0027] Connect the nodes of sub-texts with similar core elements to obtain the first structure graph;

[0028] Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities;

[0029] Receive a question input by a user, process and optimize the question based on the large language model, and confirm whether the question is specific or abstract;

[0030] When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores.

[0031] The notebook content is summarized and outputted through a large language model.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0033] The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0034] Connect the nodes of sub-texts with similar core elements to obtain the first structure graph;

[0035] Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities;

[0036] Receive a question input by a user, process and optimize the question based on the large language model, and confirm whether the question is specific or abstract;

[0037] When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores.

[0038] The notebook content is summarized and outputted through a large language model.

[0039] The above-mentioned graph-structure-based long text retrieval method, device and computer equipment first divide the long text into blocks, and then analyze the sub-texts after the blocks to obtain the text information of each sub-text, thereby constructing a first structure graph. The first structure graph contains both core elements corresponding to words or phrases and atomic facts corresponding to sentence structures. The first structure graph is then enhanced to obtain a second structure graph. The second structure graph has a hierarchical structure of a graph community. Finally, user questions are received, and the questions are optimized and processed based on a large language model to determine whether they are specific questions or abstract questions. If it is a specific question, the first structure graph is used for retrieval. If it is an abstract question, the second structure graph is used for retrieval, and finally an answer output is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 1 is a flow chart of a long text retrieval method based on a graph structure in one embodiment;

[0041] Figure 2 A schematic diagram of a process for analyzing a subtext in an embodiment;

[0042] Figure 3 A schematic diagram of a problem handling and optimization process in one embodiment;

[0043] Figure 4 Schematic diagram of a flow chart of a long text retrieval method based on a graph structure in another embodiment;

[0044] Figure 5 is a structural block diagram of a long text retrieval device based on a graph structure in one embodiment;

[0045] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] In one embodiment, Figure 1 As shown in FIG, a long text retrieval method based on a graph structure is provided, which includes the following steps:

[0048] Step 102 : Divide the input long text into blocks, guide the large language model to analyze the divided sub-texts, and obtain the text information of each sub-text.

[0049] Text information includes core elements corresponding to words or phrases and atomic facts corresponding to sentence structures.

[0050] Step 104 : Connect the nodes of the sub-texts with similar core elements to obtain a first structure diagram.

[0051] Step 106 : Process the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities.

[0052] Step 108: Receive the question input by the user, process and optimize the question based on the large language model, and confirm whether the question is a specific question or an abstract question.

[0053] Step 110: When the question is a specific question, the question is retrieved from the first structure diagram to obtain a reading queue, and notebook content is generated based on the reading queue. When the question is an abstract question, the question is retrieved from the second structure diagram to obtain notebook content containing answers and scores.

[0054] Step 112: Summarize and output the notebook content using a large language model.

[0055] In the above-mentioned long text retrieval method based on graph structure, the long text is first divided into blocks, and then the sub-texts after division are analyzed to obtain the text information of each sub-text, so as to construct a first structural graph. The first structural graph contains both core elements corresponding to words or phrases and atomic facts corresponding to sentence structures. The first structural graph is then enhanced to obtain a second structural graph. The second structural graph has a hierarchical structure of a graph community. Finally, user questions are received, and the questions are optimized and processed based on a large language model to determine whether they are specific questions or abstract questions. If it is a specific question, the first structural graph is used for retrieval. If it is an abstract question, the second structural graph is used for retrieval, and finally the answer output is generated.

[0056] In one embodiment, a long input text is segmented into chunks using predefined rules to obtain multiple semantically complete subtexts. The text chunks are the basic units used in subsequent graph techniques and also serve as source references for extracting knowledge items.

[0057] In one embodiment, a large language model is guided to analyze the segmented subtexts to obtain core elements corresponding to words or phrases in the subtexts and atomic facts corresponding to sentence structures. Core elements include named entities, keywords, subject terms, and relationships between named entities. Atomic facts represent atomic fact triplets that reflect sentence structures and the semantics of the subtexts. The first structural diagram serves as a knowledge base for answering general and specific questions.

[0058] like Figure 2As shown, if the original text is "Machine learning means that computers learn from data and then use experience to improve their performance. The algorithm will be continuously trained to discover patterns and correlations from large data sets, and then make the best decisions and predictions based on the data analysis results", the text is analyzed by a large language model, and the core element obtained is the concept of machine learning. The atomic fact obtained is: by analyzing large amounts of data, machine learning algorithms can identify patterns, establish correlations, and use this knowledge to improve their own performance and make the best predictions.

[0059] In one embodiment, a community clustering algorithm is used to recursively perform the first structure graph to generate a hierarchical structure of graph communities until the community size reaches a preset value, thereby obtaining a second structure graph.

[0060] Specifically, a recursive community clustering algorithm is applied to the constructed first structure graph using the hierarchical Leiden algorithm to generate a hierarchical structure of graph communities until a community size threshold is reached. The threshold size defaults to 5 to 10 based on the number of nodes in the actual graph, the semantic complexity, and the allocation of computing resources. This provides a method for navigating and summarizing the graph at different granularity levels. This second structure graph serves as a knowledge base for answering abstract, global questions.

[0061] In one embodiment, a large language model is used to summarize the atomic facts of each node in the second structure graph to generate atomic facts of the community. The atomic facts of the community together constitute a high-level understanding of the global structure and semantics of the entire long text.

[0062] In one embodiment, a question input by a user is received, the question is rewritten and simplified based on a large language model, and an exploration plan is obtained for the optimized question using a thought chain guidance method; and based on the exploration plan, it is determined whether it is a specific question or an abstract question.

[0063] like Figure 3 As shown, taking the question input by the user as "Does rag need to be connected to the Internet?" as an example, the question is rewritten and simplified through the large language model. Specifically, the question is rewritten to retain the core elements of the original question, while being streamlined and optimized to make the question more formal and written. The optimized question is "Does the use of RAG retrieval enhancement generation technology require a network environment?", and then the optimized question is guided by a chain of thinking to obtain an exploration plan. Specifically, it can be "We need to first identify the RAG or retrieval enhancement generation technology, and then find its usage conditions, and then determine whether the usage conditions include a network connection." Finally, according to the exploration plan, it is determined whether it is a specific problem or an abstract problem. In this case, it is determined to be "This problem is a general and specific problem, and the first structural diagram is retrieved."

[0064] In another embodiment, when the question is specific, the large language model is guided to determine the starting node to be retrieved in the first structure diagram based on the semantics of the question, and the starting node is added to the reading queue; the node is marked with an ID, and the node ID and the corresponding atomic fact are provided to the agent so that the agent can read the atomic fact and obtain an overview of the node; in the agent, based on the question and exploration plan, the pre-trained BERT small model is used to determine the neighboring nodes of the starting node that contain useful information, and the IDs of the neighboring nodes containing useful information are added to the reading queue. The original subtext of each node is traversed according to the reading queue, and the nodes in the queue that meet the question and exploration plan are read by the BERT small model and added to the notebook content. The entire process is a dynamic, iterative feedback process, in which the agent continuously collects information, updates the notebook content, and makes decisions based on the current understanding and question requirements. This process allows the agent to process long text information in a structured and goal-oriented manner, effectively improving the understanding and reasoning ability of long texts.

[0065] When the question is abstract, the large language model is guided to answer the question in parallel based on the summary text of each community and generate a value score for each answer; community answers with scores below a certain threshold are filtered out, sorted in descending order by value score, and added to the notebook content until the large language model context window size limit is reached.

[0066] Specifically, the present invention is as follows Figure 4As shown in the graph construction phase, the agent first divides long text into small text blocks according to a predefined semantic segmentation strategy. Each text block contains a certain degree of complete semantic information. It then guides the large language model to extract core elements and summarize atomic facts from the text blocks, resulting in a first structural graph, a graph structure used to answer general and specific questions. The graph structure of the first structural graph is further enhanced, and the Leiden algorithm is used to divide the graph into communities to obtain a second structural graph, a graph structure used to answer global and abstract questions. In the graph retrieval phase, the agent first uses the large language model to optimize and classify user questions, aligning the user's questions with the model's own capabilities and categorizing the questions into specific and abstract questions. For specific questions, the search is performed on the first structure graph: the large language model is used to select a starting node, and then the first structure graph node retrieval module selects appropriate nodes and adds them to the reading queue. The first structure graph content expansion module updates the notebook, while continuously feeding back to the exploration plan for updates and continuous improvement of the notebook content. For abstract questions, the search is performed on the second structure graph: the large language model is used to complete the node community division, and the second structure graph community retrieval module obtains the answer and score of each community for the question. The second structure graph content expansion module then completes the notebook content. During the answer generation phase, the notebook content is submitted to the large language model for summary output. This ensures that the large language model's context window size requirements are met while fully utilizing all textual information relevant to the question.

[0067] It should be understood that although Figure 1 The steps in the flowchart are shown in the order indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0068] In one embodiment, Figure 5 , a long text retrieval device based on a graph structure is provided, comprising: an input module 502, a first graph structure generating module 504, a second graph structure generating module 506, a question processing module 508 and a retrieval output module 510, wherein:

[0069] Input module 502 is used to divide the input long text into blocks and guide the large language model to analyze the divided sub-texts to obtain text information of each sub-text; the text information includes core elements corresponding to words or phrases and atomic facts corresponding to sentence structures;

[0070] A first graph structure generating module 504 is configured to connect nodes of subtexts with similar core elements to obtain a first structure graph;

[0071] A second graph structure generating module 506 is configured to process the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities;

[0072] A question processing module 508 is configured to receive a question input by a user, process and optimize the question based on the large language model, and determine whether the question is a specific question or an abstract question;

[0073] The retrieval output module 510 is used to retrieve the question in the first structure diagram to obtain a reading queue when the question is a specific question, and generate notebook content based on the reading queue; when the question is an abstract question, retrieve the question in the second structure diagram to obtain notebook content containing answers and scores; and summarize and output the notebook content through a large language model.

[0074] In one embodiment, the input module 502 is configured to segment the input long text into blocks using predefined rules to obtain multiple semantically complete sub-texts.

[0075] In one embodiment, the input module 502 is also used to guide the large language model to analyze the segmented sub-text to obtain the core elements corresponding to the words or phrases in the sub-text and the atomic facts corresponding to the sentence structure; the core elements include: named entities, keywords, subject words and the relationship between named entities; the atomic facts represent atomic fact triples that reflect the sentence structure and the semantics of the sub-text.

[0076] In one embodiment, the second graph structure generating module 506 is configured to recursively perform community clustering on the first structure graph to generate a hierarchical structure of graph communities until the community size reaches a preset value, thereby obtaining a second structure graph.

[0077] In one embodiment, the second graph structure generating module 506 is configured to generate atomic facts of the community based on the atomic facts of each node in the second structure graph using a large language model.

[0078] In one embodiment, the question processing module 508 is also used to receive questions input by the user, rewrite and simplify the questions based on the large language model, and obtain an exploration plan for the optimized questions using a thinking chain guidance method; and determine whether the questions are specific questions or abstract questions based on the exploration plan.

[0079] In one embodiment, the retrieval output module 510 is also used to, when the question is a specific question, guide the large language model to determine the starting node to be retrieved in the first structure diagram according to the semantics of the question, and add the starting node to the reading queue; mark the node with an ID, and provide the node ID and the corresponding atomic fact to the agent, so that the agent reads the atomic fact to obtain an overview of the node; in the agent, use the pre-trained BERT small model to determine the neighboring nodes containing useful information of the starting node according to the question and the exploration plan, add the ID of the neighboring node containing useful information to the reading queue, traverse the original sub-text of each node according to the reading queue, read the nodes in the queue that meet the question and the exploration plan through the BERT small model, and add them to the notebook content; when the question is an abstract question, guide the large language model to answer the question in parallel based on the summary text of each community, and generate a value score for each answer; arrange the answers in descending order according to the value score, and add them to the notebook content.

[0080] The specific definition of the long text retrieval device based on a graph structure can be found in the definition of the long text retrieval method based on a graph structure above, and will not be repeated here. The various modules in the above-mentioned long text retrieval device based on a graph structure can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0081] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a long text retrieval method based on a graph structure is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0082] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0083] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0084] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0085] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0086] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A long text retrieval method based on graph structure, characterized in that: The method comprises: The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure; Connect the nodes of sub-texts with similar core elements to obtain the first structure graph; Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities; Receive a question input by a user, process and optimize the question based on the large language model, and confirm whether the question is specific or abstract; When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores. The notebook content is summarized and outputted through a large language model.

2. The method according to claim 1, characterized in that Split the input long text into chunks, including: The predefined rules are used to divide the input long text into blocks to obtain multiple semantically complete sub-texts.

3. The method according to claim 1, characterized in that Guide the large language model to analyze the segmented sub-texts to obtain the text information of each sub-text, including: Guide the large language model to analyze the segmented sub-texts to obtain the core elements corresponding to the words or phrases in the sub-texts and the atomic facts corresponding to the sentence structures; the core elements include: named entities, keywords, subject terms and the relationship between named entities; the atomic facts represent atomic fact triplets that reflect the sentence structure and the semantics of the sub-texts.

4. The method according to claim 1, wherein The first structure graph is processed using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities, including: The first structure graph is recursively processed using a community clustering algorithm to generate a hierarchical structure of graph communities until the community size reaches a preset value, thereby obtaining a second structure graph.

5. The method according to claim 4, characterized in that The method further comprises: The atomic facts of the community are generated by summarizing the atomic facts of each node in the second structural graph using a large language model.

6. The method according to any one of claims 1 to 5, characterized in that Receive a question input by the user, process and optimize the question based on the large language model, and confirm whether the question is a specific question or an abstract question, including: Receive user input questions, rewrite and simplify the questions based on the large language model, and use the thinking chain to guide the exploration plan for the optimized questions; Determine whether the problem is a specific problem or an abstract problem based on the exploration plan.

7. The method according to claim 6, characterized in that When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated based on the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores, including: When the question is a specific question, guiding the large language model to determine a starting node to be retrieved in the first structure graph according to the semantics of the question, and adding the starting node to a reading queue; Mark the node with an ID, and provide the node ID and the corresponding atomic fact to the agent, so that the agent reads the atomic fact to obtain the overview information of the node; In the agent, based on the question and the exploration plan, the pre-trained BERT mini-model is used to determine the neighboring nodes of the starting node that contain useful information. The IDs of the neighboring nodes containing useful information are added to the reading queue. The original subtext of each node is traversed according to the reading queue. The nodes in the queue that match the question and exploration plan are read by the BERT mini-model and added to the notebook content. When the question is abstract, the large language model is guided to answer the question in parallel based on the summary texts of each community, and generate a value score for each answer; Arrange the answers in descending order of their stated value scores and add them to the notebook contents.

8. A long text retrieval device based on graph structure, characterized in that: The device comprises: The input module is used to divide the input long text into blocks and guide the large language model to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to words or phrases and the atomic facts corresponding to sentence structures; A first graph structure generation module is used to connect nodes of sub-texts with similar core elements to obtain a first structure graph; A second graph structure generating module, configured to process the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities; A question processing module is used to receive questions input by users, process and optimize the questions based on the large language model, and confirm whether the questions are specific or abstract; The retrieval output module is used to retrieve the question in the first structural diagram to obtain a reading queue when the question is a specific question, and generate notebook content based on the reading queue; when the question is an abstract question, retrieve the question in the second structural diagram to obtain notebook content containing answers and scores; and summarize and output the notebook content through a large language model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images