Long text retrieval method and device based on graph structure and computer equipment

Abstract

The invention relates to a long text retrieval method and device based on a graph structure and computer equipment. The method comprises the following steps: partitioning an input long text, guiding a large language model to analyze partitioned sub-texts to obtain text information of each sub-text, connecting nodes of the sub-texts with similar core elements to obtain a first structure chart, processing the first structure chart by adopting a community clustering algorithm, and obtaining a second structure chart; receiving a question input by a user, processing and optimizing the question based on a large language model, determining whether the question is a specific question or an abstract question, and when the question is the specific question, retrieving the question in the first structure chart to obtain a reading queue; and generating notebook contents according to the reading queue, when the question is an abstract question, retrieving the question in the second structure chart to obtain the notebook contents containing answers and scores, and summarizing and outputting the notebook contents through a large language model.

Description

Technical Field

[0001] The present application relates to the field of text processing and retrieval technology, and in particular to a long text retrieval method, device and computer equipment based on a graph structure. Background Art

[0002] Long text processing capability refers to the ability of the model to process and analyze information that spans a large distance in the text, and is an important feature of large language models when understanding and generating natural language. For example, the information points of a text may be scattered in different paragraphs or chapters of the text, and this information is crucial to understanding the entire text or answering specific questions. The model needs to be able to track and associate these scattered information points, which includes not only understanding the surface meaning of the text, but also extracting deep semantic information, relationships, and structures.

[0003] There are two existing methods for processing long texts: model-level methods and agent-level methods. Model-level methods focus on improving the internal structure of large language models so that they can process long texts more effectively. This method usually includes: 1) enhancing the model's understanding of the order of words in the text by adjusting position embeddings; 2) using a modified attention mechanism to improve the model's ability to handle long-distance dependencies; 3) maintaining the relationship between different parts of a long text through specific alignment techniques, so that the model can better understand and process long texts, etc.; while agent-level methods focus on using retrieval-enhanced large language models or agents to process long texts, for example: 1) retrieving task-related information after segmenting long texts to assist in response generation; 2) using LLM as an agent to handle complex problems with its powerful planning and reflection capabilities; 3) organizing documents into a graph structure and exploring nodes and edges in the graph to collect necessary information, etc.

[0004] Both methods have their own advantages and disadvantages. Model-level methods usually require model training for texts of a specific length, which leads to high training costs and difficulty in building datasets. In addition, long-context large language models optimized using model-level methods often ignore key details in the long context, thus limiting their ability to handle complex tasks such as multi-hop problems. Although agent-level methods can handle long texts, they have limitations in capturing multi-hop reasoning, global problems, and long-distance information dependencies. When the text is too long, problems such as early information forgetting and missing details may occur. Summary of the invention

[0005] Based on this, it is necessary to provide a long text retrieval method, device and computer equipment based on graph structure to address the above technical problems.

[0006] A long text retrieval method based on a graph structure, the method comprising:

[0007] The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0008] Connect the nodes of the sub-texts with similar core elements to obtain a first structure graph;

[0009] Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community;

[0010] Receive a question input by a user, process and optimize the question based on a large language model, and confirm whether the question is a specific question or an abstract question;

[0011] When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue; when the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content including an answer and a score;

[0012] The notebook content is summarized and outputted through a large language model.

[0013] In one of the embodiments, the method further includes: dividing the input long text into blocks using a predefined rule to obtain a plurality of semantically complete sub-texts.

[0014] In one of the embodiments, it also includes: guiding the large language model to analyze the segmented sub-texts to obtain core elements corresponding to words or phrases in the sub-texts and atomic facts corresponding to sentence structures; the core elements include: named entities, keywords, subject terms, and relationships between named entities; the atomic facts represent atomic fact triplets that embody sentence structures and sub-text semantics.

[0015] In one of the embodiments, the method further includes: recursively performing a community clustering algorithm on the first structure graph to generate a hierarchical structure of the graph community until the community size reaches a preset value to obtain a second structure graph.

[0016] In one of the embodiments, it also includes: using a large language model to summarize the atomic facts of each node in the second structure graph to generate the atomic facts of the community.

[0017] In one of the embodiments, the method further includes: receiving a question input by a user, rewriting and simplifying the question based on a large language model, and obtaining an exploration plan for the optimized question by guiding the thinking chain; and determining whether the question is a specific question or an abstract question based on the exploration plan.

[0018] In one of the embodiments, it also includes: when the question is a specific question, guiding the large language model to determine the starting node to be retrieved in the first structure diagram according to the semantics of the question, and adding the starting node to the reading queue; marking the node with an ID, and providing the node ID and the corresponding atomic fact to the agent, so that the agent reads the atomic fact to obtain the overview information of the node; in the agent, using the pre-trained BERT small model to determine the neighboring nodes containing useful information of the starting node according to the question and the exploration plan, adding the ID of the neighboring node containing useful information to the reading queue, traversing the original sub-text of each node according to the reading queue, reading the nodes in the queue that meet the question and the exploration plan through the BERT small model, and adding them to the notebook content; when the question is an abstract question, guiding the large language model to answer the question in parallel according to the summary text of each community, and generating a value score for each answer; arranging the answers in descending order according to the value score, and adding them to the notebook content.

[0019] A long text retrieval device based on a graph structure, the device comprising:

[0020] An input module is used to divide the input long text into blocks, guide the large language model to analyze the sub-texts after the blocks, and obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0021] A first graph structure generating module, used for connecting nodes of sub-texts with similar core elements to obtain a first structure graph;

[0022] A second graph structure generating module, used to process the first structure graph by using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community;

[0023] A question processing module, used to receive questions input by users, process and optimize the questions based on the large language model, and confirm whether the questions are specific questions or abstract questions;

[0024] The retrieval output module is used to retrieve the question in the first structure diagram to obtain a reading queue when the question is a specific question, and generate notebook content according to the reading queue; when the question is an abstract question, retrieve the question in the second structure diagram to obtain notebook content containing an answer and a score; and summarize and output the notebook content through a large language model.

[0025] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0027] Connect the nodes of the sub-texts with similar core elements to obtain a first structure graph;

[0028] Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community;

[0029] Receive a question input by a user, process and optimize the question based on a large language model, and confirm whether the question is a specific question or an abstract question;

[0030] When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue; when the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content including an answer and a score;

[0031] The notebook content is summarized and outputted through a large language model.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0033] The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure;

[0034] Connect the nodes of the sub-texts with similar core elements to obtain a first structure graph;

[0035] Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community;

[0036] Receive a question input by a user, process and optimize the question based on a large language model, and confirm whether the question is a specific question or an abstract question;

[0037] When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue; when the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content including an answer and a score;

[0038] The notebook content is summarized and outputted through a large language model.

[0039] The above-mentioned long text retrieval method, device and computer equipment based on graph structure first divide the long text into blocks, and then analyze the sub-texts after the blocks to obtain the text information of each sub-text, thereby constructing a first structure graph, which contains core elements corresponding to words or phrases and atomic facts corresponding to sentence structures. The first structure graph is then enhanced to obtain a second structure graph, which has a hierarchical structure of a graph community. Finally, user questions are received, and the questions are optimized and processed based on a large language model to determine whether they are specific questions or abstract questions. If it is a specific question, the first structure graph is used for retrieval, and if it is an abstract question, the second structure graph is used for retrieval, and finally an answer output is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of a long text retrieval method based on a graph structure in one embodiment;

[0041] Figure 2 A schematic diagram of a process for analyzing a subtext in an embodiment;

[0042] Figure 3 A schematic diagram of a process for problem handling and optimization in one embodiment;

[0043] Figure 4 A schematic diagram of a flow chart of a long text retrieval method based on a graph structure in another embodiment;

[0044] Figure 5 is a structural block diagram of a long text retrieval device based on a graph structure in one embodiment;

[0045] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In one embodiment, Figure 1 As shown, a long text retrieval method based on a graph structure is provided, comprising the following steps:

[0048] Step 102, divide the input long text into blocks, guide the large language model to analyze the divided sub-texts, and obtain the text information of each sub-text.

[0049] Text information includes core elements corresponding to words or phrases and atomic facts corresponding to sentence structures.

[0050] Step 104, connecting the nodes of the sub-texts with similar core elements to obtain a first structure graph.

[0051] Step 106: Process the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities.

[0052] Step 108, receiving the question input by the user, processing and optimizing the question based on the large language model, and confirming whether the question is a specific question or an abstract question.

[0053] Step 110, when the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores.

[0054] Step 112: Summarize and output the notebook content through a large language model.

[0055] In the above-mentioned long text retrieval method based on graph structure, the long text is first divided into blocks, and then the sub-texts after the division are analyzed to obtain the text information of each sub-text, so as to construct a first structure graph. The first structure graph contains core elements corresponding to words or phrases and atomic facts corresponding to sentence structures. The first structure graph is then enhanced to obtain a second structure graph. The second structure graph has a hierarchical structure of a graph community. Finally, user questions are received, and the questions are optimized and processed based on a large language model to determine whether they are specific questions or abstract questions. If they are specific questions, the first structure graph is used for retrieval. If they are abstract questions, the second structure graph is used for retrieval, and finally the answer output is generated.

[0056] In one embodiment, a predefined rule is used to divide the input long text into blocks to obtain multiple semantically complete sub-texts. The text block is the basic unit used in the subsequent graph technology and also serves as a source reference for extracting knowledge items.

[0057] In one embodiment, the large language model is guided to analyze the sub-text after segmentation to obtain the core elements corresponding to the words or phrases in the sub-text and the atomic facts corresponding to the sentence structure; the core elements include: named entities, keywords, subject terms and the relationship between named entities; the atomic facts represent the atomic fact triples that reflect the sentence structure and the semantics of the sub-text. The first structure diagram is used as a knowledge base to answer general and specific questions.

[0058] like Figure 2As shown, if the original text is "Machine learning means that computers learn from data and then use experience to improve their own performance. The algorithm will be continuously trained to discover patterns and correlations from large data sets, and then make the best decisions and predictions based on the results of data analysis", the text is analyzed by a large language model, and the core element obtained is the concept of machine learning, and the atomic fact obtained is: by analyzing large amounts of data, machine learning algorithms can identify patterns, establish correlations, and use this knowledge to improve their own performance and make the best predictions.

[0059] In one of the embodiments, a community clustering algorithm is used to recursively perform the first structure graph to generate a hierarchical structure of the graph community until the community size reaches a preset value, thereby obtaining a second structure graph.

[0060] Specifically, a recursive community clustering algorithm is applied to the constructed first structure graph using a hierarchical Leiden algorithm to generate a hierarchical structure of the graph community until a threshold of community size is reached. The threshold size is set to 5 to 10 by default according to the number of nodes in the actual graph, the semantic complexity, and the allocation of computing resources, thereby providing a method for navigating and summarizing graphs at different granularity levels. The second structure graph serves as a knowledge base for answering abstract and global questions.

[0061] In one embodiment, a large language model is used to generate atomic facts of a community based on the atomic facts of each node in the second structure graph. The atomic facts of the community together constitute a high-level understanding of the global structure and semantics of the entire long text.

[0062] In one of the embodiments, a question input by a user is received, the question is rewritten and simplified based on a large language model, and an exploration plan is obtained for the optimized question using a thinking chain-guided approach; based on the exploration plan, it is determined whether it is a specific question or an abstract question.

[0063] like Figure 3 As shown, taking the question input by the user as "Does rag require an internet connection?" as an example, the question is rewritten and simplified through the large language model. Specifically, the question is rewritten to retain the core elements of the original question, while streamlining and optimizing it to make the question more formal and written. The optimized question is "Does the use of RAG retrieval enhancement generation technology require a network environment?" The optimized question is then guided by a chain of thoughts to obtain an exploration plan, which can be specifically "We need to first identify the RAG or retrieval enhancement generation technology, then find its usage conditions, and then determine whether the usage conditions include a network connection." Finally, based on the exploration plan, it is determined whether it is a specific question or an abstract question. In this example, it is determined to be "This question is a general and specific question, and the first structure diagram is retrieved."

[0064] In another embodiment, when the question is a specific question, the large language model is guided to determine the starting node to be retrieved in the first structure diagram according to the semantics of the question, and the starting node is added to the reading queue; the node is marked with an ID, and the ID of the node and the corresponding atomic fact are provided to the agent, so that the agent reads the atomic fact to obtain the overview information of the node; in the agent, the neighboring nodes containing useful information of the starting node are determined by using the pre-trained BERT small model according to the question and the exploration plan, and the ID of the neighboring node containing useful information is added to the reading queue, and the original subtext of each node is traversed according to the reading queue, and the nodes in the queue that meet the question and the exploration plan are read by the BERT small model, and added to the notebook content. The whole process is a dynamic, iterative feedback process, and the agent continuously collects information, updates the notebook content, and makes decisions based on the current understanding and problem requirements. This process allows the agent to process long text information in a structured and goal-oriented manner, effectively improving the understanding and reasoning ability of long texts.

[0065] When the question is abstract, the large language model is guided to answer the question in parallel based on the summary text of each community and generate a value score for each answer; community answers with scores below a certain threshold will be filtered out, sorted in descending order according to the value score, and added to the notebook content until the large language model context window size limit is reached.

[0066] Specifically, the present invention is as follows Figure 4As shown in the graph construction stage, the agent will first divide the long text into small text blocks according to the predefined semantic segmentation strategy, and each text block contains a certain degree of complete semantic information. Then guide the large language model to extract core elements and summarize atomic facts from the text blocks to obtain the first structure graph, that is, the graph structure for general and specific question answers; further enhance the graph structure of the first structure graph, and divide the graph community through the Leiden algorithm to obtain the second structure graph, that is, the graph structure for global and abstract question answers. In the graph retrieval stage, the agent will first use the large language model to optimize and classify the user's questions, align the user's questions with the model's own capabilities, and divide the questions into two categories: specific questions and abstract questions. For specific questions, search on the first structure graph: use the large language model to select the starting node, then use the first structure graph node retrieval module to select the appropriate node and put it into the reading queue, update the notebook through the first structure graph content expansion module, and continuously feed back to the exploration plan for updating and continuously improving the notebook content; for abstract questions, search on the second structure graph: use the large language model to complete the node community division, use the second structure graph community retrieval module to obtain the answers and scores of each community for the question, and then use the second structure graph content expansion module to improve the notebook content. In the answer generation stage, the notebook content will be handed over to the large language model for summary output. At this point, it not only meets the requirements of the large language model context window size, but also can make full use of all text information related to the question.

[0067] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequentially, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0068] In one embodiment, Figure 5 In the embodiment, a long text retrieval device based on a graph structure is provided, comprising: an input module 502, a first graph structure generating module 504, a second graph structure generating module 506, a question processing module 508 and a retrieval output module 510, wherein:

[0069] An input module 502 is used to divide the input long text into blocks, guide the large language model to analyze the sub-texts after the blocks, and obtain text information of each sub-text; the text information includes core elements corresponding to words or phrases and atomic facts corresponding to sentence structures;

[0070] A first graph structure generating module 504 is used to connect nodes of sub-texts with similar core elements to obtain a first structure graph;

[0071] A second graph structure generating module 506, configured to process the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of graph communities;

[0072] A question processing module 508 is used to receive a question input by a user, process and optimize the question based on a large language model, and confirm whether the question is a specific question or an abstract question;

[0073] The retrieval output module 510 is used to retrieve the question in the first structure diagram to obtain a reading queue when the question is a specific question, and generate notebook content according to the reading queue; when the question is an abstract question, retrieve the question in the second structure diagram to obtain notebook content including an answer and a score; and summarize and output the notebook content through a large language model.

[0074] In one embodiment, the input module 502 is used to divide the input long text into blocks using predefined rules to obtain multiple semantically complete sub-texts.

[0075] In one embodiment, the input module 502 is also used to guide the large language model to analyze the segmented sub-texts to obtain core elements corresponding to words or phrases in the sub-texts and atomic facts corresponding to sentence structures; the core elements include: named entities, keywords, subject terms, and the relationship between named entities; the atomic facts represent atomic fact triplets that embody sentence structures and sub-text semantics.

[0076] In one embodiment, the second graph structure generating module 506 is used to recursively perform community clustering algorithm on the first structure graph to generate a hierarchical structure of graph communities until the community size reaches a preset value to obtain a second structure graph.

[0077] In one embodiment, the second graph structure generating module 506 is used to generate the atomic facts of the community according to the atomic facts of each node in the second structure graph by using a large language model.

[0078] In one embodiment, the question processing module 508 is also used to receive questions input by users, rewrite and simplify the questions based on the large language model, and obtain an exploration plan for the optimized questions using a thinking chain guidance method; determine whether the question is a specific question or an abstract question based on the exploration plan.

[0079] In one embodiment, the retrieval output module 510 is also used to guide the large language model to determine the starting node to be retrieved in the first structure diagram according to the semantics of the question when the question is a specific question, and add the starting node to the reading queue; mark the node with an ID, and provide the node ID and the corresponding atomic fact to the agent, so that the agent reads the atomic fact to obtain the overview information of the node; in the agent, use the pre-trained BERT small model to determine the neighboring nodes containing useful information of the starting node according to the question and the exploration plan, add the ID of the neighboring node containing useful information to the reading queue, traverse the original sub-text of each node according to the reading queue, read the nodes in the queue that meet the question and the exploration plan through the BERT small model, and add them to the notebook content; when the question is an abstract question, guide the large language model to answer the question in parallel according to the summary text of each community, and generate a value score for each answer; arrange the answers in descending order according to the value score, and add them to the notebook content.

[0080] For the specific definition of the long text retrieval device based on graph structure, please refer to the definition of the long text retrieval method based on graph structure above, which will not be repeated here. Each module in the above-mentioned long text retrieval device based on graph structure can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0081] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a long text retrieval method based on a graph structure is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0082] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0083] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0084] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0086] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A long text retrieval method based on graph structure, characterized in that: The method comprises: The input long text is divided into blocks, and the large language model is guided to analyze the sub-texts after the blocks are divided to obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure; Connect the nodes of the sub-texts with similar core elements to obtain a first structure graph; Processing the first structure graph using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community; Receive a question input by a user, process and optimize the question based on a large language model, and confirm whether the question is a specific question or an abstract question; When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue; when the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content including an answer and a score; The notebook content is summarized and outputted through a large language model.

2. The method according to claim 1, characterized in that Divide the input long text into chunks, including: The predefined rules are used to divide the input long text into blocks to obtain multiple semantically complete sub-texts.

3. The method according to claim 1, characterized in that Guide the large language model to analyze the divided sub-texts to obtain the text information of each sub-text, including: Guide the large language model to analyze the segmented sub-texts to obtain the core elements corresponding to the words or phrases in the sub-texts and the atomic facts corresponding to the sentence structures; the core elements include: named entities, keywords, subject terms and the relationship between named entities; the atomic facts represent atomic fact triplets that embody the sentence structures and the semantics of the sub-texts.

4. The method according to claim 1, characterized in that: The first structure graph is processed by using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community, including: The first structure graph is recursively processed using a community clustering algorithm to generate a hierarchical structure of graph communities until the community size reaches a preset value, thereby obtaining a second structure graph.

5. The method according to claim 4, characterized in that The method further comprises: The atomic facts of the community are generated by summarizing the atomic facts of each node in the second structural graph using a large language model.

6. The method according to any one of claims 1 to 5, characterized in that: Receive a question input by a user, process and optimize the question based on a large language model, and confirm whether the question is a specific question or an abstract question, including: Receive questions input by users, rewrite and simplify the questions based on the large language model, and use the thinking chain to guide the exploration plan for the optimized questions; It is determined whether the problem is a specific problem or an abstract problem based on the exploration plan.

7. The method according to claim 6, characterized in that When the question is a specific question, the question is retrieved in the first structure diagram to obtain a reading queue, and notebook content is generated according to the reading queue. When the question is an abstract question, the question is retrieved in the second structure diagram to obtain notebook content containing answers and scores, including: When the question is a specific question, guiding the large language model to determine a starting node to be retrieved in the first structure graph according to the semantics of the question, and adding the starting node to a reading queue; Mark the node with an ID, and provide the node ID and the corresponding atomic fact to the agent, so that the agent reads the atomic fact to obtain the overview information of the node; In the agent, according to the question and the exploration plan, the pre-trained BERT small model is used to determine the neighbor nodes of the starting node that contain useful information, the IDs of the neighbor nodes that contain useful information are added to the reading queue, the original subtext of each node is traversed according to the reading queue, the nodes in the queue that meet the question and the exploration plan are read by the BERT small model, and added to the notebook content; When the question is an abstract question, the large language model is guided to answer the question in parallel based on the summary texts of each community, and a value score is generated for each answer; The answers are sorted in descending order of the stated value score and added to the notebook contents.

8. A long text retrieval device based on graph structure, characterized in that: The device comprises: An input module is used to divide the input long text into blocks, guide the large language model to analyze the sub-texts after the blocks, and obtain the text information of each sub-text; the text information includes the core elements corresponding to the words or phrases and the atomic facts corresponding to the sentence structure; A first graph structure generating module, used for connecting nodes of sub-texts with similar core elements to obtain a first structure graph; A second graph structure generating module, used to process the first structure graph by using a community clustering algorithm to obtain a second structure graph having a hierarchical structure of a graph community; A question processing module, used to receive questions input by users, process and optimize the questions based on the large language model, and confirm whether the questions are specific questions or abstract questions; The retrieval output module is used to retrieve the question in the first structure diagram to obtain a reading queue when the question is a specific question, and generate notebook content according to the reading queue; when the question is an abstract question, retrieve the question in the second structure diagram to obtain notebook content containing an answer and a score; and summarize and output the notebook content through a large language model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images