Question and answer method and device based on large language model, electronic equipment and storage medium

Abstract

This application provides a question-answering method, apparatus, electronic device, and storage medium based on a large language model. The method includes: receiving a question statement; using a large language model to obtain a question entity and a question entity profile; obtaining a first information entity matching the question entity based on a multi-scale information database constructed from information entities, profile summaries, and entity profiles; obtaining a first profile summary based on the data relationship tags of the first information entity, and reading the data relationship tags of the first profile summary to obtain a first entity profile; obtaining a candidate text set based on the first profile summary and the first entity profile using text selection rules; obtaining first text information that satisfies the text summary extraction rules based on the candidate text set; and generating an answer statement based on the first text information and the question statement using a large language model. This application improves question-answering retrieval efficiency, response efficiency, and accuracy through multi-level semantic parsing, information association, and text optimization.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a question-answering method, apparatus, electronic device, and storage medium based on a large language model. Background Technology

[0002] With the rapid development of information technology, people's demand for efficient knowledge acquisition and problem-solving is growing. Question-answering systems (QA) can provide users with instant and accurate answers through natural language understanding technology, improving the efficiency of information retrieval. Compared with traditional search engines, QA systems can not only retrieve relevant information but also directly generate answers that meet user needs through deep semantic analysis, reducing the cognitive burden on users. For example, in enterprise applications, QA systems can help employees quickly access information from the company's internal knowledge base, optimize business processes, and improve work efficiency. The widespread application of QA systems has driven the digital transformation of many industries and changed the way people obtain information and solve problems.

[0003] In existing technologies, question-answering systems primarily rely on information retrieval and natural language processing (NLP) techniques. Common methods include rule-based and template matching, as well as semantic analysis-based retrieval. Early question-answering systems used predefined rules or templates to perform pattern matching on user questions, directly returning corresponding preset answers. This method was simple to implement and fast in response, but it could only handle questions of fixed format and limited scope, resulting in poor scalability. With the development of NLP technology, question-answering systems have gradually adopted semantic analysis and entity extraction techniques. By identifying the core elements in user questions, the level of question parsing has been improved, laying the foundation for more accurate information retrieval. Based on this, information retrieval becomes a crucial step in question-answering systems, typically searching for content related to the user question in a pre-established knowledge base or text database. Candidate answers are located using keyword or semantic matching techniques, then sorted and filtered to output the optimal solution.

[0004] However, when faced with extremely long documents, traditional information retrieval methods require calculating the similarity between each segment of the text and the user's question. This increases computational load, prolongs system response time, and negatively impacts user experience. Therefore, there is an urgent need for a question-answering method based on a large language model to address the low retrieval efficiency of existing technologies. Summary of the Invention

[0005] This application provides a question-answering method, apparatus, electronic device, and storage medium based on a large language model to solve the technical problem of low efficiency in question-answering retrieval in the prior art.

[0006] Firstly, this application provides a question-answering method based on a large language model, including:

[0007] Receive the question statement to be answered, and use a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity;

[0008] Based on a multi-scale information database, a first information entity matching the problem entity is retrieved and obtained; wherein, the multi-scale information database is constructed from information entities, profile summaries, and entity profiles;

[0009] Based on the data relationship tags carried by the first information entity, obtain the first profile summary corresponding to the first information entity, and read the data relationship tags carried by the first profile summary to obtain the first entity profile corresponding to the first profile summary;

[0010] Based on the problem entity profile, the first profile summary, and the first entity profile, a candidate text set is obtained using preset text selection rules; wherein, the candidate text set includes the first profile summary and / or the first entity profile.

[0011] Based on all text information in the candidate text set, first text information that meets the text summary extraction rules is obtained. Based on the first text information and the question statement to be answered, a large language model is used to generate and output the answer statement that matches the question statement to be answered, thus completing the question-and-answer process.

[0012] Optionally, constructing the multi-scale information database using the method described above includes:

[0013] Based on the knowledge document, multiple text blocks are obtained, and a large language model is used to perform semantic parsing on each text block to extract one or more information entities; wherein each information entity is a proper noun or object in the text block;

[0014] Based on each information entity, and according to the text block corresponding to the information entity, a large language model is used to generate one or more entity profiles of the information entity; wherein, the entity profile includes at least one of the following: feature information, background information, and conclusion information related to the information entity;

[0015] Based on the entity profile, a large language model is used to extract a summary from the entity profile to obtain a profile summary corresponding to each entity profile.

[0016] Based on the correspondence between the information entity, the portrait summary, and the entity portrait, using the information entity, the portrait summary, and the entity portrait as nodes, intra-layer node connection edges are constructed for the information entity layer, the portrait summary layer, and the entity portrait layer, respectively; inter-layer node connection edges are constructed for the information entity layer and the portrait summary layer; and inter-layer node connection edges are constructed for the portrait summary layer and the entity portrait layer.

[0017] Each node, each intra-layer node connection edge, and each inter-layer node connection edge are encoded using a preset data format to serialize all nodes, all intra-layer node connection edges, and all inter-layer node connection edges.

[0018] Record the intra-layer node connection edges and inter-layer node connection edges of the node as the data relationship labels of the node, traverse all the nodes, obtain the data relationship label of each node, and store all the nodes and the data relationship labels of each node in a multi-scale information database.

[0019] Optionally, in the method described above, obtaining multiple text blocks based on the knowledge document includes:

[0020] The document structure of the knowledge file is parsed to identify and divide the structural information of the knowledge file; wherein, the structural information includes a title item, a summary item, a table of contents item, a main text item, and an appendix item;

[0021] Obtain the main text item, and extract the nested chapters and subsections according to the table of contents item to generate the initial text;

[0022] Each initial text is preprocessed, and based on the preprocessed initial text, a preset character length threshold is used to segment the initial text to obtain multiple text blocks.

[0023] Optionally, in the method described above, the multi-scale information database includes a graph database and a vector database;

[0024] The data relationship labels for the nodes are recorded as the intra-layer node connection edges and inter-layer node connection edges. All nodes are traversed to obtain the data relationship label for each node, and all nodes and their data relationship labels are stored in a multi-scale information database, including:

[0025] Based on all the image summaries and their data relationship tags, as well as all the entity images and their data relationship tags, the topology of the image summary layer and the entity image layer is obtained, and the topology is stored in the graph database.

[0026] Based on the information entities in the information entity layer, a preset embedding model is used to obtain the information entity vector corresponding to each information entity, and all information entities, the information entity vectors, and the data relationship tags carried by each information entity are stored in the vector database.

[0027] Based on a multi-scale information database, a first information entity matching the problem entity is retrieved and obtained, including:

[0028] Based on the problem entity, a preset embedding model is used to obtain the problem entity vector corresponding to the problem entity;

[0029] In the vector database, based on vector similarity, a first information entity vector matching the question entity vector is retrieved and obtained, so as to obtain the first information entity matching the question entity.

[0030] Optionally, in the method described above, obtaining a set of candidate texts based on the problem entity profile, the first profile summary, and the first entity profile using preset text selection rules includes:

[0031] A large language model is used to obtain the semantic similarity between each first profile summary and the problem entity profile;

[0032] After determining that the semantic similarity between the first profile summary and the problem entity profile is greater than or equal to the second similarity threshold and less than the first similarity threshold, based on the data relationship tags carried by the first profile summary, the second entity profile corresponding to the first profile summary is selected from the first entity profile.

[0033] Using a large language model, the semantic similarity between each second entity profile and the question entity profile is obtained. After determining that the semantic similarity between the second entity profile and the question entity profile is greater than or equal to a third similarity threshold, the second entity profile is marked as candidate text; or...

[0034] After determining that the semantic similarity between the first profile summary and the problem entity profile is greater than or equal to the first similarity threshold, the first profile summary is marked as candidate text;

[0035] Iterate through the first image summary and obtain the candidate text set based on the second entity image and / or the first image summary.

[0036] Optionally, in the method described above, obtaining the first text information that satisfies the text summary extraction rules based on all text information in the candidate text set includes:

[0037] Based on the similarity between the text information and the problem entity profile, the text information is sorted in descending order of similarity to obtain a text information sequence; wherein, the text information is a first profile summary or a second entity profile.

[0038] According to the text summary extraction rules, multiple text information to be processed are sequentially obtained from the text information sequence in descending order of quantity threshold;

[0039] If the total length of the multiple text messages to be processed is greater than a preset length threshold, then a summary extraction is performed on the multiple text messages to be processed to obtain a first text message that meets the preset length threshold; or...

[0040] If the total length of the multiple text messages to be processed is less than or equal to the preset length threshold, then the multiple text messages to be processed are concatenated and recorded as the first text message;

[0041] Based on the first text information and the question statement to be answered, a large language model is used to generate and output an answer statement that matches the question statement to be answered, including:

[0042] The first text information and the question statement to be answered are input into the large language model. Based on the core question of the question statement to be answered, the first text information is semantically processed and supplemented to generate an answer statement that is consistent with the semantic information of the question statement to be answered, and the answer statement is output.

[0043] Optionally, in the method described above, receiving the question statement to be answered and using a large language model to obtain the question entities of the question statement to be answered and the question entity profile corresponding to each question entity includes:

[0044] The system receives a question input from a user, performs semantic parsing on the question using a large language model, and extracts one or more question entities.

[0045] Based on the context information of each question entity and the question statement to be answered, a large language model is used to obtain the question entity profile corresponding to the question entity.

[0046] Secondly, this application provides a question-and-answer device, comprising:

[0047] The question parsing module is used to receive the question statement to be answered and use a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity;

[0048] The information retrieval module is used to retrieve and obtain a first information entity that matches the problem entity based on a multi-scale information database; wherein, the multi-scale information database is constructed from information entities, profile summaries, and entity profiles;

[0049] The information acquisition module is used to acquire a first profile summary corresponding to the first information entity based on the data relationship tags carried by the first information entity, and to read the data relationship tags carried by the first profile summary to acquire a first entity profile corresponding to the first profile summary.

[0050] The information filtering module is used to obtain a candidate text set based on the problem entity profile, the first profile summary, and the first entity profile, using preset text selection rules; wherein, the candidate text set includes the first profile summary and / or the first entity profile;

[0051] The question-and-answer generation module obtains first text information that meets the text summary extraction rules based on all text information in the candidate text set, and generates and outputs an answer statement that matches the question statement based on the first text information and the question statement to be answered using a large language model, thus completing the question-and-answer process.

[0052] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0053] The memory stores computer-executed instructions;

[0054] The processor executes computer execution instructions stored in the memory to implement the question-and-answer method described above.

[0055] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described question-and-answer method.

[0056] This application provides a question-answering method, apparatus, electronic device, and storage medium based on a large language model. By receiving the question statement to be answered and utilizing the large language model to obtain the question entity and its corresponding question entity profile, accurate semantic parsing of the user's question is achieved. This process ensures that user needs are clearly expressed in a structured form, providing a clear direction for subsequent information retrieval and processing. Based on a multi-scale information database, the first information entity matching the question entity is retrieved, further enhancing the positioning capability in complex knowledge graphs. The construction of the multi-scale information database enables efficient association of information entities, profile summaries, and entity profiles through hierarchical relationships, thereby ensuring the accuracy and comprehensiveness of the retrieval results. By reading the relationship tags between the first information entity and the first profile summary, detailed information related to the user's question can be obtained layer by layer, including the profile summary and entity profile. This multi-layered information association mechanism allows for comprehensive consideration of information at different granularities when answering questions, improving the depth and information coverage of question answering.

[0057] When applying text selection rules based on the first profile summary and the first entity profile, text information highly relevant to the user's question can be dynamically filtered. Through flexible text selection rules, the core content of the candidate text set can be efficiently extracted, ensuring information quality during the question-and-answer process. This process improves the efficiency of question-and-answer when processing large-scale text, while ensuring the accuracy and relevance of the answers. Finally, by inputting the text information along with the user's question into a large language model, answer statements matching the question are generated. This process utilizes the model's language generation capabilities to perform semantic processing and information integration on the text information, ensuring that the output answer statements are both accurate and contextually coherent. This automated answer generation mechanism improves the response speed and user experience of question-and-answer, enabling the rapid provision of high-quality answers. In summary, this application improves the accuracy, relevance, and response efficiency of question-and-answer through multi-level semantic parsing, information association, and text optimization. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] Figure 1 A flowchart illustrating the question-answering method based on a large language model provided in this application embodiment;

[0060] Figure 2 This is a schematic flowchart of a method for constructing a multi-scale information database according to an embodiment of this application;

[0061] Figure 3 This is a schematic diagram of the structure of the retrieval graph provided in the embodiments of this application;

[0062] Figure 4 A flowchart illustrating a method for obtaining multiple text blocks of a knowledge document, provided in an embodiment of this application.

[0063] Figure 5 This is a schematic diagram of a method for storing all nodes and data relationship labels into a multi-scale information database, as provided in an embodiment of this application.

[0064] Figure 6 This is a schematic flowchart of a method for obtaining a first information entity provided in an embodiment of this application;

[0065] Figure 7 This is a schematic flowchart of a method for obtaining a candidate text set provided in an embodiment of this application;

[0066] Figure 8 This is a schematic flowchart of a method for obtaining first text information provided in an embodiment of this application;

[0067] Figure 9 This is a schematic diagram of the method for obtaining problem entities and problem entity profiles provided in an embodiment of this application;

[0068] Figure 10 This is a schematic diagram of the structure of the question-and-answer device provided in the embodiments of this application;

[0069] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0070] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0073] With the development of natural language processing technology, question-answering systems have gradually incorporated semantic analysis and entity extraction techniques. By performing semantic analysis on the user's input question, key elements such as entity names, time, and location can be identified. Simultaneously, entity extraction techniques can extract core information from the user's question, ensuring the accuracy of subsequent retrieval processes. This method improves the intelligence level of the question-answering system, but it still relies on subsequent information retrieval and answer selection steps. In the information retrieval stage, the question-answering system typically searches for content related to the user's question in a pre-established knowledge base or text library. The knowledge base may include structured databases, unstructured document collections, etc. Candidate answers are located from the knowledge base using keyword or semantic matching techniques. However, the information retrieval results often contain a large amount of content with varying relevance, thus requiring further ranking and selection of candidate answers. The ranking and selection steps are usually based on similarity calculations or other scoring mechanisms, ranking candidate answers according to their semantic relevance to the user's question and selecting the optimal answer for output. The performance of this stage directly affects the final result of the question-answering system. However, when processing long documents or complex questions, it is necessary to calculate the similarity between each segment of text and the user's question. This increases the computational load and prolongs system response time, impacting user experience. Clearly, existing technologies suffer from inefficient question-answering and inaccurate answers.

[0074] Based on the aforementioned technical problems and needs, the inventive concept of this application is to provide a question-answering method based on a large language model to achieve efficient parsing and accurate answers to complex questions. By receiving the user's input question, the method first uses a large language model to perform semantic parsing, extracting question entities and their corresponding semantic profiles. This approach not only accurately captures the core information of the user's question but also lays the foundation for subsequent information retrieval and semantic matching. In the information retrieval stage, the extracted question entities are matched based on a multi-scale information database. This database is constructed from information entities, profile summaries, and entity profiles, storing and managing a large amount of related information through a hierarchical structure. By analyzing the first information entity and its associated data relationship tags, the first profile summary and first entity profile related to the question are obtained layer by layer. This progressive information extraction mechanism ensures that the most relevant content can be extracted from multiple angles and levels.

[0075] To further improve the accuracy of question answering, a candidate text set is generated from the first profile summary and the first entity profile according to preset text selection rules. This rule flexibly adapts to different question requirements and can extract key information from large amounts of text. By filtering and summarizing the content of the candidate text set, first text information that meets the text summarization extraction rules is generated. Finally, the first text information and the question to be answered are input into the large language model to generate an answer statement that matches the user's question. In this process, the large language model fully leverages its powerful semantic generation and information supplementation capabilities, ensuring that the output answer not only responds to the user's needs but also possesses fluent and natural language expression.

[0076] The technical solution of this application is applicable to scenarios requiring the rapid extraction and processing of information from large amounts of complex documents. Specifically, in enterprise internal knowledge management, this technology can help employees quickly find the answers they need from the massive amounts of technical documents, project reports, and internal guidelines accumulated by the enterprise, improving work efficiency. Furthermore, in the customer service field, it can be used to build intelligent customer service platforms, supporting customers to ask natural language questions and providing accurate answers instantly, improving customer experience and reducing the workload of human customer service. In the medical field, this technology can assist doctors and medical researchers in quickly retrieving relevant information from medical literature, case databases, and guidelines, helping them make more accurate diagnostic and treatment decisions. Similarly, it has wide applications in the legal field, capable of extracting legal provisions or case analyses related to user questions from laws, regulations, and precedents, providing efficient information support for lawyers and legal practitioners. Overall, the application scenarios of this application cover multiple industries and fields requiring efficient processing and utilization of large-scale text information. Through its accurate information extraction and question-answering capabilities, it can provide powerful support to users in various complex scenarios, improving information acquisition and decision-making efficiency.

[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0078] Figure 1 This is a flowchart illustrating the question-answering method based on a large language model provided in an embodiment of this application. Figure 1 As shown, this question-answering method based on a large language model includes:

[0079] S11: Receive the question statement to be answered, and use a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity.

[0080] In this embodiment, the system first receives a question statement input by the user. This question statement may cover various fields and topics, such as economic analysis, technological trends, and markets. To accurately understand the user's question, semantic parsing is required. Here, a large language model is used to process the question statement. A large language model is a pre-trained model based on deep learning, possessing natural language understanding and generation capabilities, and able to learn the complex structure and semantic relationships of language from large-scale corpora. In this embodiment, the large language model is used to perform semantic analysis on the question statement to extract the question entities. Question entities refer to the key concepts, proper nouns, or core objects involved in the question statement, representing the main information that the user is concerned about. For example, in the question "How to evaluate the development trend of artificial intelligence technology?", "artificial intelligence technology" is a question entity. By identifying question entities, the user's focus can be clarified, which is helpful for subsequent information retrieval and matching.

[0081] After obtaining the question entities, a large language model is further employed to obtain a question entity profile for each question entity. The question entity profile is a detailed description of the question entity, including its characteristics, background information, relevant attributes, and development status. The purpose of obtaining the question entity profile is to understand the query direction regarding the question entity within the question statement. Through these steps, a comprehensive semantic analysis of the question statement to be answered is performed, obtaining the question entities and their corresponding question entity profiles. This process enhances the understanding of user needs (i.e., the statement to be answered) and ensures that information relevant to the user's question can be accurately retrieved in subsequent steps.

[0082] S12, based on a multi-scale information database, retrieve and obtain the first information entity that matches the problem entity; wherein, the multi-scale information database is constructed from information entities, profile summaries and entity profiles.

[0083] In this embodiment, a first information entity matching the question entity is retrieved and obtained based on a multi-scale information database. The multi-scale information database is constructed from information entities, profile summaries, and entity profiles, possessing a rich hierarchical structure and relationships. For example, an embedding model is used to vectorize the question entity, generating a corresponding question entity vector. The embedding model can be a natural language processing model such as Word2Vec or BERT, capable of mapping text data to a high-dimensional vector space to capture the semantic and grammatical features of words. Next, the information entity closest to the question entity vector is retrieved from the multi-scale information database. That is, the information entity is also stored in vector form. In this way, the first information entity with the highest semantic relevance to the question entity can be efficiently selected from a large number of information entities. Therefore, by retrieving the first information entity matching the question entity from the multi-scale information database, accurate location of the user's question is achieved. Utilizing the vectorized representation of information entities ensures the rapid retrieval of core information related to the user's question within a vast knowledge base, laying the foundation for generating accurate answers. This process improves the speed and accuracy of question-and-answer response and enhances the user experience.

[0084] S13, based on the data relationship tags carried by the first information entity, obtain the first profile summary corresponding to the first information entity, and read the data relationship tags carried by the first profile summary to obtain the first entity profile corresponding to the first profile summary.

[0085] In this embodiment, a first profile summary corresponding to the first information entity is obtained based on the data relationship tags carried by the first information entity. Data relationship tags are identifiers of the association relationships between nodes in a multi-scale information database, used to describe the intra-layer and inter-layer connections between nodes such as information entities, profile summaries, and entity profiles. For example, these tags contain information such as references, dependencies, and subordinate relationships between nodes, facilitating efficient location of relevant nodes in complex database structures. Specifically, the data information of the first information entity is first read, including its own attributes and the data relationship tags it carries. By parsing these data relationship tags, the profile summary directly associated with the information entity can be identified. Since the data relationship tags explicitly indicate the correspondence between the information entity and the profile summary, the first profile summary can be quickly obtained without traversing the entire multi-scale information database. This direct association retrieval method improves the efficiency of data acquisition.

[0086] After obtaining the first profile summary, the data relationship tags carried by this summary are read. These tags record the associations between the profile summary and more detailed information (i.e., entity profiles). By parsing these data relationship tags, the first entity profile corresponding to the first profile summary can be further located. The entity profile contains a comprehensive description of the information entity, including features, background information, and conclusions. By utilizing the data relationship tags carried by nodes, a hierarchical retrieval from information entities to profile summaries and then to entity profiles is achieved in a multi-scale information database. This hierarchical retrieval method avoids processing large amounts of irrelevant data, improving retrieval efficiency and accuracy. Simultaneously, it ensures that the obtained information is highly relevant to the user's question, laying a solid foundation for subsequent text selection and answer generation. In summary, through the above steps, not only is the response speed improved, but the accuracy and credibility of the final answer are enhanced by obtaining hierarchical detailed information.

[0087] S14. Based on the first portrait summary and the first entity portrait, a set of candidate texts is obtained using preset text selection rules; wherein, the set of candidate texts includes the first portrait summary and / or the first entity portrait.

[0088] In this embodiment, based on the previously obtained first profile summary and first entity profile, a candidate text set is obtained using preset text selection rules. The preset text selection rules are used to filter out content highly relevant to the question statement from a large amount of text information. For example, filtering is based on semantic similarity, ensuring that only text information sufficiently related to the question statement is selected into the candidate text set. This method effectively filters out text irrelevant to the question or with low relevance, ensuring the quality and conciseness of the candidate text set. The candidate text set includes the first profile summary and / or the first entity profile, depending on their semantic relevance to the question entity profile and the determination result of the preset text selection rules. This filtering mechanism ensures efficiency and accuracy when processing complex and long documents. Using preset text selection rules helps improve the response speed of question answering, reduce the consumption of computing resources, and enhance the user experience of obtaining accurate answers.

[0089] S15. Based on all the text information in the candidate text set, obtain the first text information that meets the text summary extraction rules, and based on the first text information and the question statement to be answered, use a large language model to generate and output the answer statement that matches the question statement to be answered, thus completing the question answering process.

[0090] In this embodiment, the selected text set is analyzed, and first text information is selected or generated based on preset text summarization extraction rules. The text summarization extraction rules are pre-set according to requirements and the processing capabilities of the large language model, and are used to extract text information that best represents the core content of the selected text set. These rules may involve factors such as text length limits, information density, and content relevance judgment. It should be noted that the specific text summarization extraction rules can be flexibly formulated according to actual conditions and requirements; their main purpose is to ensure that the first text information contains key content and meets the input requirements of the large language model. After obtaining the first text information, it is input into the large language model along with the question statement to be answered. The large language model utilizes its powerful natural language understanding and generation capabilities to generate an answer statement that matches the question statement based on the first text information and the question statement to be answered. During this process, the large language model performs semantic parsing on the question statement to understand its core requirements, while simultaneously extracting relevant information from the first text information, combining both to generate an accurate and fluent answer. Through this step, key information is extracted from a large amount of relevant text, and an answer highly matched to the user's question is generated. This step fully leverages the advantages of large language models, improving answer accuracy and user experience, and achieving efficient processing and accurate question answering for complex and long documents.

[0091] This application, based on a multi-scale information database, retrieves the first information entity matching the question entity, further enhancing its positioning capabilities within complex knowledge graphs. The construction of the multi-scale information database enables efficient association of information entities, profile summaries, and entity profiles through hierarchical relationships, ensuring the accuracy and comprehensiveness of the retrieval results. By reading the relationship tags between the first information entity and the first profile summary, detailed information related to the user's question can be obtained layer by layer, including the profile summary and entity profile. This multi-layered information association mechanism allows for comprehensive consideration of information at different granularities when answering questions, improving the depth and information coverage of question answering.

[0092] When applying text selection rules based on the first profile summary and the first entity profile, text information highly relevant to the user's question can be dynamically filtered. Through flexible text selection rules, the core content of the candidate text set can be efficiently extracted, ensuring information quality during the question-and-answer process. This process improves the efficiency of question-and-answer when processing large-scale text, while ensuring the accuracy and relevance of the answers. Finally, by inputting the text information along with the user's question into a large language model, answer statements matching the question are generated. This ensures that the output answer statements are both accurate and contextually coherent. In summary, this application improves the accuracy, relevance, and response efficiency of question-and-answer through multi-level semantic parsing, information association, and text optimization.

[0093] Figure 2This is a schematic flowchart illustrating a method for constructing a multi-scale information database according to one embodiment of this application. Based on the above embodiment, as... Figure 2 As shown, constructing a multi-scale information database includes:

[0094] S21. Based on the knowledge document, obtain multiple text blocks and use a large language model to perform semantic parsing on each text block to extract one or more information entities; wherein each information entity is a proper noun or object in the text block.

[0095] In this embodiment, a knowledge file refers to a long document containing a large amount of information, such as a research report, technical paper, or industry analysis report. Since these documents are typically lengthy and complex, direct processing may result in high computational resource consumption and low efficiency. Therefore, the knowledge file is divided into multiple text blocks. The division of text blocks can be based on natural language structures such as paragraphs, chapters, and topics, ensuring that each text block contains relatively complete information. Next, a large language model is used to perform semantic parsing on each text block. During semantic parsing, the semantic structure, contextual relationships, and implicit meanings of the text block can be determined. Through lexical, syntactic, and semantic analysis of the text blocks, the large language model can accurately capture important information and key concepts in the text. Based on semantic parsing, one or more information entities are extracted from each text block. Information entities refer to proper nouns or objects appearing in the text block, representing the core content and important concepts of the text. For example, in technical documents, information entities may be specific technical terms, algorithm names, equipment models, etc.; in industry reports, they may be company names, market indicators, product categories, etc. By extracting information entities, unstructured text information can be transformed into structured data, facilitating subsequent processing and analysis.

[0096] S22, based on each information entity, according to the text block corresponding to the information entity, a large language model is used to generate one or more entity profiles of the information entity; wherein, the entity profile includes at least one of the following: feature information, background information and conclusion information related to the information entity.

[0097] In this embodiment, for each information entity, one or more entity profiles are generated or constructed using a large language model. An entity profile is a deeper semantic and content extension of the information entity, aiming to describe the core information related to the entity to support subsequent question-answering tasks and information retrieval. Specifically, text blocks corresponding to the information entity are used as input data. These text blocks contain the entity's background, attributes, or contextual information. By analyzing these text blocks, the large language model can understand the semantic features of the information entity and its manifestations in different contexts. Then, the large language model generates entity profiles based on the text blocks. Each entity profile is a multi-scale description of the information entity, specifically including but not limited to feature information, background information, and conclusion information. Feature information describes the attributes, functions, or manifestations of the information entity. For example, for a technical term, its feature information may include the core components, working principles, or performance indicators of the technology. Background information provides the information entity's historical development, application scenarios, or related background. For example, for an industry standard, background information may involve its formulation process, scope of application, and development trends. Conclusion information summarizes the research conclusions, experimental results, or application effectiveness of the information entity in a specific field. For example, for certain experimental data, the conclusions might encompass key findings or ultimately validated theories. Each entity can correspond to multiple entity profiles. Different entity profiles may originate from different text blocks, representing different perspectives and semantic levels of the information entity in various scenarios. This multi-perspective description helps to comprehensively showcase all aspects of the information entity, enhancing the adaptability of question-answering methods in complex problem scenarios. By generating entity profiles, a deep semantic construction of information entities is achieved. These entity profiles not only endow information entities with richer semantic information but also provide crucial support for subsequent information retrieval and question-answering generation through the characterization of different dimensions such as features, background, and conclusions.

[0098] S23. Based on the entity profile, a large language model is used to extract the summary of the entity profile to obtain the profile summary corresponding to each entity profile.

[0099] In this embodiment, due to the rich content and high information density of entity profiles, direct use may lead to information redundancy, affecting retrieval efficiency and system response speed. Therefore, based on the generated entity profiles, a large language model is used for summary extraction to generate a profile summary corresponding to each entity profile. The profile summary is a highly condensed and summarized version of the entity profile content, aiming to retain the core information most relevant to the information entity, providing more refined data support for subsequent information retrieval and answer generation. The large language model extracts summaries from entity profiles through semantic analysis and information filtering. The large language model automatically identifies key sentences and important semantic segments in the entity profile, removing redundant information and retaining only the most representative and valuable content. For example, for an entity profile describing a technological breakthrough, the large language model may extract core technical indicators, key experimental results, and their impact on industry development, generating a concise summary text. That is, the profile summary not only retains the main characteristics of the information entity but also reduces the amount of data that users need to process during information retrieval and question-and-answer processes. Each profile summary is a brief summary of the entity profile, capable of quickly conveying the key information of the information entity. The generation of profile summaries improves the efficiency and response quality of question-and-answer processes. When faced with long documents or complex knowledge graphs, profile summarization can quickly focus on high-value information, avoiding the consumption of large amounts of computing resources when processing lengthy texts. At the same time, the conciseness and accuracy of profile summarization also lay the foundation for subsequent answer generation, ensuring that user questions can obtain the most relevant answers as quickly as possible.

[0100] In summary, the above steps enabled multi-level information extraction and representation of knowledge documents. From initial text block acquisition to information entity extraction, and then to entity profiling and profiling summary extraction, a deeper understanding of the knowledge document's content was achieved. This multi-level information structure not only improves the efficiency of information retrieval and matching but also provides rich and accurate data support for subsequent question-and-answer generation.

[0101] S24. Based on the correspondence between information entities, profile summaries, and entity profiles, construct intra-layer node connection edges for the information entity layer, profile summary layer, and entity profile layer, respectively, using information entities, profile summaries, and entity profiles as nodes; construct inter-layer node connection edges between the information entity layer and profile summary layer; and construct inter-layer node connection edges between the profile summary layer and entity profile layer.

[0102] In this embodiment, information entities, image summaries, and entity images are used as nodes to construct an information entity layer, an image summary layer, and an entity image layer, respectively. Specifically, the information entity layer contains core information entity nodes extracted from the knowledge document; the image summary layer consists of image summary nodes extracted from each entity image; and the entity image layer contains more detailed entity image nodes. Next, intra-layer node connection edges are constructed within each layer. For example, intra-layer edges in the information entity layer are used to connect semantically related information entity nodes. For instance, when two information entities appear multiple times in the knowledge base or are related, a connection is established between them, indicating their close semantic and content association. It should be noted that for information entity nodes within the information entity layer, the node connections can be as described in the example above; alternatively, they can exist as independent information entity nodes, i.e., without intra-layer connection edges. Whether or not connections are needed can be flexibly configured according to the specific knowledge document type and requirements.

[0103] The in-layer edges of the image summary layer are used to connect related image summary nodes, which may describe different characteristics of the same information entity in different summaries. The in-layer edges of the entity image layer are used to connect entity image nodes that are semantically or content-wise related. For example, two entity images describing different application scenarios of the same technology may be considered intrinsically related, thus establishing a connection. Subsequently, inter-layer node connection edges are established between different layers. The inter-layer connection edges between the information entity layer and the image summary layer connect each information entity node to its corresponding image summary node. These connections represent the relationship between the information entity and its abstract description. For example, an information entity may correspond to multiple image summaries, each outlining its different semantic features or application scenarios. The inter-layer edges between the image summary layer and the entity image layer connect each image summary node to its detailed entity image node. These connections provide a guiding path from summary to detailed content, enabling flexible switching between information at different granularities. Through the above steps, the three-layer graph structure is constructed. This graph structure not only stores the detailed content of information entities, image summaries, and entity images, but also reveals the semantic connections of this information at different levels through the associations of nodes and edges. By constructing a multi-layered graph structure, efficient association and organization of information are achieved. This enables multi-scale information retrieval at different levels, and allows for faster and more accurate location of relevant content when handling complex semantic queries, thereby improving question-answering performance and user experience.

[0104] Figure 3This is a schematic diagram of the retrieval graph provided in the embodiments of this application. For ease of explanation, taking a portrait summary as an example, the construction of connection edges related to the portrait summary is further explained here. The diagram uses numerical representation, while the current text uses letter representation, but they essentially express the same semantics. All portrait summary nodes are traversed. For each entity portrait summary node, the information entity contained therein is read and confirmed. When information entity A and information entity B both appear in portrait summary C, inter-layer connection edges are established between information entity A and portrait summary C, and between information entity B and portrait summary C, respectively. The information of these inter-layer connection edges is stored in the corresponding information entity node and portrait summary node to facilitate quick association of entities and summary content in subsequent searches. In addition, when both portrait summary node C and portrait summary node D have inter-layer connection edges with information entity A, intra-layer connection edges are established between portrait summary C and portrait summary D to indicate their close semantic association. At the same time, intra-layer edges are established between entity portrait C' and entity portrait D' corresponding to portrait summary C and portrait summary node D, further improving the connection relationship within the entity portrait layer. Repeat the above process to process all entity profile summaries and entity profiles one by one, and finally complete the creation of entity profile summary diagrams and entity profile text diagrams.

[0105] S25 uses a preset data format to encode each node, each intra-layer node connection edge, and each inter-layer node connection edge to serialize all nodes, all intra-layer node connection edges, and all inter-layer node connection edges; wherein, the data format includes at least one of protocol buffer and JSON-based link data.

[0106] In this embodiment, each node is encoded. During the encoding process, a unique identifier is assigned to each node and connection edge, and its attributes and association information are recorded. Node attributes may include node type (e.g., information entity, profile summary, entity profile), content summary, association tags, etc. Connection edge attributes may include connection type (intra-layer or inter-layer), association strength, etc. For example, an information entity node may include fields such as node ID, entity name, entity semantic vector, etc., while an entity profile node may include information such as node ID, associated entity, and detailed description text. This structured data facilitates subsequent storage and retrieval. The encoded information of each connection edge typically includes the starting node ID, the target node ID, and the connection edge attributes (e.g., association type, etc.). This encoded data can accurately record the association information between nodes, supporting efficient semantic retrieval and path analysis. Then, inter-layer node connection edges are encoded. For example, the connection between an information entity and its corresponding profile summary, or the connection between a profile summary and a specific entity profile. The encoded information of these connection edges not only includes the starting node ID and the target node ID, but also records the specific relationship type between the nodes, such as "abstracted to" or "detailed description". By encoding the connection edges between nodes in different layers, cross-layer information associations can be effectively managed, supporting flexible switching and querying of information at multiple granularities.

[0107] After encoding, all nodes and connecting edges are serialized. Serialization refers to converting structured data into a linear byte stream for efficient storage and transmission over a network. Preset data formats include at least one of Protocol Buffers and JSON-LD (JSON-based Linked Data). Protocol Buffers is a binary serialization format that reduces storage space and transmission latency while ensuring data integrity, making it suitable for high-performance applications. JSON-LD is a JSON-based data format that embeds data into JSON objects and makes this data meaningful and linkable through a set of predefined rules and vocabulary (such as Schema.org). Through these encoding and serialization steps, efficient management and optimized storage of multi-level graph structures are achieved. This improves the storage efficiency of multi-scale information databases.

[0108] S26, record the intra-layer node connection edges and inter-layer node connection edges of the node as the node's data relationship label, traverse all nodes, obtain the data relationship label of each node, and store all nodes and each node's data relationship label in the multi-scale information database.

[0109] In this embodiment, the intra-layer node connection edges and inter-layer node connection edges of each node are recorded. Next, a data relationship label is generated for each node. The data relationship label not only records the connection information between the node and other nodes, but also includes the attributes of each connection edge, such as connection type and semantic correlation degree. Through the data relationship label, the relationship network of each node in the graph structure can be quickly described, thereby supporting efficient path analysis and relevance calculation. Then, all nodes in the multi-scale information database are traversed, and their data relationship labels are extracted and integrated. This traversal process ensures that the relationship information of each node is completely recorded, including nodes directly related to it and nodes connected through inter-layer connections. This comprehensive extraction of relationship information provides a solid data foundation for complex semantic queries. Finally, all nodes and their corresponding data relationship labels are stored in the multi-scale information database. This storage method not only ensures the integrity and consistency of the data, but also improves retrieval efficiency and storage management flexibility. Through this step, efficient management of complex relationships between nodes is achieved. The introduction of data relationship labels provides rich semantic relationship information, enabling the rapid location of relevant nodes in complex problem scenarios, improving the accuracy and efficiency of information retrieval.

[0110] In one embodiment, Figure 4 This is a flowchart illustrating a method for obtaining multiple text blocks of a knowledge document according to an embodiment of this application. It provides a detailed explanation of the implementation of step S21 described above. Based on the above embodiment, as... Figure 4 As shown, it includes:

[0111] S41, parse the document structure of the knowledge document, identify and divide the structural information of the knowledge document; the structural information includes title items, abstract items, table of contents items, main text items and appendix items;

[0112] S42, retrieve the main text items, and extract the nested chapters and subsections according to the table of contents items to generate the initial text;

[0113] S43, preprocess each initial text, and based on the preprocessed initial text, segment the initial text using a preset character length threshold to obtain multiple text blocks.

[0114] In this embodiment, the knowledge document is first analyzed to parse its document structure, identifying and dividing its structural information. Knowledge documents typically refer to long documents containing a large amount of information, such as technical reports, research papers, and industry white papers. These documents usually have a fixed structure, including but not limited to titles, abstracts, tables of contents, main text, and appendices. By parsing the document structure, information within the document can be efficiently organized and extracted. Document structure parsing is achieved by analyzing the document's layout, tags, or specific separators. For example, titles are usually located at the beginning of the document, using a specific font and size; abstracts usually follow the titles, concisely summarizing the document's content; tables of contents list the document's chapter and section titles, providing hierarchical information; the main text is the core of the document, containing detailed research content, data analysis, etc.; and appendices are usually located at the end of the document, providing supplementary data or supporting information.

[0115] After parsing, the main text items are extracted. To facilitate subsequent processing, the main text items are further divided according to the chapter and section hierarchy provided by the table of contents. Each entry in the table of contents typically corresponds to a chapter or section in the main text. Following the hierarchical structure of the table of contents, the main text items are recursively parsed to extract nested chapters and sections. This hierarchical extraction method effectively preserves the hierarchy and logic of information in the main text, ensuring that each extracted content fragment has independent semantic integrity. After extraction, each chapter or section content is used as the initial text. The content of the initial text may vary in length; some chapters may contain thousands of words of detailed description, while some sections may contain only a few short paragraphs.

[0116] To further optimize information processing efficiency, each initial text undergoes preprocessing. The preprocessing process includes multiple steps aimed at cleaning and standardizing the initial text content. First, invalid information is removed from the initial text, such as redundant spaces, line breaks, headers and footers, page numbers, and special symbols. Furthermore, the formatting is standardized, for example, all paragraph first-line indents are standardized to a fixed format to ensure a clear and neat text structure. After preprocessing, to accommodate subsequent semantic parsing and information extraction, the initial text needs to be segmented. Since the length of the initial text may far exceed the processing capacity of the system or large language model, the segmentation step divides the initial text into multiple smaller text blocks using a preset character length threshold. It should be noted that the size of the text blocks needs to balance information integrity and processing efficiency, ensuring that each text block contains a complete semantic unit (e.g., a paragraph or a group of related paragraphs) while avoiding excessively long text blocks that could negatively impact subsequent processing performance. The specific character length threshold can be flexibly set according to the specific knowledge document type and the processing capacity of the large language model; this is not a limitation of the application.

[0117] After segmentation, multiple text blocks are obtained, each a portion of the initial text. These text blocks will serve as the foundational data for subsequent information extraction, semantic parsing, and question-answering generation. Through structured parsing, content partitioning, and text block generation of knowledge documents, the massive amounts of information in long documents can be efficiently managed and processed, providing high-quality data support for subsequent information extraction and question-answering tasks. This embodiment transforms complex knowledge documents into structured and standardized text blocks through hierarchical parsing and standardization, improving information accessibility and processing efficiency. This approach not only enhances the depth of document understanding but also lays the foundation for the efficient application of large language models, thereby further improving the performance and accuracy of question answering.

[0118] In one specific embodiment, the multi-scale information database includes a graph database and a vector database.

[0119] In this embodiment, the graph database stores complex node relationships, supporting multi-hop queries based on structure and hierarchy; the vector database stores high-dimensional vectors, supporting efficient retrieval based on semantic similarity. The combination of these two databases endows the multi-scale information database with powerful data management and query capabilities. Through this dual-database architecture, the multi-scale information database can efficiently store and manage large amounts of multi-scale data and support flexible and diverse retrieval methods. This architecture not only improves the ability to process complex data but also provides rich data support for question answering.

[0120] Furthermore, Figure 5 This is a flowchart illustrating the method for storing all nodes and data relationship labels to a multi-scale information database, as provided in this embodiment. It specifically explains the implementation of step S26. Based on the above embodiment, as... Figure 5 As shown, it includes:

[0121] S51. Based on all portrait summaries and their data relationship tags, as well as all entity portraits and their data relationship tags, obtain the topology of the portrait summary layer and the entity portrait layer, and store the topology in the graph database.

[0122] S52, based on the information entities in the information entity layer, using a preset embedding model, obtain the information entity vector corresponding to each information entity, and store all information entities, information entity vectors, and data relationship labels carried by each information entity into the vector database.

[0123] In this embodiment, the topology of a multi-scale information database is constructed and stored by analyzing the relationship between image summaries and entity images. This topology includes nodes and their connections in the image summary layer and entity image layer, supporting complex relationship queries in the graph database. First, the topology of the image summary layer and entity image layer is obtained. Each node in the image summary layer represents an image summary, and each node in the entity image layer represents an entity image. Each image summary node and entity image node carries a data relationship label. By parsing these data relationship labels, the inter-layer connections between image summaries and entity images, as well as the node connections within each layer, can be constructed. For example, an image summary may be associated with one or more entity images; this one-to-one or many-to-many relationship is represented by edges in the topology. It should be noted that the image summary is obtained by extracting the entity image from its summary, meaning one image summary corresponds to one entity image. Furthermore, if semantic analysis is performed on the image summaries and entity images, it is possible that one image summary has a strong semantic association with multiple entity images, meaning one image summary may correspond to multiple entity images. During the construction of the topology, all profile summary nodes and entity profile nodes are traversed, and their data relationship tags are parsed one by one. For each pair of related nodes, corresponding edges are added to the graph database to record their connection relationship. The edge attributes may include information such as association type and association strength, used to describe the nature and closeness of the relationship between nodes. Next, the topology is stored in the graph database. The graph database can intuitively store and manage this complex relationship data in the form of nodes and edges.

[0124] Subsequently, based on the information entities in the information entity layer, a high-dimensional vector representation of each information entity is generated. Vectorization is then performed using an embedding model. An embedding model is a deep learning-based model, such as BERT, Word2Vec, or other semantic embedding models, capable of converting text data into high-dimensional vectors. All information entities, information entity vectors, and the data relationship labels carried by each information entity are stored in a vector database (such as Faiss or Milvus).

[0125] Based on the above embodiments, Figure 6 This is a schematic flowchart illustrating the method for obtaining a first information entity according to an embodiment of this application. It provides a detailed explanation of the implementation of step S12 described above. Figure 6 As shown, it includes:

[0126] S61, Based on the problem entity, use a preset embedding model to obtain the problem entity vector corresponding to the problem entity;

[0127] S62, In the vector database, based on vector similarity, retrieve and obtain the first information entity vector that matches the question entity vector, so as to obtain the first information entity that matches the question entity.

[0128] In this embodiment, the question entity is vectorized based on a preset embedding model. For example, during vectorization, the embedding model maps the question entity to a high-dimensional vector space according to its contextual information. The generated question entity vector not only captures the semantic meaning of the entity but also preserves its semantic differences in different contexts. The advantage of this vector representation is that it can quantify the semantic similarity between different entities numerically. Next, the question entity vector is matched with information entity vectors in the vector database. During the matching process, vector similarity calculation methods, such as cosine similarity, Euclidean distance, or dot product, are used to determine the similarity between the question entity vector and each information entity vector in the database. Higher similarity indicates greater semantic closeness.

[0129] By calculating similarity, the information entity most semantically relevant to the question entity can be quickly located. In the vector database, the vector with the highest similarity to the question entity vector is selected based on similarity ranking; this is called the first information entity vector. This vector corresponds to the first information entity that best matches the question entity. The first information entity represents the knowledge unit most relevant to the user's question, containing the key information needed to solve the problem. The efficiency of this matching process benefits from the efficient retrieval capabilities of the vector database. Compared to traditional text matching methods, vector similarity-based retrieval can quickly process large-scale data and return highly relevant results in a short time. This retrieval method ensures that the most relevant information entity is quickly found in the vast information database, improving the response speed and answer quality of question answering. Therefore, the completion of this embodiment marks the accurate location of the user's question from the relevant knowledge. Through the vectorized representation of the question entity and semantic similarity-based retrieval, not only can the core semantics of the user's question be understood, but the most relevant information can also be quickly located in the knowledge base, providing basic data support for subsequent answer generation.

[0130] In one embodiment, Figure 7 This is a flowchart illustrating a method for obtaining a candidate text set according to an embodiment of this application. It is an explanation of the implementation of step S14 described above. Based on the above embodiment, as... Figure 7 As shown, it includes:

[0131] S71 uses a large language model to obtain the semantic similarity between each first profile summary and the problem entity profile;

[0132] S72, after determining that the semantic similarity between the first profile summary and the problem entity profile is greater than or equal to the second similarity threshold and less than the first similarity threshold, according to the data relationship tags carried by the first profile summary, the second entity profile corresponding to the first profile summary is selected from the first entity profile.

[0133] S73, using a large language model, obtain the semantic similarity between each second entity profile and the question entity profile, and after determining that the semantic similarity between the second entity profile and the question entity profile is greater than or equal to the third similarity threshold, mark the second entity profile as candidate text;

[0134] S74, after determining that the semantic similarity between the first profile summary and the problem entity profile is greater than or equal to the first similarity threshold, the first profile summary is marked as candidate text;

[0135] S75, traverse the first image summary, and obtain the candidate text set based on the second entity image and / or the first image summary.

[0136] In this embodiment, a large language model is used to obtain the semantic similarity between each first profile summary and the question entity profile. For example, the text content of each first profile summary and the question entity profile is input into the large language model to obtain their corresponding semantic vectors. Then, similarity calculation methods such as cosine similarity are used to calculate the semantic similarity between each first profile summary and the question entity profile. Through this process, the semantic relevance between each first profile summary and the question entity profile can be quantitatively analyzed, providing a basis for subsequent text filtering.

[0137] When the semantic similarity between a first entity profile summary and the question entity profile is greater than or equal to a second similarity threshold but less than a first similarity threshold, a certain correlation is considered to exist between the two, but it is not sufficient to directly serve as the basis for answering the question. At this point, based on the data relationship tags carried by the first entity profile summary, further filtering is performed on the associated first entity profiles to obtain the second entity profile corresponding to that first entity profile summary. The filtered first entity profile is then recorded as the second entity profile. This process utilizes the association relationships between multi-level data, linking the profile summary with more detailed entity profiles, thereby further refining more valuable textual information. The large language model is used again to obtain the semantic similarity between each second entity profile and the question entity profile. If the similarity between the second entity profile and the question entity profile is greater than or equal to a third similarity threshold, it indicates that the second entity profile contains key information for answering the user's question. At this point, the second entity profile is marked as candidate text and included in the candidate text set.

[0138] When the semantic similarity between a first profile summary and the question entity profile is greater than or equal to the first similarity threshold, it indicates that the first profile summary and the question entity profile have a high semantic relevance, meaning that the profile summary can directly provide an answer to the user's question. At this point, the first profile summary is directly marked as candidate text and included in the candidate text set, providing the most direct and relevant text support for answer generation. This direct selection method ensures that highly relevant text can quickly enter the subsequent processing flow, improving the efficiency of question answering. Finally, all first profile summaries are traversed, and based on the above logic, each profile summary and possible second entity profiles are filtered for similarity. By including first profile summaries and / or second entity profiles with high semantic relevance in the candidate text set, a high-quality text set is obtained, containing the text content most likely to answer the user's question. In summary, through multi-level similarity calculation and dynamic filtering mechanisms, the relevance and accuracy of the candidate text set are ensured. The multi-level analysis of semantic similarity not only fully explores the useful information in the first profile summary and the first entity profile, but also utilizes the potential connections between the first entity profile and the second entity profile, further enhancing the depth and breadth of information filtering. Ultimately, the generated set of candidate texts provides high-quality input for subsequent answer generation, which helps improve the accuracy of question answering and the user experience.

[0139] In one embodiment, Figure 8 This is a schematic flowchart illustrating a method for obtaining first text information provided in an embodiment of this application. It specifically explains the implementation of obtaining the first text information in step S15 above. Based on the above embodiment, as... Figure 8 As shown, it includes:

[0140] S81, based on the similarity between the text information and the problem entity profile, sort the text information in descending order of similarity to obtain a text information sequence; wherein, the text information is either a first profile summary or a second entity profile;

[0141] S82, according to the text summary extraction rules, in the text information sequence, multiple text information to be processed are sequentially obtained from high to low, corresponding to the quantity threshold;

[0142] S83, if the total length of multiple text messages to be processed is greater than a preset length threshold, then a summary extraction is performed on the multiple text messages to be processed to obtain the first text message that meets the preset length threshold.

[0143] S84, if the total length of multiple text messages to be processed is less than or equal to a preset length threshold, then the multiple text messages to be processed are concatenated and recorded as the first text message.

[0144] In this embodiment, the text information is sorted according to the similarity between the text information and the question entity profile, from highest to lowest similarity value, generating an ordered text information sequence. The first few items in the sequence represent the text information with the strongest semantic relevance to the question entity profile, and these texts have high reference value. The sorting process ensures that highly relevant text information is given priority in subsequent processing, improving the response efficiency and accuracy of question answering. Next, according to the text summarization extraction rules, several text information to be processed are selected sequentially from the sorted text information sequence. Specifically, based on a preset quantity threshold, the text information with the highest similarity is selected one by one until the quantity threshold is met. This selection method ensures that the number of text information to be processed matches the processing capacity, while retaining the most relevant text content. If the total length of multiple text information to be processed exceeds a preset length threshold, a summary extraction is required for these text information. The summary extraction process uses a large language model to analyze the content of each text information, extracting the core and key information, thereby generating a more concise summary text. This summarization method retains the essence of the original text while effectively compressing its length, ensuring that the generated first text meets the length requirements. If the total length of multiple text messages to be processed is less than or equal to a preset length threshold, these text messages are directly concatenated to form a complete text sequence. During the concatenation process, the original semantics and logical structure of each text message are preserved to ensure that the concatenated first text fully reflects the information content of the text to be processed. By dynamically adjusting the text length and content density, high-quality answers are generated while efficiently processing large amounts of text information. Whether extracting and compressing redundant information through summarization or preserving complete semantics through concatenation, it can flexibly adapt to the processing needs of different scenarios, providing users with accurate and reliable question-and-answer services.

[0145] Furthermore, based on the first text information and the question statement to be answered, a large language model is used to generate and output the answer statement that matches the question statement to be answered, including:

[0146] S151, input the first text information and the question statement to be answered into the large language model, perform semantic processing and information supplementation on the first text information according to the core question of the question statement to be answered, generate an answer statement that is consistent with the semantic information of the question statement to be answered, and output the answer statement.

[0147] In this embodiment, first text information and the question statement to be answered are input into the large language model. The first text information contains core content related to the user's question, selected or generated from knowledge files, while the question statement to be answered is the question posed by the user in natural language. These two parts together constitute the input data of the large language model, providing semantic and contextual support for generating the answer. Upon receiving the input, the large language model performs semantic parsing on the question statement to be answered. This parsing process ensures that the large language model can clearly understand the user's true needs and the key points of the question.

[0148] Subsequently, the large language model performs semantic processing and information supplementation on the initial text information. Semantic processing refers to the model extracting relevant content from the initial text information based on the core question of the question to be answered, filtering out irrelevant information, and reorganizing the answer according to the semantics of the question. This process may involve content reconstruction, logical reasoning, or supplementing background information. Information supplementation refers to the large language model enhancing the completeness and accuracy of the answer by inferring or introducing additional information from its pre-trained knowledge when the initial text information is insufficient to fully answer the question. Through the above processing, the large language model generates an answer statement that is semantically consistent with the question to be answered and outputs the generated answer statement to the user. In summary, by fully leveraging the semantic understanding and generation capabilities of the large language model, accurate conversion from question to answer is achieved. The large language model ensures that it can generate accurate and practically valuable answers when dealing with complex questions. This efficient question-and-answer process improves the user experience.

[0149] In one embodiment, Figure 9 This is a schematic flowchart illustrating a method for obtaining a problem entity and a problem entity profile according to an embodiment of this application. It describes one implementation of obtaining the problem entity and problem entity profile in step S11 above. Based on the above embodiment, as... Figure 9 As shown, it includes:

[0150] S91 receives the question statement to be answered from the user, performs semantic parsing on the question statement using a large language model, and extracts one or more question entities;

[0151] S92, based on the context information of each question entity and the question statement to be answered, a large language model is used to obtain the question entity profile corresponding to the question entity.

[0152] In this embodiment, the system receives user-inputted questions. These questions, expressed in natural language, represent the user's information needs and may cover various fields and topics. For example, a question like, "What are the specific applications of data mining techniques in this research report?" typically contains specific objects, goals, or conditions, requiring an accurate response to the user's needs based on its semantic structure. Next, a large language model is used to semantically parse the questions. Through a comprehensive analysis of the sentence's grammatical structure, semantic hierarchy, and contextual relationships, the large language model accurately captures the key elements of the question. During parsing, the large language model identifies core objects and descriptive phrases in the sentence, such as proper nouns, technical terms, and entity names. These key elements are extracted as question entities. For example, in the question, "What are the most commonly used renewable energy sources globally?", "renewable energy" is a question entity. By extracting question entities, the user's focus can be clarified, providing a basis for subsequent information retrieval and processing. After extracting one or more question entities, information related to each entity (i.e., a question entity profile) is extracted based on each entity and its context. Semantic parsing and profile generation enhance the understanding of the user's question. This not only optimizes the response speed and accuracy of Q&A, but also enhances the user experience, enabling the provision of high-quality Q&A services in a wide range of application scenarios.

[0153] Figure 10 This is a schematic diagram of the question-and-answer device provided in an embodiment of this application. Figure 10 As shown, the question-and-answer device 10 includes:

[0154] The problem parsing module 101 is used to receive the question statement to be answered and use a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity.

[0155] The information retrieval module 102 is used to retrieve and obtain the first information entity that matches the question entity based on a multi-scale information database; wherein, the multi-scale information database is constructed from information entities, profile summaries and entity profiles;

[0156] The information acquisition module 103 is used to acquire the first profile summary corresponding to the first information entity based on the data relationship tags carried by the first information entity, and read the data relationship tags carried by the first profile summary to acquire the first entity profile corresponding to the first profile summary.

[0157] The information filtering module 104 is used to obtain a set of candidate texts based on the first portrait summary and the first entity portrait, using preset text selection rules; wherein the set of candidate texts includes the first portrait summary and / or the first entity portrait.

[0158] The question-and-answer generation module 105 obtains the first text information that meets the text summary extraction rules based on all text information in the candidate text set, and generates and outputs the answer statement that matches the question statement based on the first text information and the question statement to be answered using a large language model, thus completing the question-and-answer process.

[0159] The question-and-answer device provided in this embodiment can execute the question-and-answer method of the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0160] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 11 As shown, the electronic device 11 includes at least one processor 111 and a memory 112. The electronic device 11 also includes a communication component 113. The processor 111, the memory 112, and the communication component 113 are connected via a bus 114.

[0161] In the specific implementation process, at least one processor 111 executes computer execution instructions stored in memory 112, causing at least one processor 111 to execute the entity question-and-answer generation method executed on the electronic device side as described above.

[0162] The specific implementation process of processor 111 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0163] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0164] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0165] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0166] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0167] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above method.

[0168] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0169] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0170] This application also provides a computer program product, comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the scheme provided in any of the above embodiments.

[0171] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0172] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0173] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0174] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0175] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0177] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0178] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0179] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0180] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A question-answering method based on a large language model, characterized in that, include: Receive the question statement to be answered, and use a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity; Based on a multi-scale information database, a first information entity matching the problem entity is retrieved and obtained; wherein, the multi-scale information database is constructed from information entities, profile summaries, and entity profiles; Based on the data relationship tags carried by the first information entity, obtain the first profile summary corresponding to the first information entity, and read the data relationship tags carried by the first profile summary to obtain the first entity profile corresponding to the first profile summary; Based on the problem entity profile, the first profile summary, and the first entity profile, a candidate text set is obtained using preset text selection rules; wherein, the candidate text set includes the first profile summary and / or the first entity profile. Based on all text information in the candidate text set, first text information that meets the text summary extraction rules is obtained. Based on the first text information and the question statement to be answered, a large language model is used to generate and output the answer statement that matches the question statement to be answered, thus completing the question-and-answer process.

2. The method according to claim 1, characterized in that, Constructing the multi-scale information database includes: Based on the knowledge document, multiple text blocks are obtained, and a large language model is used to perform semantic parsing on each text block to extract one or more information entities; wherein each information entity is a proper noun or object in the text block; Based on each information entity, and according to the text block corresponding to the information entity, a large language model is used to generate one or more entity profiles of the information entity; wherein, the entity profile includes at least one of the following: feature information, background information, and conclusion information related to the information entity; Based on the entity profile, a large language model is used to extract a summary from the entity profile to obtain a profile summary corresponding to each entity profile. Based on the correspondence between the information entity, the portrait summary, and the entity portrait, using the information entity, the portrait summary, and the entity portrait as nodes, intra-layer node connection edges are constructed for the information entity layer, the portrait summary layer, and the entity portrait layer, respectively; inter-layer node connection edges are constructed for the information entity layer and the portrait summary layer; and inter-layer node connection edges are constructed for the portrait summary layer and the entity portrait layer. Each node, each intra-layer node connection edge, and each inter-layer node connection edge are encoded using a preset data format to serialize all nodes, all intra-layer node connection edges, and all inter-layer node connection edges. Record the intra-layer node connection edges and inter-layer node connection edges of the node as the data relationship labels of the node, traverse all the nodes, obtain the data relationship label of each node, and store all the nodes and the data relationship labels of each node in a multi-scale information database.

3. The method according to claim 2, characterized in that, The process of obtaining multiple text blocks based on the knowledge file includes: The document structure of the knowledge file is parsed to identify and divide the structural information of the knowledge file; wherein, the structural information includes a title item, a summary item, a table of contents item, a main text item, and an appendix item; Obtain the main text item, and extract the nested chapters and subsections according to the table of contents item to generate the initial text; Each initial text is preprocessed, and based on the preprocessed initial text, a preset character length threshold is used to segment the initial text to obtain multiple text blocks.

4. The method according to claim 2, characterized in that, The multi-scale information database includes a graph database and a vector database; The data relationship labels for the nodes are recorded as the intra-layer node connection edges and inter-layer node connection edges. All nodes are traversed to obtain the data relationship label for each node, and all nodes and their data relationship labels are stored in a multi-scale information database, including: Based on all the image summaries and their data relationship tags, as well as all the entity images and their data relationship tags, the topology of the image summary layer and the entity image layer is obtained, and the topology is stored in the graph database. Based on the information entities in the information entity layer, a preset embedding model is used to obtain the information entity vector corresponding to each information entity, and all information entities, the information entity vectors, and the data relationship tags carried by each information entity are stored in the vector database. Based on a multi-scale information database, a first information entity matching the problem entity is retrieved and obtained, including: Based on the problem entity, a preset embedding model is used to obtain the problem entity vector corresponding to the problem entity; In the vector database, based on vector similarity, a first information entity vector matching the question entity vector is retrieved and obtained, so as to obtain the first information entity matching the question entity.

5. The method according to claim 1, characterized in that, The step of obtaining a candidate text set based on the problem entity profile, the first profile summary, and the first entity profile, using preset text selection rules, includes: A large language model is used to obtain the semantic similarity between each first profile summary and the problem entity profile; After determining that the semantic similarity between the first profile summary and the problem entity profile is greater than or equal to the second similarity threshold and less than the first similarity threshold, based on the data relationship tags carried by the first profile summary, the second entity profile corresponding to the first profile summary is selected from the first entity profile. Using a large language model, the semantic similarity between each second entity profile and the question entity profile is obtained. After determining that the semantic similarity between the second entity profile and the question entity profile is greater than or equal to a third similarity threshold, the second entity profile is marked as candidate text; or... After determining that the semantic similarity between the first profile summary and the problem entity profile is greater than or equal to the first similarity threshold, the first profile summary is marked as candidate text; Iterate through the first image summary and obtain the candidate text set based on the second entity image and / or the first image summary.

6. The method according to claim 1, characterized in that, The step of obtaining first text information that satisfies the text summary extraction rules based on all text information in the candidate text set includes: Based on the similarity between the text information and the problem entity profile, the text information is sorted in descending order of similarity to obtain a text information sequence; wherein, the text information is a first profile summary or a second entity profile. According to the text summary extraction rules, multiple text information to be processed are sequentially obtained from the text information sequence in descending order of quantity threshold; If the total length of the multiple text messages to be processed is greater than a preset length threshold, then a summary extraction is performed on the multiple text messages to be processed to obtain a first text message that meets the preset length threshold; or... If the total length of the multiple text messages to be processed is less than or equal to the preset length threshold, then the multiple text messages to be processed are concatenated and recorded as the first text message; Based on the first text information and the question statement to be answered, a large language model is used to generate and output an answer statement that matches the question statement to be answered, including: The first text information and the question statement to be answered are input into the large language model. Based on the core question of the question statement to be answered, the first text information is semantically processed and supplemented to generate an answer statement that is consistent with the semantic information of the question statement to be answered, and the answer statement is output.

7. The method according to any one of claims 1-6, characterized in that, The step of receiving the question statement to be answered and using a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity includes: The system receives a question input from a user, performs semantic parsing on the question using a large language model, and extracts one or more question entities. Based on the context information of each question entity and the question statement to be answered, a large language model is used to obtain the question entity profile corresponding to the question entity.

8. A question-and-answer device, characterized in that, include: The question parsing module is used to receive the question statement to be answered and use a large language model to obtain the question entity of the question statement to be answered and the question entity profile corresponding to each question entity; The information retrieval module is used to retrieve and obtain a first information entity that matches the problem entity based on a multi-scale information database; wherein, the multi-scale information database is constructed from information entities, profile summaries, and entity profiles; The information acquisition module is used to acquire a first profile summary corresponding to the first information entity based on the data relationship tags carried by the first information entity, and to read the data relationship tags carried by the first profile summary to acquire a first entity profile corresponding to the first profile summary. The information filtering module is used to obtain a candidate text set based on the problem entity profile, the first profile summary, and the first entity profile, using preset text selection rules; wherein, the candidate text set includes the first profile summary and / or the first entity profile; The question-and-answer generation module obtains first text information that meets the text summary extraction rules based on all text information in the candidate text set, and generates and outputs an answer statement that matches the question statement based on the first text information and the question statement to be answered using a large language model, thus completing the question-and-answer process.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images