Large model question answering method and device based on knowledge graph, medium and electronic equipment

Abstract

The invention discloses a large model question-answering method and device based on a knowledge graph, a medium and electronic equipment, and relates to the technical field of large models, and the large model question-answering method comprises the following steps: obtaining knowledge segments corresponding to knowledge documents; identifying first metadata related to a data field in the knowledge segment, and acquiring second metadata related to the first metadata from a data warehouse according to the first metadata; according to a first target context, the relationship between entities in the knowledge segment is determined, and the first target context comprises the second metadata; and according to the entity and the relationship, generating a knowledge graph, the knowledge graph being used for supporting questions and answers of the first large model. The unified knowledge graph is constructed based on the metadata of the data warehouse and the knowledge documents, the reliability and accuracy of the knowledge graph are improved, and then the performance of large model questions and answers achieved by depending on the knowledge graph is improved.

Description

Technical Field

[0001] This disclosure relates to the field of large model technology, and more specifically, to a large model question-answering method, apparatus, medium, and electronic device based on knowledge graphs. Background Technology

[0002] Graph-based RAG (Retrieval Augmented Generation) is a paradigm that combines knowledge graphs with RAG. It upgrades the retrieval at the plain text block level in traditional RAG to the retrieval at the graph structure level, which can improve the accuracy, interpretability, and ability to handle complex questions in large models. Therefore, building a reliable and accurate knowledge graph is an important foundation for improving the performance of large models in question answering. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] Firstly, this disclosure provides a large-scale question-answering method based on knowledge graphs, including: Obtain the knowledge segments corresponding to the knowledge document; Identify the first metadata related to the data domain in the knowledge segment, and obtain the second metadata associated with the first metadata from the data warehouse based on the first metadata; Based on the first target context, the entities and relationships between entities in the knowledge segment are determined, wherein the first target context includes the second metadata; A knowledge graph is generated based on the entities and the relationships, wherein the knowledge graph is used to support question answering in the first major model.

[0005] Secondly, this disclosure provides a large-scale question-answering device based on knowledge graphs, including: The first acquisition module is used to acquire the knowledge segments corresponding to the knowledge document; The identification module is used to identify the first metadata related to the data domain in the knowledge segment, and to obtain the second metadata associated with the first metadata from the data warehouse based on the first metadata; The first determining module is configured to determine the entities and relationships between entities in the knowledge segment based on a first target context, wherein the first target context includes the second metadata; The first generation module is used to generate a knowledge graph based on the entities and the relationships, wherein the knowledge graph is used to support question answering in the first major model.

[0006] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] The above technical solution uses structured metadata stored in the data warehouse as context to determine entities and relationships between entities in knowledge segments. This solves the problem of not being able to obtain relationships between certain entities through semantic understanding of knowledge segments. Thus, a unified knowledge graph is constructed based on the metadata and knowledge documents of the data warehouse, improving the reliability and accuracy of the knowledge graph, and consequently improving the performance of large-scale model question answering that relies on the knowledge graph.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is an architecture diagram of a system for implementing a large-model question-answering method according to an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a large-scale question-answering method based on a knowledge graph, according to an embodiment of this disclosure. Figure 3 This is a schematic diagram illustrating a process for generating a knowledge graph according to an embodiment of the present disclosure; Figure 4 This is a schematic diagram illustrating a large-scale question-and-answer process according to an embodiment of the present disclosure; Figure 5This is a block diagram illustrating a knowledge graph-based large-scale question-answering device according to an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0015] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and their permission should be obtained.

[0019] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0020] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understood that the above notification and user permission acquisition process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0022] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] Figure 1 This is an architecture diagram of a system for implementing a large-model question-answering method according to an embodiment of this disclosure, with reference to... Figure 1 The architecture diagram includes the application layer, processing layer, and storage layer. Figure 1 The system shown is used to generate knowledge graphs in the data domain and to implement large-scale question answering based on knowledge graphs.

[0024] The application layer is Figure 1 The system shown interacts with external systems / users. The application layer can provide knowledge graph generation services, retrieval coordination services, entity and entity relationship extraction services, and graph retrieval services. The graph construction service is used to coordinate the entire process of knowledge graph generation; the retrieval coordination service is used to determine the retrieval path based on the user's query intent (such as the hybrid retrieval path mentioned below); the entity and entity relationship extraction service is used to extract entities and entity relationships from knowledge segments of knowledge documents.

[0025] The processing layer is Figure 1The core capability layer of the system shown includes entity type configuration, metadata mapping, graph storage service, vectorization service, and graph fusion service. Entity type configuration defines specific entity types for different business scenarios (e.g., e-commerce) in the data domain; metadata mapping determines the second metadata associated with the first metadata based on information such as the definition of metadata in the data warehouse; the graph storage and retrieval service is used to store the knowledge graph and perform knowledge graph-based retrieval; the vectorization service stores vectors (including entity vectors and entity relation vectors) and performs vector-based retrieval; the graph fusion service is used to merge incremental data extracted based on knowledge segmentation (i.e., entities and entity relations in knowledge segments) with existing data (data in the current knowledge graph) and to update the knowledge graph.

[0026] The storage layer is Figure 1 The system's persistence layer, as shown, is used to store various types of data. For example, a graph database storing knowledge graphs describing entities and entity relationships; a vector database storing entity vectors and entity relationship vectors; a data warehouse storing secondary metadata; and a relational database storing configuration information (such as the target types configured below). Figure 1 The system shown illustrates the message service for asynchronous communication between modules.

[0027] Figure 2 This is a flowchart illustrating a knowledge graph-based large-scale model question answering method according to an embodiment of this disclosure. This knowledge graph-based large-scale model question answering method can be applied to... Figure 1 The system shown is referenced. Figure 2 The knowledge graph-based large model question answering method includes steps 210, 220, 230 and 240.

[0028] In step 210, the knowledge segments corresponding to the knowledge document are obtained.

[0029] Knowledge segmentation is obtained by dividing knowledge documents. For information on knowledge document segmentation, please refer to relevant technologies. This embodiment will not elaborate on it here.

[0030] In some embodiments, each knowledge segment is associated with contextual information, which may include the source, version, title, level, and chapter path of the knowledge document corresponding to the knowledge segment. This contextual information can be used to trace the source of retrieved target entities, target relationships, and target knowledge segments, thereby improving the interpretability of the large-scale question answering model. The source and version can serve as reference information for the first preprocessing or fusion described below. The first preprocessing or fusion can be referred to in the relevant embodiments below, which will not be elaborated upon here.

[0031] In step 220, first metadata related to the data domain in the knowledge segment is identified, and second metadata associated with the first metadata is obtained from the data warehouse based on the first metadata.

[0032] The first metadata can be keywords related to the data domain in the knowledge segmentation. Specifically, keyword recognition can be achieved according to the configured entity type. The configured entity type can be associated with specific business scenarios in the data domain. The configuration of entity type can be referred to the following related embodiments.

[0033] A data warehouse stores structured data consisting of primary metadata and related secondary metadata. This structured data describes the relationship between the primary metadata and the corresponding secondary metadata. For example, in the e-commerce domain, the primary metadata might be the e-commerce-related keyword "sales data table" in a knowledge segment, and the secondary metadata might be the metadata containing "field A" within the "sales data table." Therefore, if "sales data table" and "field A" are extracted from the knowledge segment, and the semantics of the knowledge segment cannot represent an inclusion relationship between "sales data table" and "field A," then the inclusion relationship between "sales data table" and "field A" can be obtained based on the secondary metadata.

[0034] In step 230, entities and relationships between entities in the knowledge segment are determined based on the first target context, wherein the first target context includes second metadata.

[0035] Following the example above, entities in different business scenarios within the data domain can be information such as datasets, data tables, fields, metrics, and definitions. Relationships can be the inclusion relationship between data tables and fields, the definition relationship between metrics and definitions, and so on.

[0036] In some embodiments, step 230 can be implemented using a large model (i.e., a large language model). For example, step 230 can be implemented by: constructing prompt words based on a first target context; and determining entities and relationships between entities in the knowledge segment based on the prompt words using a second large model.

[0037] As can be seen from the above, the first target context may include the second metadata. Figure 3 This is a schematic diagram illustrating a process for generating a knowledge graph according to an embodiment of this disclosure. (Refer to...) Figure 3 The system identifies the first metadata in the knowledge segment and the data domain, obtains the second metadata from the data warehouse based on the first metadata, generates the first target context based on the second metadata, uses the knowledge segment and the first target context as input to the second major model, and uses the second major model to extract entities and relationships between entities in the knowledge segment based on the first target context.

[0038] As an example, the first target context may include second metadata, target type, and target entity relationship template.

[0039] In some embodiments, the target type can be determined by: displaying a configuration interface that shows the types of candidate entities in different business scenarios under the data domain; and determining the target type in response to a first configuration operation for the types of candidate entities in the configuration interface.

[0040] As an example, the candidate entity type includes at least one of the following: dataset type, caliber type, indicator type, data table type, field type, task type, and interface type, and the entity extracted from the knowledge segment belongs to the target type.

[0041] The first configuration operation for the configuration interface can be the selection of candidate entities, and the target type is determined based on the type of the selected candidate entity.

[0042] After the target type is determined, the configuration interface can further display the entity relationship template associated with the target type. In response to the second configuration operation on the entity relationship template associated with the target type displayed on the configuration interface, the target entity relationship template is determined.

[0043] The entity relationship template describes the definition of the relationship between entities in the corresponding business scenario, such as the inclusion relationship mentioned above. Different entity relationship definition templates can be pre-configured for different business scenarios. After the target type is determined, the corresponding entity relationship template is displayed for users to select directly or edit further, thereby supporting users to quickly configure entity relationships.

[0044] In step 240, a knowledge graph is generated based on entities and relationships, whereby the knowledge graph is used to support large model question answering.

[0045] A knowledge graph is structured data consisting of nodes and edges between them. Each node corresponds to an entity, and edges describe the relationship between two connected entities.

[0046] In the data domain of knowledge segmentation, two independent concepts (i.e. entities) may not be able to determine whether they are related through semantics. However, by using the structured metadata stored in the data warehouse as background knowledge to determine the relationship between two entities in the knowledge segmentation, the problem of not being able to obtain the relationship between certain entities through the semantic understanding results of knowledge segmentation can be solved. A unified knowledge graph is constructed based on the metadata and knowledge documents of the data warehouse, which improves the reliability and accuracy of the knowledge graph, and thus improves the performance of large model question answering that relies on the knowledge graph.

[0047] In some embodiments, the knowledge segments of a knowledge document may include multiple segments. To improve the efficiency of entity recognition and relationship recognition, steps 220 and 230 may be executed in parallel for different knowledge segments.

[0048] In some embodiments, before performing step 240, the following steps may also be performed: performing a first preprocessing on the first data to obtain second data, wherein the first data includes entities and relations determined from all corresponding knowledge segments, and generating a knowledge graph through the second data.

[0049] The first preprocessing step is used to improve the quality of the data used to generate the knowledge graph. As an example, the first preprocessing step may include at least one of standardization and filtering processes.

[0050] Standardization ensures that all extraction results (i.e., primary data) follow a uniform format. For example, entity identifiers or names must use a uniform naming convention.

[0051] Filtering is used to improve the accuracy of data and resolve redundant relationships between data. Filtering can be performed from multiple dimensions, which may include confidence, duplication, and conflict dimensions.

[0052] The second model carries a confidence level for each extraction result (entity or entity relationship). Extraction results with confidence levels higher than the confidence level threshold can be retained, while extraction results with confidence levels lower than the confidence level threshold can be deleted.

[0053] The second model may contain duplicate extraction results. You can keep one of the duplicate extraction results and delete the redundant ones.

[0054] The second model may extract conflicting entity relations. To resolve these conflicts, multiple conflicting entity relations can be selected based on the source and version of the knowledge segment to which they belong. Specifically, the entity relation with the highest source credibility among the conflicting entity relations can be retained, or the latest version of the conflicting entity relations can be retained.

[0055] By using the above method, the extraction results of the second major model are subjected to the first preprocessing, thereby improving the quality of the data used to generate the knowledge graph.

[0056] In some embodiments, step 240 above can be implemented in the following way: dividing the second data into multiple data groups; sequentially traversing the multiple data groups and performing the following steps for the traversed data groups until all data groups in the multiple data groups have been traversed: fusing the second data in the currently traversed data group with the data in the current knowledge graph to generate a fused knowledge graph.

[0057] In this way, the second data is divided into multiple data groups, and by traversing each data group, it is integrated with the current knowledge graph in batches, thereby avoiding the situation of consuming a large amount of computing resources due to the one-time reconstruction of the knowledge graph and reducing the reconstruction risk.

[0058] Continue to refer to Figure 2 Fusion can refer to the integration of incremental and existing data. Incremental data refers to the results of performing a first preprocessing on entities and relationships extracted from knowledge documents, while existing data refers to the data in the current knowledge graph, which is obtained from the current knowledge graph. For example, fusion can refer to aggregating entities with the same name or construction in the second data set and the current knowledge graph, and establishing a mapping relationship of aliases. When aggregating entities with the same name or construction, if there is a conflict between the entity relationship in the second data set and the entity relationship in the current knowledge graph, the entity relationship with the highest source credibility is selected based on the source of the knowledge segment to which the entity relationship belongs in the current knowledge graph and the source of the knowledge segment to which the entity relationship belongs in the second data set; or, based on the version of the knowledge segment to which the entity relationship belongs in the current knowledge graph and the version of the knowledge segment to which the entity relationship belongs in the second data set, the latest version of the entity relationship is selected.

[0059] Understandably, the current knowledge graph in the next traversal is the fused knowledge graph generated in the previous traversal, and the fused knowledge graph generated in the last traversal can be used to support the question answering of the first major model.

[0060] The above methods improve the accuracy of the generated knowledge graph, thus providing a reliable data foundation for enhancing the performance of large-scale question answering models that rely on knowledge graphs.

[0061] Continue to refer to Figure 2 In some embodiments, the knowledge graph and vector database are updated synchronously based on the fusion of the incremental and existing data. The vector database stores entity vectors and relation vectors to support knowledge graph-based retrieval. For information on knowledge graph retrieval, please refer to the following related embodiments, which will not be elaborated upon here.

[0062] In some embodiments, the above-described knowledge graph-based large model question answering method may further include the following steps: determining the query intent of a user query; if the query intent satisfies a hybrid retrieval chain, performing a retrieval in the knowledge graph based on the user query to obtain a first retrieval result, the first retrieval result including target entities and target relationships; and obtaining a second retrieval result, the second retrieval result including target knowledge fragments; and determining a second target context based on the first retrieval result and the second retrieval result, wherein the second target context is used to support question answering of the first large model.

[0063] Figure 4 This is a schematic diagram illustrating a large-scale question-and-answer process according to an embodiment of this disclosure. (Refer to...) Figure 4 In knowledge graph-based retrieval, both local and global search modes are supported. These modes have different query scopes and are suitable for different scenarios. Local search mode searches within a specific sub-region of the knowledge graph—the neighborhood of the anchor entity in the user's query. It is faster and suitable for single-entity queries. Global search mode covers the entire knowledge graph, is slower, and is suitable for queries involving multiple entities and complex relationships.

[0064] In some embodiments, the retrieval mode can be adaptively selected based on the user's query intent. For example, if the query intent is determined to be to retrieve a single entity, a local retrieval mode can be selected; if the query intent is determined to be to retrieve complex relationships, a global retrieval mode can be selected.

[0065] Continue to refer to Figure 4 After determining the retrieval mode, starting from the entity in the user query, and based on the semantics of that entity and the semantics of each node in the knowledge graph, semantic relevance is used to locate nodes in the knowledge graph with the same semantics as the entity in the user query. Then, using graph traversal, based on either a local retrieval mode or a global retrieval mode, the target entities and target relationships related to the located nodes are retrieved from the knowledge graph. For example, for the entity "payment amount field" in the user query, locating the node "payment amount field" in the knowledge graph, in a local retrieval mode, the adjacent nodes of "payment amount field" can be directly found, such as the "indicator" directly associated with "payment amount field"; in a global retrieval mode, the path of multi-step relationships can be queried, such as "payment amount field - order table - associated report - indicator in the report".

[0066] In some embodiments, the vector database may store vectors of each knowledge segment of each knowledge document, and a retrieval may be performed based on the vector of the user query and the vectors of each knowledge segment in the document knowledge base to obtain target knowledge fragments that are semantically related to the user query.

[0067] In some embodiments, entities and relationships can be associated with corresponding knowledge fragments. Therefore, after obtaining the target entity and target relationship, the knowledge fragment associated with the target entity and target relationship can be obtained and used as the target knowledge fragment.

[0068] In some embodiments, graph indexes and vector indexes can be constructed simultaneously to achieve efficient retrieval of knowledge graphs in vector databases and graph databases.

[0069] Continue to refer to Figure 4 This allows for a second preprocessing step on the target entity and target knowledge segments to improve the quality of the data used to generate the second target context.

[0070] As an example, the second preprocessing may include standardization processing, performing standardization processing on different target knowledge segments from different sources and / or different versions, to achieve the unification of terms and alignment of entities in different target knowledge segments from different sources and / or different versions.

[0071] As an example, the second preprocessing includes deduplication, which can be performed on the target entity or target knowledge in segments to reduce redundancy in the search results.

[0072] As an example, the second preprocessing includes ranking, such as ranking the search results after performing standardization and deduplication sequentially, and selecting the top-ranked search results to generate the second target context. Ranking can be performed according to multiple dimensions, including relevance, coverage, and credibility. Coverage describes the amount of information a target entity covers in the knowledge graph; nodes with more edges in the knowledge graph cover more information than nodes with fewer edges. Credibility can be determined by source or version; for example, a target entity with a newer version has higher credibility than a target entity with an older version.

[0073] In addition, different ranking dimensions can be configured with different priorities. In terms of relevance, coverage and credibility mentioned above, relevance has a higher priority than credibility, and credibility has a higher priority than coverage.

[0074] For example, the search results in the first set are sorted from high to low relevance to obtain the second set. The first set includes the search results after standardization and deduplication have been performed on the first and second search results respectively. The search results in the second set are sorted a second time according to the coverage from high to low to obtain the third set. In the third set, the search results with high relevance are placed before the search results with low relevance. The search results in the third set are sorted a third time according to their credibility from high to low, resulting in the fourth set. In the fourth set, the search results with high relevance are placed before the search results with low relevance, and the search results with high coverage are placed before the search results with low coverage.

[0075] Based on the same concept, embodiments of this disclosure provide a large-scale question-answering device based on a knowledge graph. Figure 5 This is a block diagram illustrating a knowledge graph-based large-scale question-answering device according to an embodiment of this disclosure, with reference to... Figure 5 The knowledge graph-based large model question-answering device 500 includes: The first acquisition module 501 is used to acquire the knowledge segments corresponding to the knowledge document; The identification module 502 is used to identify the first metadata related to the data domain in the knowledge segment, and to obtain the second metadata associated with the first metadata from the data warehouse based on the first metadata; The first determining module 503 is used to determine the entities and relationships between entities in the knowledge segment based on the first target context, wherein the first target context includes the second metadata; The first generation module 504 is used to generate a knowledge graph based on the entities and the relationships, wherein the knowledge graph is used to support question answering in the first major model.

[0076] Optionally, the knowledge graph-based large model question-answering device 500 further includes: The display module is used to display the configuration interface, wherein the configuration interface displays the types of candidate entities for different business scenarios under the data domain; The first response module is used to determine the target type in response to a first configuration operation for the type of the candidate entity in the configuration interface. The second response module is used to respond to a second configuration operation on the entity relationship template associated with the target type displayed on the configuration interface, and to determine the target entity relationship template; The knowledge graph-based large model question answering device 500 also includes: The second generation module is used to generate a first target context based on the second metadata, the target type, and the target entity relationship template.

[0077] Optionally, the candidate entity type includes at least one of the following: Dataset type; Caliber type; Indicator type; Data table type; Field type; Task type; Interface type.

[0078] Optionally, the knowledge graph-based large model question-answering device 500 further includes: A preprocessing module is configured to perform a first preprocessing on first data to obtain second data before generating a knowledge graph based on the entities and relationships. The first data includes the entities and relationships determined from all the knowledge segments. The knowledge graph is generated from the second data. The first preprocessing is used to improve the quality of the data used to generate the knowledge graph.

[0079] Optionally, the first generation module 504 is used for: The second data is divided into multiple data groups; The plurality of data groups are traversed sequentially, and the following steps are performed on each traversed data group until all data groups have been traversed: The second data in the currently traversed data group and the data in the current knowledge graph are merged to generate a merged knowledge graph.

[0080] Optionally, the first determining module 503 is used to: Based on the context of the first target, construct prompt words; The second major model uses the prompt words to determine the entities and relationships between entities in the knowledge segment.

[0081] Optionally, the knowledge graph-based large model question-answering device 500 further includes: The second determination module is used to determine the user's query intent; The retrieval module is configured to perform a retrieval in the knowledge graph based on the user query when the query intent satisfies the hybrid retrieval link, so as to obtain a first retrieval result, wherein the first retrieval result includes target entities and target relationships; The second acquisition module is used to acquire a second search result, wherein the second search result includes the target knowledge fragment; The third determining module is used to determine a second target context based on the first retrieval result and the second retrieval result, wherein the second target context is used to support the question answering of the first large model.

[0082] The implementation methods of each module in the above-mentioned knowledge graph-based large model question answering device 500 can refer to the above-mentioned related method embodiments, and will not be repeated here.

[0083] Based on the same concept, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the above-described knowledge graph-based large model question-answering method.

[0084] Based on the same concept, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described knowledge graph-based large model question-answering method.

[0085] Based on the same concept, embodiments of this disclosure provide an electronic device, including: A storage device on which computer programs are stored; A processing device is used to execute the computer program in the storage device to implement the steps of the above-described knowledge graph-based large model question answering method.

[0086] The following is for reference. Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 1 The diagram below shows the structure of the system 600. The terminal devices in this embodiment may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0087] like Figure 6 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0088] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0089] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0090] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0091] In some implementations, electronic devices can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communications (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0092] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0093] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a knowledge segment corresponding to a knowledge document; identify first metadata related to the data domain in the knowledge segment, and acquire second metadata associated with the first metadata from a data warehouse based on the first metadata; determine entities and relationships between entities in the knowledge segment based on a first target context, wherein the first target context includes the second metadata; and generate a knowledge graph based on the entities and the relationships, wherein the knowledge graph is used to support question answering of a first major model.

[0094] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The modules described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the modules are not necessarily limiting in certain circumstances; for example, the first acquisition module can also be described as "a module for acquiring knowledge segments corresponding to knowledge documents".

[0097] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0098] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0099] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0100] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

Claims

1. A large-scale question-answering method based on knowledge graphs, characterized in that, include: Obtain the knowledge segments corresponding to the knowledge document; Identify the first metadata related to the data domain in the knowledge segment, and obtain the second metadata associated with the first metadata from the data warehouse based on the first metadata; Based on the first target context, the entities and relationships between entities in the knowledge segment are determined, wherein the first target context includes the second metadata; A knowledge graph is generated based on the entities and the relationships, wherein the knowledge graph is used to support question answering in the first major model.

2. The large-scale question-answering method according to claim 1, characterized in that, The large-model question answering method also includes: The configuration interface displays the types of candidate entities for different business scenarios under the data domain. In response to a first configuration operation for the type of the candidate entity in the configuration interface, a target type is determined; In response to a second configuration operation on the entity relationship template associated with the target type displayed on the configuration interface, a target entity relationship template is determined; The large-model question answering method also includes: A first target context is generated based on the second metadata, the target type, and the target entity relationship template.

3. The large-scale question-answering method according to claim 2, characterized in that, The candidate entity types include at least one of the following: Dataset type; Caliber type; Indicator types; Data table type; Field type; Task type; Interface type.

4. The large-scale question-answering method according to claim 2, characterized in that, Before generating the knowledge graph based on the entities and relationships, the large model question answering method further includes: The first data is subjected to a first preprocessing to obtain the second data, wherein the first data includes the entities and relationships determined from all the knowledge segments, and the knowledge graph is generated from the second data. The first preprocessing is used to improve the quality of the data used to generate the knowledge graph.

5. The large-scale question-answering method according to claim 4, characterized in that, The step of generating a knowledge graph based on the entities and the relationships includes: The second data is divided into multiple data groups; The plurality of data groups are traversed sequentially, and the following steps are performed on each traversed data group until all data groups have been traversed: The second data in the currently traversed data group and the data in the current knowledge graph are merged to generate a merged knowledge graph.

6. The large-scale question-answering method according to claim 1 or 2, characterized in that, The step of determining the entities and relationships between entities in the knowledge segment based on the first target context includes: Based on the context of the first target, construct prompt words; The second major model uses the prompt words to determine the entities and relationships between entities in the knowledge segment.

7. The large-scale question-answering method according to claim 1 or 2, characterized in that, The large-model question answering method also includes: Determine the user's query intent; When the query intent satisfies the hybrid retrieval link, a retrieval is performed in the knowledge graph based on the user query to obtain a first retrieval result, wherein the first retrieval result includes the target entity and the target relationship; Obtain a second search result, wherein the second search result includes the target knowledge fragment; Based on the first search result and the second search result, a second target context is determined, wherein the second target context is used to support the question answering of the first large model.

8. A large-scale question-answering device based on knowledge graphs, characterized in that, include: The first acquisition module is used to acquire the knowledge segments corresponding to the knowledge document; The identification module is used to identify the first metadata related to the data domain in the knowledge segment, and to obtain the second metadata associated with the first metadata from the data warehouse based on the first metadata; The first determining module is configured to determine the entities and relationships between entities in the knowledge segment based on a first target context, wherein the first target context includes the second metadata; The first generation module is used to generate a knowledge graph based on the entities and the relationships, wherein the knowledge graph is used to support question answering in the first major model.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processing device, the computer program performs the steps of the method according to any one of claims 1-7.

10. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-7.

Images