Information processing method and device, nonvolatile storage medium and computer equipment

Abstract

The invention discloses an information processing method and device, a nonvolatile storage medium and computer equipment. The method comprises the steps that text data and atlas data information corresponding to a target question are obtained, the text data comprise multiple pieces of entity information in the target question and corresponding entity categories, and the atlas data information comprises multiple nodes and the relation between the multiple nodes; the text data and the graph data information are input into a large language model, a target reasoning path corresponding to the text data is determined, the large language model is used for reasoning a target result corresponding to the target problem, and the target reasoning path represents the relation among multiple entity categories in the text data; and based on the target reasoning path, determining a target result corresponding to the target problem through a large language model. According to the method, the technical problem that an existing large language model is prone to model illusion in the field of knowledge retrieval vertical application is solved.

Description

Technical Field

[0001] The present invention relates to the application fields of large language models and knowledge graphs. Specifically, it relates to an information processing method, apparatus, non-volatile storage medium, and computer device. Background Art

[0002] In the context of the rapidly developing information technology, large language models, as the core tools in the field of natural language processing, are gradually penetrating into all walks of life and becoming the key force to promote the informatization and intelligentization of industries. Especially in vertical fields, such as highly professional industries like law, medicine, and finance, the application potential of large language models is huge. They can process complex language structures, understand deep semantic information, and provide efficient data analysis and decision-making support services for the industries.

[0003] However, although large language models demonstrate powerful general understanding and generation capabilities, in vertical fields, the limitations of their reasoning abilities are becoming increasingly prominent. The knowledge characteristics in vertical fields are highly professional, diverse, and complex. Large language models are difficult to accurately understand the professional terms and business logics in these fields, and sometimes even generate self-contradictory or completely irrelevant information, namely the so-called "model hallucination". This phenomenon reduces the practicality of the models. In addition, when large language models process logical reasoning and computational problems, their inherent logic and interpretability also face challenges. Their output results often lack clear reasoning paths and explanations, which makes it difficult for users to verify the correctness of the results when facing complex problems.

[0004] To address the above problems, no effective solutions have been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide an information processing method, apparatus, non-volatile storage medium, and computer device to at least solve the technical problem that current large language models are prone to model hallucinations in the vertical application field of knowledge retrieval.

[0006] According to one aspect of the embodiments of the present invention, an information processing method is provided, including: obtaining text data and graph data information corresponding to a target problem, where the text data includes multiple entity information and corresponding entity categories in the target problem, the graph data information includes multiple nodes and relationships between the multiple nodes, each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category, and the relationships include relationships between entity categories corresponding to each of the multiple nodes and relationships between entity information corresponding to different entity categories; inputting the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information, where the large language model is used to infer a target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data; and determining a target result corresponding to the target problem through the large language model based on the target inference path.

[0007] Optionally, inputting the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information includes: based on the large language model, determining an initial node matching the text data in the graph data information, where the initial node includes an entity category and corresponding entity information; determining multiple inference paths corresponding to the text data based on the initial node; and using the inference paths among the multiple inference paths whose entity category quantity and path length meet a preset condition as the target inference path.

[0008] Optionally, based on the large language model, determining an initial node matching the text data in the graph data information includes: using the entity categories matching the multiple entity categories in the text data in the graph data information as target entity categories; mapping the multiple entity information corresponding to the target entity categories and the multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to the target entity categories and multiple vectors corresponding to the text data; calculating the correlation degrees between the multiple vectors corresponding to the target entity categories and the multiple vectors corresponding to the text data respectively based on a vector similarity algorithm; using the entity information corresponding to the vectors with a correlation degree greater than a first preset threshold among the multiple vectors corresponding to the target entity categories as initial entity information; and using the initial entity information and the corresponding entity category as the initial node.

[0009] Optionally, determining multiple inference paths corresponding to the text data based on the initial node includes: based on the relationships between the multiple nodes in the graph data information, using the initial node as a starting point to determine multiple inference paths including the multiple entity categories in the text data.

[0010] Optionally, based on the target inference path, determine the target result corresponding to the target question through a large language model, including: mapping the multiple entity information corresponding to each of the multiple entity categories of the nodes other than the starting point and the ending point in the target inference path and the multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to each of the multiple entity categories and multiple vectors corresponding to the text data; calculating the relevance between the multiple vectors corresponding to each of the multiple entity categories and the multiple vectors corresponding to the text data respectively based on the vector similarity algorithm and the relationships between the multiple nodes; respectively taking the entity information corresponding to the vectors with a relevance exceeding the second preset threshold among the multiple vectors corresponding to each of the multiple entity categories as the target entity information corresponding to each of the multiple entity categories; determining, based on the large language model, whether the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question; and in the case where the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, determining the target result based on the target entity information.

[0011] Optionally, in the case where the target entity information corresponding to each of the multiple entity categories is not sufficient to answer the target question, it includes: modifying the second preset threshold; repeating the above steps of determining the target entity information until the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, and determining the target result based on the target entity information.

[0012] Optionally, display the target result, the target inference path, and the entity information corresponding to each of the multiple nodes in the target inference path on a preset screen.

[0013] According to another aspect of the embodiments of the present invention, there is also provided an information processing device, including: an acquisition module, configured to acquire text data and graph data information corresponding to a target question, where the text data includes multiple entity information and corresponding entity categories in the target question, the graph data information includes multiple nodes and relationships between the multiple nodes, each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category, and the relationships include relationships between entity categories corresponding to each of the multiple nodes and relationships between entity information corresponding to different entity categories; a first determination module, configured to input the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information, where the large language model is used to infer the target result corresponding to the target question, and the target inference path represents the relationships between multiple entity categories in the text data; and a second determination module, configured to determine the target result corresponding to the target question through the large language model based on the target inference path.

[0014] According to another aspect of the embodiments of the present invention, a non-volatile storage medium is further provided. The non-volatile storage medium includes a stored program, wherein when the program runs, it controls the device where the non-volatile storage medium is located to execute any one of the above information processing methods.

[0015] According to still another aspect of the embodiments of the present invention, a computer device is further provided. The computer device includes a processor for running a program, wherein when the program runs, it executes any one of the above information processing methods.

[0016] According to still another aspect of the embodiments of the present invention, a computer program product is further provided, including a computer program that implements any one of the above information processing methods when executed by a processor.

[0017] In the embodiments of the present invention, an information processing method is adopted. By obtaining text data and graph data information corresponding to a target problem, wherein the text data includes multiple entity information and corresponding entity categories in the target problem, and the graph data information includes multiple nodes and relationships between multiple nodes. Each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category, and the relationships include relationships between entity categories corresponding to each of the multiple nodes and relationships between entity information corresponding to different entity categories; the text data and graph data information are input into a large language model to determine a target inference path corresponding to the text data in the graph data information, wherein the large language model is used to infer a target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data; based on the target inference path, the target result corresponding to the target problem is determined through the large language model, achieving the purpose of integrating a knowledge graph into the inference method of the large language model, thereby realizing the technical effect of improving the accuracy and reliability of the output result of the large language model, and further solving the technical problem that the current large language model is prone to model hallucinations in the vertical application field of knowledge retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 Shows a hardware structure block diagram of a computer terminal for implementing the information processing method;

[0020] Figure 2 Is a flowchart of the information processing method provided according to the embodiments of the present invention;

[0021] Figure 3It is a flowchart of a large language model reasoning method based on a knowledge graph provided by an alternative embodiment of the present invention;

[0022] Figure 4 It is a structural block diagram of an information processing device provided by an embodiment of the present invention. Detailed implementation manners

[0023] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to an embodiment of the present invention, an embodiment of an information processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that here.

[0026] The method embodiment provided in the first embodiment of the present application can be executed on a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structural block diagram of a computer terminal for implementing the information processing method is shown. As Figure 1As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ……, 102n in the figure) (the processor may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown in, or have a different configuration from Figure 1 shown.

[0027] It should be noted that the above one or more processors and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10. As involved in the embodiments of the present application, the data processing circuit is a kind of processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0028] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage devices corresponding to the information processing method in the embodiments of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned application program (information processing method). The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided with respect to the processor, and these remote memories may be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The display may be, for example, a touch-screen liquid crystal display (LCD), which enables a user to interact with the user interface of the computer terminal 10.

[0030] Figure 2 is a schematic flowchart of an information processing method provided according to an embodiment of the present invention. As Figure 2 shown, the method includes the following steps:

[0031] Step S201: Obtain the text data and graph data information corresponding to the target question. Among them, the text data includes multiple entity information and corresponding entity categories in the target question, and the graph data information includes multiple nodes and the relationships between multiple nodes. Each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category. The relationships include the relationships between the entity categories corresponding to each of the multiple nodes, and the relationships between the entity information corresponding to different entity categories.

[0032] In this step, the text data comes from the target question proposed by the user, which can be a natural language query, such as "Person A went on a business trip to Location B for 5 days. What was his accommodation cost during this period?". In this question, there are multiple key entity information, such as "Person A" (representing a specific person), "Location B" (representing a specific location), and "accommodation cost" (representing the specific value to be calculated). Each entity has its corresponding entity category, such as "person", "location", "accommodation cost", etc.

[0033] The graph data information, on the other hand, comes from a pre-constructed knowledge graph. A knowledge graph is a graph database composed of nodes and edges, where nodes represent entities and edges represent the relationships between entities. Each node contains entity category information and multiple entity information corresponding to that category, such as multiple specific people under the "person" category, or multiple specific locations under the "location" category. At the same time, the graph data can also include the relationships between entity categories and the relationships between entity information under different entity categories, such as the relationship between "person" and "job level", or the relationship between "location" and "accommodation standard". By using the graph data information in the knowledge graph as the input information link, the reliability of the large language model in obtaining information can be ensured, effectively reducing the occurrence of model hallucinations. Relying on the rich structured data in the knowledge graph as the information source, the large language model can understand and answer deeper intention questions, expanding the application scope of the model.

[0034] Step S202: Input the text data and graph data information into the large language model to determine the target inference path corresponding to the text data in the graph data information. Among them, the large language model is used to infer the target result corresponding to the target question, and the target inference path represents the relationships between multiple entity categories in the text data.

[0035] In this step, determining the target reasoning path is the core link in the whole process. Its purpose is to find one or more paths connecting the entities in the problem to support the solution of the problem. This path not only reflects the direct relationship between entities but also implies the logical chain behind the problem. For example, in the above target problem, the accommodation cost from "Person A" to "Location B" needs to consider the rank of "Person A", the corresponding accommodation standard for this rank, and the applicability of this standard in "Location B", etc. Based on the determined target reasoning path, the large language model uses the coherent information of the entities and relationships in the path to perform reasoning to generate the target result. This reasoning process combines the model's semantic understanding ability and the structured professional knowledge in the graph to ensure the reliability and accuracy of the result. For example, in the above problem, the model infers the specific value of the accommodation cost of "Person A" during the business trip in "Location B" based on the relationship links of "person" - "rank", "rank" - "accommodation standard", "accommodation standard" - "location", and the information of the number of days of the business trip.

[0036] Step S203: Based on the target reasoning path, determine the target result corresponding to the target problem through the large language model.

[0037] In this step, the target reasoning path can enable the large language model to combine the input problem and the node information of the graph, judge the categories required for the user's problem reasoning, and find the shortest path containing the most category information in the graph structure as the optimal link for subsequent traversal through constraints such as path length optimization and loop information removal. For example, in the above target problem, the target reasoning path can be "person" - "rank" - "accommodation standard" - "location". The large language model can perform the reasoning process based on this structured target reasoning path to generate the target result.

[0038] Specifically, based on the target inference path, in the graph data information, entities in the link are gradually traversed from the starting point, and entity information under each category is pruned and filtered, removing redundant data irrelevant to the user's question, streamlining the link information, and indirectly improving the accuracy of the inference results of the large language model. After the pruning operation is completed, the current link information can be converted into a format with "head entity - relationship - tail entity" triples as the basic unit. For example, in the above target question, the link information can be converted into the format of <1. A [Person]{Position: Development Manager}, corresponding rank of the person, P5 [Rank]{} 2. P5 [Rank]{}, corresponding accommodation standard for the rank, 500 yuan [Accommodation Standard]{} 3. 500 yuan [Accommodation Standard]{}, corresponding area for the accommodation standard, B [Area]{Type: Provincial Capital}>. Among them, <A [A']{A*}, R, B [B']{B*}> in the link represents <Name of the head entity [Category of the head entity]{Attributes of the head entity}, Name of the relationship, Name of the tail entity [Category of the tail entity]{Attributes of the tail entity}> in the relationship triple. The model can first understand the meaning of each triple in the path, that is, the meaning of "head entity - relationship - tail entity", and then make logical inferences based on these relationships and entity information. For example, in the above question, the model will understand that "Person A" has the rank of "Development Manager", and the accommodation standard corresponding to the rank of "Development Manager" in "Location B" is "500 yuan per day". Based on this information, the model can calculate the total accommodation cost required for Person A to travel on business in Location B for 5 days, that is, 500 yuan / day × 5 days = 2500 yuan.

[0039] Through the above steps, the purpose of integrating the knowledge graph into the inference method of the large language model is achieved, thus realizing the technical effect of improving the accuracy and reliability of the output results of the large language model, and further solving the technical problem that the current large language model is prone to model hallucinations in the vertical application field of knowledge retrieval.

[0040] As an optional embodiment, inputting the text data and the graph data information into the large language model to determine the target inference path corresponding to the text data in the graph data information includes: based on the large language model, determining an initial node matching the text data in the graph data information, where the initial node includes an entity category and corresponding entity information; based on the initial node, determining multiple inference paths corresponding to the text data; and using the inference paths whose number of entity categories and path length meet the preset conditions among the multiple inference paths as the target inference paths.

[0041] Optionally, the determination of the initial node is the starting point of the entire reasoning process. Based on the question text proposed by the user, key entity information and its corresponding entity categories can be identified and extracted through a large language model, and the entity categories in the question are matched to the entity categories in the knowledge graph. Determining the reasoning path means finding a path in the knowledge graph data information that can explain and answer the question, representing the connections and relationships between entity categories in the knowledge graph. The large language model determines multiple possible reasoning paths by traversing the knowledge graph and identifying the entity categories related to the question and the relationships between entity categories.

[0042] After determining multiple possible reasoning paths, the next step is to filter out the target reasoning path, that is, the path that can most effectively answer the question. This filtering can be based on preset conditions, such as preferring the path length and removing loop information. Removing loop information ensures that, on the basis of meeting the number of entity categories corresponding to the text data, the situation of loop paths is avoided, and preferring the path length can optimize the length of the reasoning path. The target reasoning path should contain as many entity categories related to the question as possible to ensure that the reasoning process covers all necessary information. For example, for the above question, the path should contain entity categories such as "person", "job level", "accommodation standard", and "location" to ensure the integrity and accuracy of the reasoning result. The length of the path is another consideration factor because it is directly related to the efficiency of reasoning. An overly long path means more calculations and information retrieval, which may introduce unnecessary complexity and errors. Therefore, the length of the target reasoning path should be controlled within a limited range to ensure that the reasoning is both fast and accurate.

[0043] As an optional embodiment, based on the large language model, determining the initial node that matches the text data in the knowledge graph data information includes: using the entity categories that match multiple entity categories in the text data in the knowledge graph data information as the target entity categories; mapping the multiple entity information corresponding to the target entity categories and the multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to the target entity categories and multiple vectors corresponding to the text data; calculating the correlation degrees between the multiple vectors corresponding to the target entity categories and the multiple vectors corresponding to the text data respectively based on the vector similarity algorithm; using the entity information corresponding to the vectors with a correlation degree greater than the first preset threshold among the multiple vectors corresponding to the target entity categories as the initial entity information; using the initial entity information and the corresponding entity categories as the initial node.

[0044] Optionally, in order to associate the target question proposed by the user with the graph data information, the key entity information and entity categories in the target question can be extracted based on a large language model, and at the same time, the node information included in the graph data information can be read. For example, in the above question, the extracted entity categories and entity information can be "Person A" and "Location B". Next, the model will search for entity categories in the graph that match these entity categories. For example, "Person A" matches the "Person" category, and "Location B" matches the "Location" category. These matching entity categories will be used as the target entity categories. Since the entity information in the target question usually cannot be exactly the same as the entity information of the nodes in the graph (such as abbreviations, synonyms, etc.), the extracted entity information cannot have a direct corresponding relationship with the graph nodes. Therefore, multiple entity information under the target entity category and the entity information in the text data can be transformed into a preset vector space, and a vector similarity algorithm can be used to calculate the association score between the two. Finally, the entity category and entity information corresponding to the top 1 node with a relevance score higher than the first preset threshold under the target entity category are used as the initial nodes of the subsequent inference chain.

[0045] For example, for the input text "Person A travels on a business trip to Location B for 5 days. What is the accommodation cost?", the entity categories and entity information extracted by the large language model from this sentence are {"Person": A, "Location": B'}, where, since the location name stored in the graph may be {"Location": B}, and B' is an alias or abbreviation of B, etc., the entity B and the entity B' can be mapped to a preset vector space, and the relevance between the two can be calculated based on the vector similarity. Finally, {"Person": A, "Location": B'} in the input text is mapped to {"Person": A, "Location": B} in the graph as the initial node.

[0046] As an optional embodiment, based on the initial nodes, multiple inference paths corresponding to the text data are determined, including: based on the relationships between multiple nodes in the graph data information, using the initial nodes as the starting points, multiple inference paths including multiple entity categories in the text data are determined.

[0047] Optionally, after determining the initial node, the large language model can use it as a starting point and begin to construct an inference path in the graph data information. The initial node usually contains the most critical entity categories and entity information in the question. The model can start from these entity categories and search for other entity categories and entity information related to the question along the relationship edges in the graph. During the process of constructing the inference path, the large language model will ensure that the path contains multiple entity categories in the text data, such as "person", "location", "accommodation standard", etc. These categories are abstract representations of the attributes or relationships of the entities in the question. The model searches for the entity categories and relationships in the graph that best match the entity categories in the question and constructs the inference path step by step until all entity categories are covered. For example, to answer the question of "the accommodation cost for person A on a 5-day business trip to location B", the model can start from the "person A" node in the "person" category, search for the relationship with the "rank" category (such as having the position of "development manager"), then start from the "development manager" node and search for the relationship with the "accommodation standard" category (such as the accommodation standard is "500 yuan per day"), and finally start from the "accommodation standard" node and search for the relationship with the "location" category (such as the location is "City B"), thus constructing a complete inference path.

[0048] As an optional embodiment, based on the target inference path, the large language model determines the target result corresponding to the target question, including: mapping the multiple entity information corresponding to the multiple entity categories of the nodes in the target inference path except the starting point and the ending point to a preset vector space respectively, together with the multiple entity information in the text data, to obtain multiple vectors corresponding to each of the multiple entity categories and multiple vectors corresponding to the text data; calculating the correlation between the multiple vectors corresponding to each of the multiple entity categories and the multiple vectors corresponding to the text data respectively based on the vector similarity algorithm and the relationships between the multiple nodes; respectively taking the entity information corresponding to the vectors with a correlation exceeding the second preset threshold in the multiple vectors corresponding to each of the multiple entity categories as the target entity information corresponding to each of the multiple entity categories; based on the large language model, determining whether the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question; and in the case where the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, determining the target result based on the target entity information.

[0049] Optionally, based on the pre-screened target inference paths, the entity information under each category can be refined and pruned to remove irrelevant or redundant information. Specifically, after determining the target inference path, the large language model can map the entity categories corresponding to all nodes (except the starting and ending points) in the path and their multiple entity information into a preset vector space to quantify the relevance between the entity information and the question text. Based on the calculated relevance, the large language model can further screen out the entity information corresponding to the vectors whose relevance exceeds the second preset threshold as the target entity information for each entity category. The second preset threshold is a preset value used to filter out entity information with low relevance to the question, only retaining those entities that can provide key information and discarding the remaining entity information to achieve pruning. The pruning strategy effectively filters out redundant data irrelevant to the user's question, streamlines the link information, and indirectly improves the accuracy of the inference results of the large language model.

[0050] For example, in the above question, taking the target inference path "Person" - "Job Level" - "Accommodation Standard" - "Location" as an example, starting from the initial nodes {"Person": A, "Location": B}, multiple entity information under the "Job Level" and "Accommodation Standard" categories are obtained through traversal: {"Job Level": p5, "Job Level": p4, "Job Level": p3}, {"Accommodation Standard": 300 yuan, "Accommodation Standard": 400 yuan, "Accommodation Standard": 500 yuan}. Through the pruning strategy, the relevance between each entity information and the input text is calculated, and the top n entities with a relevance score higher than the preset threshold are retained and added to the current link as first-order entities. For example, in the above information, finally, the nodes {"Job Level": P5} and {"Accommodation Standard": 500 yuan} are retained and added to the existing link.

[0051] Based on the large language model, it can be determined whether the target entity information of each entity category is sufficient to answer the target question. This step involves the model's in-depth understanding of the question and the comprehensive analysis of the entity information to determine whether the currently collected information is sufficient to solve the question. If it is determined that the target entity information is sufficient to answer the target question, the large language model can perform final inference and calculation based on this target entity information to determine the target result. For example, for the above question, the model can calculate the total accommodation cost as 2500 yuan based on the information of "Accommodation standard of 500 yuan per day" and "5 days", and then determine the target result as "The accommodation cost is 2500 yuan".

[0052] As an alternative embodiment, in the case where the target entity information corresponding to each of the multiple entity categories is not sufficient to answer the target question, it includes: modifying the second preset threshold; repeating the above steps of determining the target entity information until the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, and based on the target entity information, determining the target result.

[0053] Optionally, when the model determines that the target entity information currently collected is insufficient to answer the target question, the second preset threshold, that is, the screening criterion for the relevance between entity information and question text, can be adjusted. The originally relatively high second preset threshold may exclude some entity information that has a certain association with the question but a low relevance (lower than the threshold). By lowering this threshold, the model can reconsider the entity information that was previously excluded, thereby increasing the amount of entity data that may be relevant to the question, improving the comprehensiveness of information and the potential to answer the question. After modifying the second preset threshold, the system will re-execute the above-mentioned target entity information determination step, that is, map the entity information corresponding to the nodes other than the starting point and the ending point in the target inference path to the vector space again, and recalculate the relevance between the entity information and the question text based on the new threshold until the target entity information is sufficient to answer the target question.

[0054] As an optional embodiment, the target result, the target inference path, and the entity information corresponding to each of the multiple nodes in the target inference path are displayed on a preset screen.

[0055] Optionally, by displaying the target result, the target inference path, and the entity information of each node in the path, the detailed background and basis of the inference process can be provided, the traceability of the question-and-answer process can be realized, and the transparency and credibility of the result can be enhanced. Users can clearly see how the model starts from the initial entity information, obtains the required information step by step through the entity relationship chain in the knowledge graph, and finally draws a conclusion.

[0056] As an optional embodiment, Figure 3 is a flowchart of a large language model inference method based on a knowledge graph according to an optional embodiment of the present invention, as Figure 3As shown in the figure, the above large language model inference method mainly includes: constructing an entity extraction algorithm based on a large model to identify and extract relevant entities and their categories from the input text, and using a vector similarity calculation method to establish an accurate mapping relationship between these entities and the corresponding entities in the knowledge graph; constructing a pre-screening algorithm for graph links based on a large language model, combining the input text with the category information stored in the graph to find the categories relevant to the input text, and pre-screening the optimal links through these categories, narrowing the search scope and improving the efficiency of information retrieval; constructing an entity pruning algorithm based on a large language model, which, on the basis of the pre-screened optimal links, performs refined pruning and filtering on the entities under each category to remove irrelevant or redundant information, shortening the task processing time and improving the overall efficiency; constructing a link information evaluation algorithm based on a large language model, which is used to filter duplicate information in the link and evaluate whether the filtered link information is sufficient as background knowledge to answer the input question. At the same time, for links with complete information, the question answer and the node and relationship data will be output as the reasoning basis to achieve question and answer traceability and enhance the credibility of the output result.

[0057] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0058] Through the description of the above embodiments, those skilled in the art can clearly understand that the information processing method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0059] According to an embodiment of the present invention, there is also provided an information processing device for implementing the above information processing method. Figure 4 It is a structural block diagram of the information processing device provided according to an embodiment of the present invention. As Figure 4 shown, the information processing device includes: an acquisition module 41, a first determination module 42, and a second determination module 43. The information processing device will be described below.

[0060] An acquisition module 41 is configured to acquire text data and graph data information corresponding to a target problem. The text data includes multiple entity information and corresponding entity categories in the target problem. The graph data information includes multiple nodes and relationships between the multiple nodes. Each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category. The relationships include relationships between entity categories corresponding to each of the multiple nodes and relationships between entity information corresponding to different entity categories.

[0061] A first determination module 42, connected to the acquisition module 41, is configured to input the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information. The large language model is used to infer a target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data.

[0062] A second determination module 43, connected to the first determination module 42, is configured to determine a target result corresponding to the target problem through the large language model based on the target inference path.

[0063] It should be noted here that the above acquisition module 41, first determination module 42, and second determination module 43 correspond to steps S201 to S203 in the embodiment. The instances and application scenarios implemented by the multiple modules and the corresponding steps are the same, but are not limited to the content disclosed in the above embodiment. It should be noted that the above modules, as a part of the device, can run in the computer terminal 10 provided in the embodiment.

[0064] An embodiment of the present invention can provide a computer device. Optionally, in this embodiment, the above computer device can be located in at least one of multiple network devices in a computer network. The computer device includes a memory and a processor.

[0065] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the information processing method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above information processing method. The memory may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely provided with respect to the processor, and these remote memories can be connected to the computer terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0066] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain the text data and graph data information corresponding to the target problem, where the text data includes multiple entity information and corresponding entity categories in the target problem, and the graph data information includes multiple nodes and the relationships between multiple nodes. Each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category. The relationships include the relationships between the entity categories corresponding to each of the multiple nodes, and the relationships between the entity information corresponding to different entity categories; input the text data and graph data information into a large language model to determine the target inference path corresponding to the text data in the graph data information, where the large language model is used to infer the target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data; based on the target inference path, determine the target result corresponding to the target problem through the large language model.

[0067] Optionally, the above-mentioned processor can also execute the program code of the following steps: input the text data and graph data information into a large language model to determine the target inference path corresponding to the text data in the graph data information, including: based on the large language model, determine the initial nodes matching the text data in the graph data information, where the initial nodes include entity categories and corresponding entity information; based on the initial nodes, determine multiple inference paths corresponding to the text data; use the inference paths whose entity category quantity and path length meet the preset conditions among the multiple inference paths as the target inference path.

[0068] Optionally, the above-mentioned processor can also execute the program code of the following steps: based on the large language model, determine the initial nodes matching the text data in the graph data information, including: use the entity categories matching the multiple entity categories in the text data in the graph data information as the target entity categories; map the multiple entity information corresponding to the target entity categories and the multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to the target entity categories and multiple vectors corresponding to the text data; based on the vector similarity algorithm, calculate the correlation degrees between the multiple vectors corresponding to the target entity categories and the multiple vectors corresponding to the text data respectively; use the entity information corresponding to the vectors with a correlation degree greater than the first preset threshold among the multiple vectors corresponding to the target entity categories as the initial entity information; use the initial entity information and the corresponding entity category as the initial node.

[0069] Optionally, the above-mentioned processor can also execute the program code of the following steps: based on the initial nodes, determine multiple inference paths corresponding to the text data, including: based on the relationships between multiple nodes in the graph data information, use the initial nodes as the starting point to determine multiple inference paths including multiple entity categories in the text data.

[0070] Optionally, the above-mentioned processor can also execute the program code of the following steps: Based on the target inference path, determine the target result corresponding to the target question through a large language model, including: mapping the multiple entity information corresponding to the multiple entity categories of the nodes other than the starting point and the ending point in the target inference path and the multiple entity information in the text data into a preset vector space to obtain the multiple vectors corresponding to the multiple entity categories and the multiple vectors corresponding to the text data; calculating the relevance between the multiple vectors corresponding to the multiple entity categories and the multiple vectors corresponding to the text data respectively based on the vector similarity algorithm and the relationships between the multiple nodes; respectively taking the entity information corresponding to the vectors with a relevance exceeding the second preset threshold among the multiple vectors corresponding to the multiple entity categories as the target entity information corresponding to the multiple entity categories; judging whether the target entity information corresponding to the multiple entity categories is sufficient to answer the target question based on the large language model; in the case where the target entity information corresponding to the multiple entity categories is sufficient to answer the target question, determine the target result based on the target entity information.

[0071] Optionally, the above-mentioned processor can also execute the program code of the following steps: In the case where the target entity information corresponding to the multiple entity categories is not sufficient to answer the target question, including: modifying the second preset threshold; repeating the above steps of determining the target entity information until the target entity information corresponding to the multiple entity categories is sufficient to answer the target question, and determining the target result based on the target entity information.

[0072] Optionally, the above-mentioned processor can also execute the program code of the following steps: Display the target result, the target inference path, and the entity information corresponding to each of the multiple nodes in the target inference path on a preset screen.

[0073] Embodiments of the present invention provide an information processing method. By obtaining text data and graph data information corresponding to a target problem, where the text data includes multiple entity information and corresponding entity categories in the target problem, and the graph data information includes multiple nodes and relationships between the multiple nodes, each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category, and the relationships include relationships between entity categories corresponding to each of the multiple nodes and relationships between entity information corresponding to different entity categories; inputting the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information, where the large language model is used to infer a target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data; based on the target inference path, determining a target result corresponding to the target problem through the large language model, thereby achieving the purpose of integrating a knowledge graph into the inference method of the large language model, thus realizing the technical effect of improving the accuracy and reliability of the output result of the large language model, and further solving the technical problem that the current large language model is prone to model hallucinations in the vertical application field of knowledge retrieval.

[0074] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware of the terminal device through a program, and the program can be stored in a non-volatile storage medium, and the storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disc, etc.

[0075] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the above non-volatile storage medium can be used to store the program code executed by the information processing method provided in the above embodiment.

[0076] Optionally, in this embodiment, the above non-volatile storage medium can be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0077] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining text data and graph data information corresponding to a target problem, where the text data includes multiple entity information and corresponding entity categories in the target problem, and the graph data information includes multiple nodes and relationships between the multiple nodes. Each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category. The relationships include relationships between entity categories corresponding to each of the multiple nodes, and relationships between entity information corresponding to different entity categories; inputting the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information, where the large language model is used to infer a target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data; and determining a target result corresponding to the target problem through the large language model based on the target inference path.

[0078] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: inputting the text data and the graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information, including: based on the large language model, determining an initial node matching the text data in the graph data information, where the initial node includes an entity category and corresponding entity information; determining multiple inference paths corresponding to the text data based on the initial node; and using the inference paths among the multiple inference paths whose number of entity categories and path length meet a preset condition as the target inference path.

[0079] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: based on the large language model, determining an initial node matching the text data in the graph data information, including: using the entity categories matching the multiple entity categories in the text data in the graph data information as target entity categories; mapping the multiple entity information corresponding to the target entity categories and the multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to the target entity categories and multiple vectors corresponding to the text data; calculating the correlation degrees of the multiple vectors corresponding to the target entity categories and the multiple vectors corresponding to the text data respectively based on a vector similarity algorithm; using the entity information corresponding to the vectors whose correlation degrees are greater than a first preset threshold among the multiple vectors corresponding to the target entity categories as initial entity information; and using the initial entity information and the corresponding entity category as the initial node.

[0080] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining multiple inference paths corresponding to the text data based on the initial node, including: based on the relationships between multiple nodes in the graph data information, using the initial node as a starting point to determine multiple inference paths including multiple entity categories in the text data.

[0081] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: Based on the target inference path, determine the target result corresponding to the target question through a large language model, including: mapping the multiple entity information corresponding to each of the multiple entity categories corresponding to the nodes other than the starting point and the ending point in the target inference path and the multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to each of the multiple entity categories and multiple vectors corresponding to the text data; calculating the correlation degrees between the multiple vectors corresponding to each of the multiple entity categories and the multiple vectors corresponding to the text data respectively based on the vector similarity algorithm and the relationships between the multiple nodes; respectively taking the entity information corresponding to the vectors with correlation degrees exceeding the second preset threshold among the multiple vectors corresponding to each of the multiple entity categories as the target entity information corresponding to each of the multiple entity categories; judging whether the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question based on the large language model; and in the case where the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, determining the target result based on the target entity information.

[0082] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: In the case where the target entity information corresponding to each of the multiple entity categories is not sufficient to answer the target question, including: modifying the second preset threshold; repeating the above steps of determining the target entity information until the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, and determining the target result based on the target entity information.

[0083] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: Display the target result, the target inference path, and the entity information corresponding to each of the multiple nodes in the target inference path on a preset screen.

[0084] An embodiment of the present invention also provides a computer program product, including a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it can implement: obtaining text data and graph data information corresponding to a target problem, where the text data includes multiple entity information and corresponding entity categories in the target problem, the graph data information includes multiple nodes and relationships between the multiple nodes, each of the multiple nodes includes an entity category and multiple entity information corresponding to the entity category, and the relationships include relationships between entity categories corresponding to each of the multiple nodes, and relationships between entity information corresponding to different entity categories; inputting the text data and graph data information into a large language model to determine a target inference path corresponding to the text data in the graph data information, where the large language model is used to infer a target result corresponding to the target problem, and the target inference path represents the relationships between multiple entity categories in the text data; and determining a target result corresponding to the target problem through the large language model based on the target inference path.

[0085] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0086] In the above embodiments of the present invention, the descriptions of each embodiment have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0087] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in the various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0090] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc.

[0091] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. An information processing method, characterized in that: include: Acquire text data and graph data information corresponding to a target question, wherein the text data includes multiple entity information and corresponding entity categories in the target question, the graph data information includes multiple nodes and relationships between the multiple nodes, the multiple nodes each include an entity category and multiple entity information corresponding to the entity category, and the relationships include relationships between entity categories corresponding to the multiple nodes and relationships between entity information corresponding to different entity categories; Input the text data and the graph data information into a large language model, and determine a target reasoning path corresponding to the text data in the graph data information, wherein the large language model is used to infer a target result corresponding to the target question, and the target reasoning path represents the relationship between multiple entity categories in the text data; Based on the target reasoning path, a target result corresponding to the target question is determined through the large language model.

2. The method according to claim 1, characterized in that The step of inputting the text data and the graph data information into a large language model and determining a target reasoning path corresponding to the text data in the graph data information includes: Based on the large language model, determining an initial node matching the text data in the graph data information, wherein the initial node includes an entity category and corresponding entity information; Based on the initial node, determining multiple reasoning paths corresponding to the text data; The reasoning path whose number of entity categories and path length meet preset conditions among the multiple reasoning paths is taken as the target reasoning path.

3. The method according to claim 2, characterized in that The determining, based on the large language model, an initial node matching the text data in the graph data information includes: Using entity categories in the graph data information that match multiple entity categories in the text data as target entity categories; Mapping a plurality of entity information corresponding to the target entity category and a plurality of entity information in the text data into a preset vector space to obtain a plurality of vectors corresponding to the target entity category and a plurality of vectors corresponding to the text data; Based on a vector similarity algorithm, respectively calculating the correlation between a plurality of vectors corresponding to the target entity category and a plurality of vectors corresponding to the text data; Taking entity information corresponding to a vector having a correlation greater than a first preset threshold among multiple vectors corresponding to the target entity category as initial entity information; The initial entity information and the corresponding entity category are used as the initial node.

4. The method according to claim 2, characterized in that: The step of determining a plurality of reasoning paths corresponding to the text data based on the initial node includes: Based on the relationship between the multiple nodes in the graph data information, the initial node is used as a starting point to determine the multiple reasoning paths containing the multiple entity categories in the text data.

5. The method according to claim 1, characterized in that The determining, based on the target reasoning path, a target result corresponding to the target question through the large language model includes: Mapping multiple entity information corresponding to multiple entity categories corresponding to nodes other than the starting point and the end point in the target reasoning path and multiple entity information in the text data into a preset vector space to obtain multiple vectors corresponding to the multiple entity categories and multiple vectors corresponding to the text data; Based on the vector similarity algorithm and the relationship between the multiple nodes, respectively calculating the correlation between the multiple vectors corresponding to the multiple entity categories and the multiple vectors corresponding to the text data; Respectively taking entity information corresponding to vectors whose correlation exceeds a second preset threshold among the multiple vectors corresponding to the multiple entity categories as target entity information corresponding to the multiple entity categories; Based on the large language model, determining whether the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question; In a case where the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, the target result is determined based on the target entity information.

6. The method according to claim 5, characterized in that When the target entity information corresponding to each of the plurality of entity categories is insufficient to answer the target question, the method includes: Modifying the second preset threshold; Repeat the above step of determining the target entity information until the target entity information corresponding to each of the multiple entity categories is sufficient to answer the target question, and determine the target result based on the target entity information.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: The target result, the target reasoning path, and entity information corresponding to each of the multiple nodes in the target reasoning path are displayed on a preset screen.

8. An information processing device, characterized in that: include: An acquisition module, used to acquire text data and graph data information corresponding to a target question, wherein the text data includes multiple entity information and corresponding entity categories in the target question, the graph data information includes multiple nodes and relationships between the multiple nodes, the multiple nodes each include an entity category and multiple entity information corresponding to the entity category, and the relationship includes a relationship between entity categories corresponding to each of the multiple nodes and a relationship between entity information corresponding to different entity categories; A first determination module is used to input the text data and the graph data information into a large language model to determine a target reasoning path corresponding to the text data in the graph data information, wherein the large language model is used to infer a target result corresponding to the target question, and the target reasoning path represents the relationship between multiple entity categories in the text data; The second determination module is used to determine a target result corresponding to the target question through the large language model based on the target reasoning path.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the information processing method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor executes the information processing method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the information processing method according to any one of claims 1 to 7 is implemented.

Images