Enhancement method and device of large language model, electronic equipment, medium and product
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AERO ENGINE ACAD OF CHINA
- Filing Date
- 2026-02-04
- Publication Date
- 2026-06-02
Smart Images
Figure CN122132531A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, medium, and product for enhancing a large language model. Background Technology
[0002] With the rapid development of information technology, the amount of data on the internet is increasing daily, and people's demand for information retrieval and acquisition is also rising. In the current field of artificial intelligence, large language models are particularly important for various natural language processing tasks.
[0003] In related technologies, large language models, through pre-training on massive text corpora, can generate rich, coherent, and context-appropriate responses. However, when dealing with domain-specific problems, large language models still face challenges such as high training costs and insufficient domain knowledge. Summary of the Invention
[0004] This disclosure addresses some of the shortcomings mentioned in the background art by providing a method, apparatus, electronic device, medium, and product for enhancing large language models.
[0005] In a first aspect, embodiments of this disclosure provide a method for enhancing a large language model, comprising:
[0006] Identify multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target node corresponding to the target business and the target association relationship between the target nodes; By integrating the target nodes and the target associations, a target knowledge graph corresponding to the target business is obtained; Based on the target knowledge graph, specific nodes and specific relationships that meet the preset similarity conditions with the target question are determined, and the specific nodes and specific relationships are determined as relevant knowledge information; The target question and the relevant knowledge information are input into the large language model to obtain the target prediction result of the target question.
[0007] In one embodiment of the first aspect, determining the multi-source heterogeneous data related to the target service and processing the multi-source heterogeneous data to obtain the target node corresponding to the target service and the target association relationship between the target nodes includes: Collect multi-source heterogeneous data corresponding to the target service; Based on the multi-source heterogeneous data, the construction information of the knowledge graph for the target business is determined; wherein, the construction information is used to indicate the nodes and relationships required to construct the knowledge graph; Based on the constructed information, entity extraction is performed on multi-source heterogeneous data to obtain the target node and the target association relationship.
[0008] In one embodiment of the first aspect, the step of extracting data from multi-source heterogeneous data based on the construction information to obtain the target node and the target association relationship includes: The multi-source heterogeneous data is split into text to obtain target statements related to the target business. Entity extraction is performed on the target statement to obtain the initial entity; Based on the construction information, the initial entities are classified to obtain the target nodes and the target association relationships.
[0009] In one embodiment of the first aspect, integrating the target node and the target association to obtain the target knowledge graph corresponding to the target service includes: Clustering is performed on the target nodes and the target associations to obtain an entity set; wherein, the entity set includes several clusters, and each cluster includes target nodes and target associations of the same type; The target knowledge graph is constructed based on the entity set.
[0010] In one embodiment of the first aspect, determining, based on the target knowledge graph, designated nodes and designated associations that meet preset similarity conditions with the target question, and determining the designated nodes and designated associations as relevant knowledge information, includes: Determine the degree of association between each target entity in the target knowledge graph and the target question; Based on the degree of association, the target entities are sorted in descending order to obtain the target sequence; The first preset number of target entities in the target sequence are identified as designated entities that meet the preset similarity conditions; The nodes and relationships in the specified entity are respectively defined as specified nodes and specified relationships; Based on the target question, the specified node and the specified relationship are organized to obtain the relevant knowledge information related to the target question.
[0011] In one embodiment of the first aspect, determining the degree of association between each target entity in the target knowledge graph and the target question includes: The similarity between each target entity and the target question is calculated to obtain the degree of association between each target entity and the target question.
[0012] In a second aspect, embodiments of this disclosure provide an enhancement apparatus for a large language model, comprising: The first determining module is used to determine multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target node corresponding to the target business and the target association relationship between the target nodes; An integration module is used to integrate the target node and the target association to obtain the target knowledge graph corresponding to the target business; The second determining module is used to determine, based on the target knowledge graph, a specified node and a specified association that meet the preset similarity conditions with the target question, and to determine the specified node and the specified association as relevant knowledge information; The prediction module is used to input the target question and the relevant knowledge information into the large language model to obtain the target prediction result of the target question.
[0013] In a third aspect, an electronic device is provided, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the memory via the bus, and the machine-readable instructions, when executed by the processor, perform the steps of the first aspect above, or any possible implementation of the first aspect.
[0014] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, performs the steps of the first aspect or any possible implementation thereof.
[0015] In a fifth aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described in the preceding aspects.
[0016] As will be described in detail below, there are methods, apparatuses, electronic devices, media, and products for enhancing large language models according to embodiments of this disclosure. By processing multi-source heterogeneous data related to target businesses, target nodes and target relationships are extracted and integrated into a target knowledge graph, supplementing the model with structured knowledge of vertical domains and making up for the domain knowledge shortcomings of large language models.
[0017] By selecting knowledge information highly relevant to the target question from the target knowledge graph and inputting it along with the target question into the large language model to generate target prediction results, no additional pre-training of the large language model is required, reducing the training cost of adapting the model to specific domains. At the same time, with the support of structured knowledge, the accuracy and reliability of the large language model's answers to questions in specific domains are improved, thus enhancing the user experience. Attached Figure Description
[0018] Figure 1A flowchart illustrating a method for enhancing a large language model as provided in this disclosure embodiment; Figure 2 A method for determining the entity set in an enhancement method for a large language model provided in this disclosure embodiment; Figure 3 A schematic diagram of an enhancement device for a large language model provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation
[0019] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present disclosure and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the drawings, not the entire structure.
[0020] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0021] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.
[0022] Research has shown that with the rapid development of information technology, the amount of data on the internet is increasing daily, and people's demand for information retrieval and acquisition is also rising. In the current field of artificial intelligence, large language models are particularly important for various natural language processing tasks.
[0023] In related technologies, large language models, through pre-training on massive text corpora, can generate rich, coherent, and context-appropriate responses. However, when dealing with domain-specific problems, large language models still face challenges such as high training costs and insufficient domain knowledge.
[0024] Based on the above research, this disclosure provides an enhancement method for a large language model. By processing multi-source heterogeneous data related to the target business, the target nodes and target relationships are extracted and integrated into a target knowledge graph, which supplements the model with structured knowledge of the vertical domain and makes up for the domain knowledge shortcomings of the large language model.
[0025] By selecting knowledge information highly relevant to the target question from the target knowledge graph and inputting it along with the target question into the large language model to generate target prediction results, no additional pre-training of the large language model is required, reducing the training cost of adapting the model to specific domains. At the same time, with the support of structured knowledge, the accuracy and reliability of the large language model's answers to questions in specific domains are improved, thus enhancing the user experience.
[0026] See Figure 1 The diagram shows a flowchart of a method for enhancing a large language model provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein: S101. Determine the multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target nodes corresponding to the target business and the target association relationships between the target nodes.
[0027] In the embodiments of this disclosure, after identifying multi-source heterogeneous data, preprocessing can be performed on the multi-source heterogeneous data to obtain preprocessed multi-source heterogeneous data. Preprocessing includes, but is not limited to, cleaning and removing redundant data.
[0028] Next, coarse-grained preprocessing can be performed on the preprocessed multi-source heterogeneous data to obtain intermediate text data, and then the intermediate text data can be split to obtain the target sentences. Here, coarse-grained preprocessing refers to the process of preprocessing and splitting the text data.
[0029] Next, fine-grained information extraction can be performed on the target statement to obtain the target nodes corresponding to the target business and the target association relationships between the target nodes. Among them, fine-grained information extraction is used to indicate the process of entity extraction from the statement.
[0030] S102. Integrate the target nodes and target relationships to obtain the target knowledge graph corresponding to the target business.
[0031] In the embodiments of this disclosure, the degree of association between target nodes and target relationships can be determined to obtain several clusters. Within each cluster, the degree of association between target nodes and target relationships satisfies preset association conditions.
[0032] Here, the basic structure and units of the knowledge graph can be generated based on the target nodes and target relationships within each cluster. Then, the relationships between the basic and structural units within each cluster are determined, resulting in the target knowledge graph corresponding to the target business.
[0033] S103. Based on the target knowledge graph, identify the specified nodes and specified relationships that meet the preset similarity conditions with the target question, and determine the specified nodes and specified relationships as relevant knowledge information.
[0034] In the embodiments of this disclosure, designated nodes and designated relationships that meet preset similarity conditions with the target question can be determined in the target knowledge graph.
[0035] Then, knowledge points related to the target question can be generated based on specified nodes and specified relationships, and these knowledge points can be identified as relevant knowledge information.
[0036] S104. Input the target question and related knowledge information into the large language model to obtain the target prediction result of the target question.
[0037] In the embodiments of this disclosure, relevant knowledge information can be identified as supplementary knowledge for the large language model in the target business. Subsequently, the large language model can predict the target question based on the relevant knowledge information, obtaining the target prediction result for the target question.
[0038] Here, a three-part prompt can be set for the courtyard model. The three-part prompt includes: a task description for information extraction; key information extracted from the graph database (i.e., the aforementioned relevant knowledge information); and a standardized output format.
[0039] The task description for information extraction can be used to introduce different tasks in different scenarios and set corresponding goals and requirements.
[0040] Here, when the density of the provided entity and relation information is relatively loose, it is necessary to organize the information into fluent natural language text, while when the key information contains a large amount of text, it is necessary to extract the answers that are truly relevant to the user's query.
[0041] Here, the key information extracted from the graph database is concatenated from the target knowledge graph by using arrays or dictionaries to form a structured data representation. This provides rich and accurate key knowledge for the large language model, thereby helping the large language model to better understand the question and generate answers.
[0042] Here, for the output format, corresponding specifications are set according to the target question to ensure that the large language model's answer balances high accuracy and concise and clear format, providing users with a better user experience.
[0043] In the embodiments of this disclosure, firstly, multi-source heterogeneous data related to the target business are identified, and the multi-source heterogeneous data are processed to obtain the target nodes corresponding to the target business and the target association relationships between the target nodes; secondly, the target nodes and target association relationships are integrated to obtain the target knowledge graph corresponding to the target business; thirdly, based on the target knowledge graph, specified nodes and specified association relationships that meet the preset similarity conditions with the target question are identified, and the specified nodes and specified association relationships are identified as relevant knowledge information; finally, the target question and relevant knowledge information are input into the large language model to obtain the target prediction result of the target question.
[0044] In the above implementation, by processing multi-source heterogeneous data related to the target business, the target nodes and target relationships are extracted and integrated into a target knowledge graph, which supplements the model with structured knowledge of the vertical domain and makes up for the domain knowledge shortcomings of the large language model.
[0045] By selecting knowledge information highly relevant to the target question from the target knowledge graph and inputting it along with the target question into the large language model to generate target prediction results, no additional pre-training of the large language model is required, reducing the training cost of adapting the model to specific domains. At the same time, with the support of structured knowledge, the accuracy and reliability of the large language model's answers to questions in specific domains are improved, thus enhancing the user experience.
[0046] In an optional embodiment, multi-source heterogeneous data related to the target service is determined, and the multi-source heterogeneous data is processed to obtain the target nodes corresponding to the target service and the target association relationships between the target nodes. Specifically, the steps include the following: First, the multi-source heterogeneous data corresponding to the target business; Secondly, based on multi-source heterogeneous data, the construction information of the knowledge graph for the target business is determined; among which, the construction information is used to indicate the nodes and relationships required to construct the knowledge graph.
[0047] Finally, based on the constructed information, entity extraction is performed on the multi-source heterogeneous data to obtain the target nodes and target association relationships.
[0048] In the embodiments of this disclosure, multi-source heterogeneous data (including structured data, unstructured text, semi-structured documents, etc.) related to the target business can be collected based on the core scenario of the target business (e.g., vertical domain question answering).
[0049] Subsequently, the construction information of the knowledge graph can be clarified by analyzing the content characteristics, business attributes, and core requirements of multi-source heterogeneous data.
[0050] The construction information includes the types of nodes, relationships, and their corresponding quantities required to build the knowledge graph. Node types must cover all business-related entities in the dataset, and relationship types must fully cover the inter-node relationships in the data. Furthermore, the design of all nodes and relationships must closely align with the domain knowledge characteristics of the target business, ensuring a high degree of consistency between the construction information and the target business scenario.
[0051] Subsequently, data can be extracted from multi-source heterogeneous data based on the construction information to obtain target nodes and target relationships that conform to the construction information.
[0052] In an optional embodiment, data extraction is performed on multi-source heterogeneous data based on construction information to obtain target nodes and target association relationships, specifically including the following steps: First, the multi-source heterogeneous data is split into text to obtain target statements related to the target business. Secondly, entity extraction is performed on the target statement to obtain the initial entity; Finally, the initial entities are classified based on the construction information to obtain the target nodes and target associations.
[0053] In the embodiments of this disclosure, unstructured text, semi-structured documents, and other data in multi-source heterogeneous data can be split into intermediate sentences.
[0054] Then, duplicate text and text unrelated to the target business can be deleted from the intermediate statements to obtain the target statements related to the knowledge characteristics of the target business domain.
[0055] Here, for unstructured text (i.e., target statements) with obvious identifying words in multi-source heterogeneous data, such as unstructured text that directly contains hints such as named entities and relationships, the keyword matching method is used directly to classify the text, that is, to determine the text category of the target statement.
[0056] Here, for unstructured text without obvious identifying words in multi-source heterogeneous data, information extraction models can be used to extract specific text content based on the semantics of the unstructured text. Specifically, a model can use a self-attention mechanism to capture the overall semantic information of the sentence and then determine its category based on the semantics.
[0057] After identifying the target statement, entity extraction can be performed on the target statement to obtain the initial nodes and initial relationships.
[0058] Next, we can analyze the relationship between the initial nodes and the initial associations, that is, analyze the associations between the initial nodes.
[0059] Here, after the initial entities are determined, they can be classified and organized according to the node type classification criteria specified in the construction information. Entities of the same type are uniformly assigned to the corresponding node type to form target nodes (including node name and type attributes).
[0060] Here, we can combine the relation type definition in the constructed information, mine the semantic association between target nodes after classification through the target model, identify the connection relationship between nodes that conforms to business logic, clarify the relation type and the corresponding node pair, and finally obtain the target node and the target association relationship.
[0061] In an optional embodiment, the target nodes and target associations are integrated to obtain the target knowledge graph corresponding to the target business, specifically including the following steps: First, the target nodes and their relationships are clustered to obtain an entity set. The entity set includes several clusters, and each cluster contains target nodes and their relationships of the same type. Then, a target knowledge graph is constructed based on the entity set.
[0062] In the disclosed embodiments, the target node and target association can be vectorized to obtain the vectorized target entity. Specifically, the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm can be used to vectorize the target node and target association.
[0063] Here, the vectorized target entities can be clustered to obtain an entity set. For example, each vectorized target entity can be considered as an initial cluster, generating a target number of clusters. Here, the target number refers to the number of vectorized target entities.
[0064] Next, the similarity between any two clusters can be calculated using Manhattan distance as a similarity metric. Then, iterative merging can be performed, finding the two clusters with the highest similarity in each iteration and merging them into a new cluster. Simultaneously, the similarity between the new cluster and other clusters is recalculated so that merging can continue in the next iteration.
[0065] Finally, the above steps are repeated until all vectorized target entities are merged into a large cluster, or until a preset iteration condition is met, resulting in an entity set. The preset iteration condition is either a preset number of iterations or the similarity between vectorized target entities in each cluster of the entity set is greater than a preset first similarity threshold.
[0066] Here, Neo4j graph database can be used to construct knowledge graphs for vertical domains based on entity sets. Neo4j boasts more powerful graph database features and the intuitive and easy-to-use query language Cypher, enabling efficient storage, retrieval, and querying of large amounts of complex, interconnected, and unstructured data.
[0067] Here, by writing Cypher statements, entities, relationships, and semantic information in the fused knowledge can be mapped to nodes and relationships between nodes in the graph database.
[0068] Specifically, by executing Cypher statements, entities and relationships can be organized in the form of a graph to form a structured knowledge network (i.e., the target knowledge graph mentioned above).
[0069] Here, when new external knowledge is generated, entity references in the new data can be linked to corresponding entities in the knowledge base, establishing semantic relationships between them and updating the structure and content of the target knowledge graph. During the construction process, potential contradictions or conflicts can be resolved to ensure the consistency and integrity of the target knowledge graph.
[0070] Here, refer to Figure 2 The diagram illustrates a method for determining the entity set in the large language model enhancement method provided in this embodiment of the disclosure, wherein: S21. Vectorize the target nodes and target associations to obtain vectorized target entities, and determine each vectorized target entity as a cluster.
[0071] S22. Determine the similarity between any two clusters using Manhattan distance.
[0072] S23. Merge the two clusters with the highest similarity into a new cluster.
[0073] S24. Recalculate the similarity between the new cluster and other clusters using Manhattan distance.
[0074] S25. Determine whether each cluster meets the preset iteration conditions.
[0075] S26. If the preset iteration conditions are met, obtain the entity set; otherwise, repeat S22.
[0076] In an optional embodiment, based on the target knowledge graph, designated nodes and designated relationships that meet preset similarity conditions with the target question are determined, and the designated nodes and designated relationships are identified as relevant knowledge information, specifically including the following steps: First, determine the degree of association between each target entity in the target knowledge graph and the target question; Secondly, the target entities are sorted in descending order based on their degree of relevance to obtain the target sequence; Secondly, a preset number of target entities in the target sequence are identified as specified entities that meet the preset similarity conditions; Secondly, the nodes and relationships within the specified entity are defined as specified nodes and specified relationships, respectively. Finally, based on the target question, the specified nodes and their relationships are organized to obtain relevant knowledge information related to the target question.
[0077] In embodiments of this disclosure, each target entity in the target knowledge graph can be encoded to obtain a text representation corresponding to each target entity. This encoding of target entities can be performed using a neural network model.
[0078] For example, the text2vec sentence representation model based on ranking loss uses the CoSENT calculation method in the last layer of the model. It takes the target entity as input and outputs a numerical vector as the text representation of the target entity.
[0079] Here, when the user inputs a target question, the target question can be vectorized to obtain the vectorized target question.
[0080] Subsequently, based on the above text representation, the similarity between the vectorized target question and each target entity in the target knowledge graph can be determined, and this similarity can be determined as the degree of association between each target entity in the target knowledge graph and the target question.
[0081] Here, determining the degree of association between each target entity in the target knowledge graph and the target question involves the following steps: The similarity between each target entity and the target question is calculated to obtain the degree of association between each target entity and the target question.
[0082] Among them, the cosine similarity between the target entity and the target question can be determined as the degree of association between each target entity and the target question.
[0083] Cosine similarity compares the differences between two vectors by calculating the cosine of the angle between them in a vector space. For example, the more similar two vectors are, the smaller the angle between them, and the greater the cosine similarity.
[0084] Here, the selection threshold for the number of designated entities can be preset based on the complexity of the target question, the size of the target knowledge graph, and the data composition; the first preset number of target entities are extracted from the target sequence and identified as designated entities that meet the preset similarity conditions, ensuring that the selected entities have the best semantic relevance to the target question.
[0085] Here, based on the structured storage logic of the target knowledge graph, the node attributes (including entity name, type, etc.) corresponding to each specified entity are extracted and identified as specified nodes; at the same time, all relationship types and corresponding connected node pairs associated with the specified entity in the knowledge graph are mined and identified as specified associations to ensure the integrity of specified nodes and specified associations.
[0086] Here, based on the semantic requirements of the target question, the specified nodes and specified relationships are structured and organized. Entity types, relationship types and node connection logic are encapsulated in the form of arrays or dictionaries. Redundant related information that is irrelevant to the target question is eliminated, forming clear and semantically accurate relevant knowledge information, which provides reliable support for the subsequent generation of answers by the large language model.
[0087] Based on the same inventive concept, this disclosure also provides a large language model enhancement device corresponding to the large language model enhancement method. Since the principle of the device in this disclosure for solving the problem is similar to the large language model enhancement method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0088] Reference Figure 3 The diagram shown is a schematic of an enhancement device for a large language model provided in an embodiment of this disclosure. The device includes: a first determining module 31, an integration module 32, a second determining module 33, and a prediction module 34; wherein: The first determining module is used to determine multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target node corresponding to the target business and the target association relationship between the target nodes; An integration module is used to integrate the target node and the target association to obtain the target knowledge graph corresponding to the target business; The second determining module is used to determine, based on the target knowledge graph, a specified node and a specified association that meet the preset similarity conditions with the target question, and to determine the specified node and the specified association as relevant knowledge information; The prediction module is used to input the target question and the relevant knowledge information into the large language model to obtain the target prediction result of the target question.
[0089] This disclosure embodiment processes multi-source heterogeneous data related to the target business, extracts target nodes and target relationships, and integrates them into a target knowledge graph, supplementing the model with structured knowledge of the vertical domain and making up for the domain knowledge shortcomings of the large language model.
[0090] By selecting knowledge information highly relevant to the target question from the target knowledge graph and inputting it along with the target question into the large language model to generate target prediction results, no additional pre-training of the large language model is required, reducing the training cost of adapting the model to specific domains. At the same time, with the support of structured knowledge, the accuracy and reliability of the large language model's answers to questions in specific domains are improved, thus enhancing the user experience.
[0091] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.
[0092] Corresponding to Figure 1 In addition to methods for enhancing large language models, this disclosure also provides an electronic device 400, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 400 provided in an embodiment of this disclosure, including: The system includes a processor 41, a memory 42, and a bus 43. The memory 42 stores execution instructions and includes main memory 421 and external memory 422. The main memory 421, also called internal memory, temporarily stores the computational data in the processor 41, as well as data exchanged with external memory such as a hard disk. The processor 41 exchanges data with the external memory 422 through the main memory 421. When the electronic device 400 is running, the processor 41 communicates with the memory 42 through the bus 43, causing the processor 41 to execute the following instructions: Identify multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target node corresponding to the target business and the target association relationship between the target nodes; By integrating the target nodes and the target associations, a target knowledge graph corresponding to the target business is obtained; Based on the target knowledge graph, specific nodes and specific relationships that meet the preset similarity conditions with the target question are determined, and the specific nodes and specific relationships are determined as relevant knowledge information; The target question and the relevant knowledge information are input into the large language model to obtain the target prediction result of the target question.
[0093] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0094] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0095] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.
[0096] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.
[0097] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.
[0098] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.
[0099] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.
Claims
1. A method for enhancing a large language model, characterized in that, include: Identify multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target node corresponding to the target business and the target association relationship between the target nodes; By integrating the target nodes and the target associations, a target knowledge graph corresponding to the target business is obtained; Based on the target knowledge graph, specific nodes and specific relationships that meet the preset similarity conditions with the target question are determined, and the specific nodes and specific relationships are determined as relevant knowledge information; The target question and the relevant knowledge information are input into the large language model to obtain the target prediction result of the target question.
2. The method according to claim 1, characterized in that, The process of determining multi-source heterogeneous data related to the target service and processing the multi-source heterogeneous data to obtain the target node corresponding to the target service and the target association relationship between the target nodes includes: Collect multi-source heterogeneous data corresponding to the target service; Based on the multi-source heterogeneous data, the construction information of the knowledge graph for the target business is determined; wherein, the construction information is used to indicate the nodes and relationships required to construct the knowledge graph; Based on the constructed information, entity extraction is performed on multi-source heterogeneous data to obtain the target node and the target association relationship.
3. The method according to claim 2, characterized in that, The step of extracting data from multi-source heterogeneous data based on the constructed information to obtain the target node and the target association relationship includes: The multi-source heterogeneous data is split into text to obtain target statements related to the target business. Entity extraction is performed on the target statement to obtain the initial entity; Based on the construction information, the initial entities are classified to obtain the target nodes and the target association relationships.
4. The method according to claim 1, characterized in that, The process of integrating the target nodes and the target associations to obtain the target knowledge graph corresponding to the target business includes: Clustering is performed on the target nodes and the target associations to obtain an entity set; wherein, the entity set includes several clusters, and each cluster includes target nodes and target associations of the same type; The target knowledge graph is constructed based on the entity set.
5. The method according to claim 1, characterized in that, The step of determining, based on the target knowledge graph, designated nodes and designated relationships that meet preset similarity conditions with the target question, and defining the designated nodes and designated relationships as relevant knowledge information, includes: Determine the degree of association between each target entity in the target knowledge graph and the target question; Based on the degree of association, the target entities are sorted in descending order to obtain the target sequence; The first preset number of target entities in the target sequence are identified as designated entities that meet the preset similarity conditions; The nodes and relationships in the specified entity are respectively defined as specified nodes and specified relationships; Based on the target question, the specified node and the specified relationship are organized to obtain the relevant knowledge information related to the target question.
6. The method according to claim 1, characterized in that, Determining the degree of association between each target entity in the target knowledge graph and the target question includes: The similarity between each target entity and the target question is calculated to obtain the degree of association between each target entity and the target question.
7. An enhancement device for a large language model, characterized in that, include: The first determining module is used to determine multi-source heterogeneous data related to the target business, and process the multi-source heterogeneous data to obtain the target node corresponding to the target business and the target association relationship between the target nodes; An integration module is used to integrate the target node and the target association to obtain the target knowledge graph corresponding to the target business; The second determining module is used to determine, based on the target knowledge graph, a specified node and a specified association that meet the preset similarity conditions with the target question, and to determine the specified node and the specified association as relevant knowledge information; The prediction module is used to input the target question and the relevant knowledge information into the large language model to obtain the target prediction result of the target question.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method for enhancing a large language model according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the enhancement method for the large language model as described in any one of claims 1 to 6.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the enhancement method for the large language model as described in any one of claims 1 to 6.