Large language model natural language processing method based on knowledge graph and preference alignment

Abstract

The invention relates to a big language model natural language processing method based on mapping knowledge domain and preference alignment, which comprises the following steps: acquiring a mapping knowledge domain of a specific field, and converting a triple in the mapping knowledge domain into a natural language text; generating a corresponding negative sample tail entity for the triple, and forming a negative sample tetrad; performing optimization training on the large language model through a preference alignment optimization algorithm by utilizing a negative sample tetrad, so that the optimized large language model generates stronger preference for answers conforming to the knowledge graph; and obtaining a specific question and answer data set in the field, and further finely adjusting the large language model trained by the preference alignment optimization algorithm through the question and answer data set to adapt to the question and answer task of the target field. By means of the design, the content conforming to knowledge graph facts is preferentially selected when the answers are generated, the illusion phenomenon is remarkably reduced, and the accuracy and consistency of the facts for generating the answers are improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a natural language processing method based on a large language model with knowledge graph and preference alignment. Background Technology

[0002] In recent years, methods based on Large Language Models (LLMs) have made significant progress in Natural Language Processing (NLP), particularly in tasks such as question answering, dialogue generation, and text generation. However, despite their excellence in many tasks, LLMs still face hallucination, where generated answers, while fluent and grammatically correct, lack factual basis and may even contain logical errors. This problem is particularly severe in knowledge-intensive fields such as medicine, law, finance, and engineering, significantly impacting the practicality and reliability of the models.

[0003] To improve the performance of large language models in domain-specific question answering, many studies have attempted to introduce knowledge graphs (KGs) as external knowledge sources, leveraging structured domain-specific knowledge to enhance the model's reasoning ability and knowledge consistency. Through knowledge graph injection, the model can rely on real-world professional knowledge rather than language pattern learning during the question-answering process, thereby improving the accuracy of the answers. However, existing knowledge graph injection methods often face the following problems: 1. The integration of knowledge graphs and language models is not tight, and there is a lack of effective mechanisms to ensure that the information in the graphs can be effectively utilized by large language models.

[0004] 2. The generated answers are inconsistent. Despite the introduction of knowledge graphs, the model is still prone to making unfounded or erroneous inferences on complex problems.

[0005] 3. The phenomenon of hallucination is quite serious. Even with the use of knowledge graph injection, the generated answers may still be inconsistent with the facts, especially when dealing with tasks involving reasoning and complex knowledge.

[0006] Therefore, there is an urgent need in this field for a technical solution that can deeply integrate knowledge graphs into model training and ensure the accuracy of generated answers through effective optimization mechanisms.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The technical problem addressed in this application is how to solve the issue of low credibility of answers generated by large language models in knowledge-intensive tasks due to a lack of factual basis. Specifically, this manifests as how to overcome the "illusion" phenomenon produced by large language models in knowledge-intensive question-answering tasks and improve the factual accuracy, consistency, and reliability of their generated answers. To this end, this application proposes a natural language processing method for large language models based on knowledge graph and preference alignment to solve the aforementioned technical problem.

[0009] The technical solution adopted in this application to solve the above-mentioned technical problems is as follows.

[0010] The first aspect of this application provides a natural language processing method based on a large language model aligned with knowledge graphs and preferences, comprising the following steps: S1. Knowledge Graph Textification Steps: Obtain a knowledge graph for a specific domain, and convert the triples (h, r, t) in the knowledge graph into natural language text, where h represents the head entity, r represents the relationship between entities, and t represents the tail entity; S2. Negative sample construction steps: For the triple (h, r, t), generate the corresponding negative sample tail entity t′ to form a negative sample quad (h, r, t, t′). S3. Preference Alignment Optimization Steps: Using negative sample quadruplets (h, r, t, t′), the large language model is optimized and trained through the preference alignment optimization algorithm, so that the optimized large language model generates a stronger preference for answers that conform to the knowledge graph. S4. Task-Specific Fine-Tuning Step: Obtain a specific question-and-answer dataset within the domain, and further fine-tune the large language model trained in step S2 using the question-and-answer dataset to adapt it to the question-and-answer task of the target domain.

[0011] In some embodiments, the knowledge graph originates from the fields of medicine, law, finance, or engineering.

[0012] In some embodiments, the specific way to convert the triples (h, r, t) in the knowledge graph into natural language text in step S1 is to concatenate the head entity h, the inter-entity relationship r, and the tail entity t according to a preset template.

[0013] In some embodiments, the generation of the negative sample tail entity t′ in step S2 includes: sampling and replacing other entities in the knowledge graph that are semantically similar to or belong to the same category as the tail entity t.

[0014] In some embodiments, sampling and replacement are performed from other entities in the knowledge graph that are semantically similar to or belong to the same class as the tail entity t. Specifically, a local neighborhood negative sampling strategy is adopted, including the following steps: sorting all entities in the knowledge graph according to a preset rule to form an entity sequence; for a given triple (h, r, t), locating the position index i of the tail entity t in the entity sequence; constructing a local neighborhood candidate set C with index i as the center and a preset window size w, which contains entities from index iw to i+w; uniformly and randomly sampling an entity from the local neighborhood candidate set C as the negative sample tail entity t′.

[0015] In some embodiments, the preference alignment optimization algorithm in step S3 is the DPO algorithm. When the negative sample quadruple (h, r, t, t′) is used for optimization training through the DPO algorithm, the corresponding loss function is constructed based on the preference difference between the rejected negative sample output and the selected correct sample output.

[0016] The second aspect of this application provides a natural language processing system for a large language model based on knowledge graphs and preference alignment, comprising: a knowledge graph processing module configured to acquire a knowledge graph of a specific domain, convert triples (h, r, t) in the knowledge graph into natural language text, and generate corresponding negative sample tail entities t′, forming negative sample quadruplets (h, r, t, t′), where h represents the head entity, r represents the relationship between entities, and t represents the tail entity; a preference alignment optimization module configured to use the negative sample quadruplets (h, r, t, t′) to optimize and train the large language model using a preference alignment optimization algorithm, so that the optimized large language model generates a stronger preference for answers that conform to the knowledge graph; and a question-answering generation and reasoning module configured to acquire a specific question-answering dataset within the domain, and further fine-tune the large language model trained by the preference alignment optimization module using the question-answering dataset to adapt it to the question-answering task of the target domain.

[0017] In some embodiments, when the knowledge graph processing module generates a negative sample tail entity t′ by entity replacement, it is configured to sample and replace from other entities in the knowledge graph that are semantically similar to or belong to the same class as the tail entity t.

[0018] A third aspect of this application provides an electronic device comprising: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs contain instructions for performing the method of the first aspect of this application.

[0019] The fourth aspect of this application provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the method of the first aspect of this application.

[0020] The present invention has the following beneficial effects: This application injects accurate and reliable sources of factual knowledge into the model by transforming structured triples in a domain-specific knowledge graph into natural language text, ensuring from the outset that the generated content is factually grounded. Building upon this, high-quality negative sample quadruples are constructed by generating negative sample tail entities that are semantically similar to the correct tail entity. This design creates a series of adversarial examples, enabling the large language model to clearly distinguish subtle factual differences during training. This significantly enhances the model's ability to identify and avoid generating false or erroneous information (i.e., illusions), solving the technical problem of "illusion" phenomena generated by existing large language models in knowledge-intensive question-answering tasks.

[0021] Furthermore, this application utilizes negative sample quadruples to optimize the large language model through a preference alignment optimization algorithm (such as the DPO algorithm), which systematically strengthens the large language model's preference for correct answers that conform to the knowledge graph, while suppressing the selection of incorrect negative samples, thereby significantly improving the factual accuracy of the generated answers.

[0022] Furthermore, this application achieves final application deployment and performance enhancement through a task-specific fine-tuning step. This step acquires a domain-specific question-and-answer dataset and further fine-tunes the large language model optimized in the aforementioned steps. This enables the large language model to not only master general domain knowledge and error-detection capabilities but also adapt to the specific context and format requirements of the target domain's question-and-answer tasks. This phased training approach ensures that the large language model can provide stable, reliable, and domain-conventional answers to domain problems of varying complexity, ultimately guaranteeing high consistency and reliability of the output results.

[0023] In summary, this application, through the synergy of knowledge injection, negative sample adversarial training, preference alignment, and task-specific fine-tuning, enables large language models to form long-term, stable preferences for correct facts in the parameter space, thereby fundamentally improving the factual accuracy and reliability of generated answers.

[0024] Other beneficial effects of the present invention will be further described below. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the training framework of this application. Figure 2 This is a sampling diagram corresponding to a local neighborhood negative sampling strategy. Figure 3 This is a schematic diagram of the experimental results corresponding to Example 1; Figure 4 This is a schematic diagram of the experimental results corresponding to Comparative Example 2. Detailed Implementation

[0026] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0027] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] The core of this invention lies in using an end-to-end training framework to transform domain-specific knowledge graphs into natural language text for fine-tuning large language models. Based on this, high-quality negative samples are constructed and combined with preference alignment optimization algorithms (such as DPO) to optimize the model's training. This enables the model to generate answers with a strong preference for factual content consistent with the knowledge graph, significantly reducing illusion phenomena and improving accuracy and reliability in professional domain question answering. Specifically, the execution flow of the training framework is as follows: Figure 1 As shown.

[0029] In some embodiments, the training framework comprises three modules: a knowledge graph processing module, a preference alignment optimization module, and a question-answering generation and inference module. The first two modules are the core innovative modules, which perform preference training on the knowledge graph based on the construction of negative samples. Together with the last module, they ensure the efficiency and reliability of the model in professional domain question-answering tasks. Furthermore, this technical framework is only for model training and can also be used in conjunction with post-processing methods such as retrieval enhancement.

[0030] In some embodiments, the core task of the knowledge graph processing module is to acquire a knowledge graph of a specific domain (such as medical knowledge, legal clauses, financial risks, engineering specifications, etc.), and to transform the triples (h, r, t) in the knowledge graph into natural language text that a large language model can understand. The large language model is then fine-tuned to obtain a knowledge-enhanced model. The core task of the negative sample generation module is to generate corresponding negative samples for the triples (h, r, t) in the knowledge graph using an entity replacement strategy, forming negative sample quadruples.

[0031] First, the knowledge graph for a specific domain is preprocessed. Taking medicine as an example, the authoritative Unified Medical Language System (UMLS) is selected. This graph includes a medical metadata dictionary, semantic relationships between entities, and a natural language conversion dictionary. Entities and relationships are mapped onto this graph, and abnormal entity / relation mappings are filtered out to obtain a standardized triplet dataset. .

[0032] Next, the triples are textualized. The triples (h, r, t) in the knowledge graph are transformed into natural language text. For any triple... Construct corresponding samples The instruction section uses the following template: The prompt explicitly provides the head entity name and relation name in English, requiring the model to complete the tail entity. The relation 'r' needs to be adjusted appropriately based on the specific relation. The input portion provides no additional context and is uniformly set to an empty string; the output portion is the tail entity name, enclosed in English quotation marks. For example, for a given triple... The corresponding instruction sample is as follows:

[0033] Then, refer to Figure 2 A local neighborhood negative sampling strategy based on the entity dictionary is adopted. After completing the concept mapping and cleaning in the previous step, the entity set V is arranged into a linear list according to a stable order (lexicographical order of concept unique identifiers): Each This corresponds to a normalized entity name. For any triplet... Find the position corresponding to the tail entity in the entity sequence. And construct a local candidate set centered on this: ,in This refers to the window size. In this patent experiment, the size is... And in the implementation, out-of-bounds indexes are truncated. Then, from the local neighborhood... A single entity is uniformly and randomly sampled as a negative sample. .

[0034] This strategy has the following characteristics: Negative samples originate from the local neighborhood of tail entities in the entity sequence. Except for boundary cases, they usually have certain similarities in semantic type, specialty, or term structure, making them more likely to constitute representative interference items and avoiding the generation of "overly obvious" wrong options.

[0035] Window size The range of the "semantic distance" between negative and positive samples was controlled: smaller It tends to generate more "difficult" negative samples, but the risk of noise also increases accordingly; larger This approach offers a trade-off between difficulty and noise. This paper selects a relatively stable medium window size within the scope of this task through preliminary experiments.

[0036] In construction Explicit exclusion For entities that have aliases or synonyms, they are uniformly merged during the concept preprocessing stage to further reduce noise caused by ambiguity in preference labels.

[0037] By employing the aforementioned local negative sampling strategy, semantically "similar but incorrect" tail entities can be constructed for the vast majority of triples while ensuring controllable noise, thereby obtaining... Set of quadruplets of the same size .

[0038] In some embodiments, corresponding negative samples can also be generated through relation substitution strategies or inconsistent combination strategies. The core idea of ​​the relation substitution strategy is to construct semantically "seemingly correct" erroneous quadruplets by substituting relations while keeping the head and tail entities unchanged. For a correct triple (h, r, t), a different relation r' (r' ≠ r) is sampled from its knowledge graph relation set, thus forming a negative sample quadruplet. The core idea of ​​the inconsistent combination strategy is to break the inherent logical and type constraints in the knowledge graph and randomly combine head entities, relations, and tail entities from different triples, thereby creating "illusory" quadruplets that are neither factually nor logically valid. The negative samples generated by this strategy are not only factually incorrect but may also violate basic common sense or type constraints (e.g., incorrectly pointing the "treatment" relation of "medicine" to the "city" entity). It aims to train the model to recognize deeper logical inconsistencies, rather than simply memorizing facts.

[0039] It's important to note that the main problems with relation substitution strategies compared to inconsistent combination strategies—specifically, local neighborhood negative sampling strategies—lies in their inefficiency and low quality. They struggle to systematically generate negative samples that effectively challenge the model and target its knowledge weaknesses, resulting in either high costs or poor performance. In contrast, local neighborhood negative sampling strategies cleverly leverage data locality to stably produce a large number of high-quality negative samples in an extremely simple and low-cost manner. These negative samples directly address the essence of the model's "illusion": the confusion of semantically similar entities. This allows for the most efficient improvement of the model's factual accuracy and consistency, systematically solving the key problem of how to generate high-quality (i.e., difficult yet reasonable) negative samples in batches.

[0040] In some embodiments, the preference alignment optimization module improves the quality of the model-generated answers through the preference alignment (DPO) optimization algorithm.

[0041] Optimization goal: During training, adjust the model parameters to make it more biased towards triples that conform to the knowledge graph, thereby prioritizing the generation of factual answers during inference.

[0042] set up The strategy to be optimized Given a reference strategy, The dataset consists of preference samples, each sample containing conditions. With a pair of candidates with preference labels A typical DPO loss can be written as:

[0043] in For the Sigmoid function, For temperature hyperparameters, and Defined as the logarithmic probability difference between the target policy and the reference policy between positive and negative candidates, respectively:

[0044]

[0045] Taking the medical field from the previous section as an example, we prefer data. Derived from the UMLS negative sample quadruple set constructed in the previous section For any quadruple Conditional instructions are Positive and negative candidate outputs The preference relationship is reflected as follows: given a head entity With Relationship Under these conditions, entity Superior to physical .

[0046] Therefore, the core role of DPO in this scenario can be viewed as a kind of "knowledge preference alignment": by optimizing the DPO loss function within the UMLS instruction context, the model is guided to align with the given... The probability structure that is closer to the facts of the knowledge graph is formed in the subspace of the "predicted tail entity", thereby suppressing the output that seriously conflicts with the graph.

[0047] This module effectively reduces hallucinations during the generation process, improving the accuracy and consistency of the answers.

[0048] In some embodiments, the question-answering generation reasoning module is configured to provide more tailored answers to domain-specific questions, requiring task-specific supervised fine-tuning of the relevant data.

[0049] Taking the medical field as an example, multiple multiple-choice question-answering datasets for medical professional examinations, including MedQA and USMLE, can be selected, and these datasets can be divided into training and testing sets. The training set can then be fine-tuned in a supervised manner using Low-Rank Adaptation (LoRA).

[0050] To further illustrate the feasibility, practical constraints, and advantages of this invention, comparative embodiments are provided below. These embodiments may be adopted under engineering implementation or resource constraints, but their effects are significantly inferior to the optimal solution of this invention. The core point of comparison lies in whether or not the crucial step of combining negative samples with preference alignment (DPO) is incorporated.

[0051] Comparative Example 1: No DPO preference optimization was introduced.

[0052] This approach directly performs supervised fine-tuning of the base model without introducing negative samples. Although this approach can directly internalize knowledge graph knowledge through fine-tuning, it lacks adversarial training with erroneous samples and has a relatively high rate of answering illusions.

[0053] Taking the medical field as an example, we selected a subset of the UMLS knowledge graph (containing approximately 1.17 million triples) and used Llama 3.1-8B-Instruct as the base model. During fine-tuning, the learning rate was set to 1e-6, and training was performed for 5 epochs. We tested on the MedQA dataset test set, using accuracy as the primary evaluation metric. Figure 3 As shown, the results indicate that the accuracy of the model in this invention improved from 63.55% to 67.40% compared to the model without DPO.

[0054] Comparison and disadvantages with the present invention: This invention incorporates knowledge graph facts into the model training process as positive and negative samples through negative sample generation and preference alignment optimization. This allows the model to learn a "preference for correct facts" in the parameter space, unlike the approach that only introduces the correct answer. Therefore, compared to Example 1, it still has shortcomings in terms of generation consistency and illusion suppression.

[0055] Comparative Example 2: No task-specific fine-tuning was introduced.

[0056] This approach does not perform task-specific fine-tuning after DPO (Diagnosis Based on Test Results), directly using the model after DPO as the final solution. This approach is very similar to the training process of this patent, but due to the lack of relevant training sets, it will produce a small bias in the answers.

[0057] Taking the medical field as an example from Comparative Example 1, we selected a subset of the UMLS knowledge graph (containing approximately 1.17 million triples) and used Llama 3.1-8B-Instruct as the base model. During fine-tuning, the learning rate was set to 1e-6, and training lasted for 5 epochs. This comparative example uses LoRA for DPO, with the rank set to 128 and the beta set to 0.1. Full-parameter DPO can also be performed when the training cost is sufficient. We directly tested on the MedQA test set, such as... Figure 4 As shown, compared with the model in Control Example 2, the accuracy of the model in this invention patent increased from 65.67% to 67.40%.

[0058] In summary, this invention has clear innovative points and substantial technical contributions in the following core aspects: Firstly, knowledge graph injection strategies and textualization mechanisms.

[0059] This invention proposes a knowledge graph injection method for knowledge-intensive scenarios, which transforms structured triplet facts into natural language expressions and systematically incorporates them into the question-answering training process. This injection is not merely used as retrieval text, but becomes a normalized input structure during the model training phase, ensuring that the model can learn over a long period and use domain facts for reasoning, unlike pure retrieval enhancement methods.

[0060] Secondly, negative sample and preference alignment optimization integration. This invention constructs a quadruple (h, r, t, t′) with negative samples and integrates DPO into the domain knowledge alignment process. This significantly enhances the model's preference for outputs that conform to the knowledge graph, rather than relying solely on linguistic statistics or the surface distribution of training data. This strategy enables large language models to learn to distinguish between true facts and erroneous reasoning, thereby reinforcing their preference for correct facts. This adversarial training method has rarely been systematically applied to question-answering generation tasks that combine knowledge graphs in existing research. Its innovation lies in improving the model's anti-illusion ability through negative samples.

[0061] Third, a general domain adaptation framework This invention provides a universal framework adaptable to various knowledge-intensive fields (law, finance, engineering, medicine, etc.), eliminating the need for developing dedicated engines for each field; only the domain knowledge graph and appropriate training data need to be replaced. This significantly enhances the practicality and application value of the technology. Furthermore, this framework is designed solely for model training and can be combined with post-processing methods such as retrieval enhancement.

[0062] Fourth, improve factual consistency and generate explainability. Because the model training incorporates knowledge graph facts and adversarial example constraints, the generated answers exhibit higher consistency and interpretability. Compared to existing systems that solely rely on local hints or retrieval strategies, this invention significantly improves the credibility and traceability of the model output.

[0063] In summary, this invention significantly improves the performance of large language models in domain-specific question answering by combining knowledge graph injection with DPO optimization. Specific technical effects include: 1. Reduce hallucination phenomenon: Through negative sample adversarial training and DPO preference alignment optimization, this invention effectively reduces hallucination phenomenon, making the generated answers more consistent with the facts and have higher reliability.

[0064] 2. Improve reasoning accuracy and consistency: By optimizing the model with preference alignment, it prioritizes the selection of the correct answer during the reasoning process, thereby improving the accuracy and consistency of the question-answering system.

[0065] 3. High scalability: This invention employs a universal knowledge graph injection method, applicable to knowledge-intensive tasks in various fields, such as law, finance, and engineering. By replacing knowledge graphs and datasets from different fields, it can quickly adapt to question-answering tasks in various professional domains.

[0066] 4. Enhance the professionalism and interpretability of the model: The generated answers can be traced back to their logical basis, which greatly enhances the interpretability of the model in question answering in various fields.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] The background section of this invention may include background information about the problems or environment in which the invention is being developed, and is not necessarily a description of prior art. Therefore, the content included in the background section does not constitute an admission of prior art by the applicant.

[0072] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A natural language processing method based on a large language model with knowledge graph and preference alignment, characterized in that, Includes the following steps: S1. Knowledge Graph Textification Steps: Obtain a knowledge graph for a specific domain, and convert the triples (h, r, t) in the knowledge graph into natural language text, where h represents the head entity, r represents the relationship between entities, and t represents the tail entity; S2. Negative sample construction steps: For the triple (h, r, t), generate the corresponding negative sample tail entity t′ to form a negative sample quad (h, r, t, t′). S3. Preference Alignment Optimization Step: Using the negative sample quadruple (h, r, t, t′), the large language model is optimized and trained using the preference alignment optimization algorithm, so that the optimized large language model generates a stronger preference for answers that conform to the knowledge graph. S4. Task-specific fine-tuning step: Obtain a specific question-and-answer dataset within the domain, and further fine-tune the large language model trained in step S2 using the question-and-answer dataset to adapt it to the question-and-answer task of the target domain.

2. The method according to claim 1, characterized in that, The knowledge graph comes from the fields of medicine, law, finance, or engineering.

3. The method according to claim 1, characterized in that, The specific method for converting the triples (h, r, t) in the knowledge graph into natural language text in step S1 is as follows: the head entity h, the inter-entity relationship r, and the tail entity t are concatenated according to a preset template.

4. The method according to claim 1, characterized in that, The method for generating the negative sample tail entity t′ in step S2 includes: sampling and replacing other entities in the knowledge graph that are semantically similar to or belong to the same category as the tail entity t.

5. The method according to claim 4, characterized in that, The sampling and replacement is performed on other entities in the knowledge graph that are semantically similar to or belong to the same category as the tail entity t. Specifically, a local neighborhood negative sampling strategy is adopted, which includes the following steps: sorting all entities in the knowledge graph according to a preset rule to form an entity sequence; for a given triple (h, r, t), locating the position index i of the tail entity t in the entity sequence; constructing a local neighborhood candidate set C with index i as the center and a preset window size w, which contains entities from index iw to i+w; uniformly and randomly sampling an entity from the local neighborhood candidate set C as the negative sample tail entity t′.

6. The method according to claim 1, characterized in that, The preference alignment optimization algorithm in step S3 is the DPO algorithm. When the negative sample quadruple (h, r, t, t′) is optimized and trained using the DPO algorithm, the corresponding loss function is constructed based on the preference difference between the rejected negative sample output and the selected correct sample output.

7. A natural language processing system based on a large language model aligned with knowledge graphs and preferences, characterized in that, include: The knowledge graph processing module is configured to acquire a knowledge graph of a specific domain, convert the triples (h, r, t) in the knowledge graph into natural language text, and generate corresponding negative sample tail entities t′, forming a negative sample quadruple (h, r, t, t′), where h represents the head entity, r represents the relationship between entities, and t represents the tail entity. The preference alignment optimization module is configured to use the negative sample quadruple (h, r, t, t′) to optimize and train a large language model through a preference alignment optimization algorithm, so that the optimized large language model generates a stronger preference for answers that conform to the knowledge graph. The question-answer generation and reasoning module is configured to acquire a specific question-answer dataset within the domain, and further fine-tune the large language model trained by the preference alignment optimization module using the question-answer dataset to adapt it to the question-answering task of the target domain.

8. The system according to claim 7, characterized in that, When the knowledge graph processing module generates a negative sample tail entity t′ by entity replacement, it is configured to sample and replace other entities in the knowledge graph that are semantically similar to or belong to the same category as the tail entity t.

9. An electronic device, characterized in that, include: One or more processors; Memory; One or more computer programs; wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs containing instructions for performing the method as claimed in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program / instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 6.

Images