Method, System, Medium, and Electronic Device for Correcting Hallucinations in Output of Large Language Models

Abstract

The present invention provides a method, system, medium, and electronic device for correcting hallucinations in the output of a large language model, including the following steps: constructing knowledge graph triples of a product, where the knowledge graph triple database is used to store the product name, product features, and feature parameters of the product; extracting supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and updating the knowledge graph triples; obtaining a product feature query statement of the user; inputting the product feature query statement into the large language model, and the large language model obtains the feature parameters corresponding to the product feature query statement from the knowledge graph triples as a reference response; inputting the reference response and the product feature query statement into the large language model to obtain a corrected response output by the large language model. The method, system, medium, and electronic device for correcting hallucinations in the output of the large language model of the present invention effectively corrects the output hallucinations of the large language model based on knowledge graph triples and interactive Q&A.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular to a method, system, medium, and electronic device for correcting hallucinations in the output of large language models. Background Art

[0002] Large Language Models (LLMs) are usually natural language processing models with large-scale parameters and computing capabilities, such as OpenAI's GPT-3 model. These models can be trained with a large amount of data and parameters to generate human-like text or answer natural language questions. Large language models can not only generate natural language text, but also deeply understand the meaning of the text and handle various natural language tasks, such as text summarization, question answering, translation, etc. Therefore, they have a wide range of applications in the fields of natural language processing, text generation, and intelligent dialogue.

[0003] In the Internet marketing industry, the application of large language models is becoming increasingly common, providing many conveniences and efficient solutions for advertising copywriting generation, market analysis reports, consumer behavior prediction, etc. However, existing large language models still have some challenges and problems, such as large language model hallucinations. The large language model hallucination refers to the fact that large language models will generate inaccurate, irrelevant, or even fictional information, which is neither accurate nor true. Hallucinations can have a serious impact on the application of large language models in various industries, including customer service, financial services, legal decision-making, and medical diagnosis. Summary of the Invention

[0004] In view of the above problems, the purpose of the present invention is to provide a method, system, medium, and electronic device for correcting hallucinations in the output of large language models, which effectively corrects the hallucinations in the output of large language models based on knowledge graph triples and interactive question answering.

[0005] In a first aspect, the present invention provides a method for correcting hallucinations in the output of a large language model, the method comprising the following steps: constructing knowledge graph triples of a product, the knowledge graph triple database being used to store the product name, product features, and feature parameters of the product; extracting supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and updating the knowledge graph triples based on the supplementary product features and the supplementary feature parameters; obtaining a product feature query statement of the user; inputting the product feature query statement into the large language model, and the large language model obtaining the feature parameters corresponding to the product feature query statement from the knowledge graph triples as a reference reply; inputting the reference reply and the product feature query statement into the large language model to obtain a corrected reply output by the large language model.

[0006] In an implementation of the first aspect, updating the knowledge graph triple based on the supplementary product feature and the supplementary feature parameter includes the following steps:

[0007] Determine whether the supplementary product feature is the same as the product feature in the knowledge graph triple;

[0008] When the supplementary product feature is the same as the product feature in the knowledge graph triple, update the corresponding feature parameter in the knowledge graph triple with the supplementary feature parameter;

[0009] When the supplementary product feature is different from the product feature in the knowledge graph triple, add the supplementary product feature and the supplementary feature parameter to the knowledge graph triple.

[0010] In an implementation of the first aspect, determining whether the supplementary product feature is the same as the product feature in the knowledge graph triple includes:

[0011] When the vector similarity between the supplementary product feature and the product feature is greater than a preset threshold, it is determined that the supplementary product feature is the same as the product feature in the knowledge graph triple.

[0012] In an implementation of the first aspect, determining whether the supplementary product feature is the same as the product feature in the knowledge graph triple includes:

[0013] Express the product feature in the knowledge graph triple using the standard product feature field, and map the supplementary product feature to the standard product feature field; when the mapped standard product feature field is the same as the product feature in the knowledge graph triple, it is determined that the supplementary product feature is the same as the product feature in the knowledge graph triple.

[0014] In an implementation of the first aspect, it further includes sorting the product features in the knowledge graph based on the query frequency of the product features.

[0015] In an implementation of the first aspect, inputting the reference reply and the product feature query statement into the large language model includes the following steps:

[0016] Construct an input statement for the large language model based on the question guiding field, the reference reply, and the end-of-sentence prompt for question answering field, where the question guiding field is used to indicate the source of the reference reply; the end-of-sentence prompt for question answering field is used to indicate re-answering the product feature query statement;

[0017] Input the input statement into the large language model to obtain the corrected reply.

[0018] In one implementation of the first aspect, it further includes storing the knowledge graph triples in a storage unit.

[0019] In a second aspect, the present invention provides a system for correcting hallucinations in the output of a large language model, the system including a construction module, an update module, an acquisition module, a primary processing module, and a secondary processing module;

[0020] The construction module is used to construct knowledge graph triples of a product, and the knowledge graph triple database is used to store the product name, product features, and feature parameters of the product;

[0021] The update module is used to extract supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and update the knowledge graph triples based on the supplementary product features and the supplementary feature parameters;

[0022] The acquisition module is used to acquire a product feature query statement of the user;

[0023] The primary processing module is used to input the product feature query statement into the large language model, and the large language model obtains the feature parameters corresponding to the product feature query statement from the knowledge graph triples as a reference reply;

[0024] The secondary processing module is used to input the reference reply and the product feature query statement into the large language model to obtain a corrected reply output by the large language model.

[0025] In a third aspect, the present invention provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above-mentioned method for correcting hallucinations in the output of a large language model.

[0026] In a fourth aspect, the present invention provides an electronic device, including: a processor and a memory;

[0027] The memory is used to store a computer program;

[0028] The processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned method for correcting hallucinations in the output of a large language model.

[0029] As described above, the method, system, medium, and electronic device for correcting hallucinations in the output of a large language model of the present invention have the following

[0030] Beneficial effects:

[0031] (1) Based on knowledge graph triples and interactive Q&A, the hallucinations in the output of the large language model are effectively corrected;

[0032] (2) Optimize the content of the input large language model by recombining the input data, achieving the purpose of solving the hallucination problem in the output of the large language model and realizing more accurate and reliable language generation;

[0033] (3) Solve the inconsistency problem in data processing for enterprises, realize the accurate parsing and efficient management of information flow, promote the rapid development of natural language processing technology, and reduce costs and improve efficiency for enterprises. The present invention not only significantly improves the output quality of the large language model, but also reduces the costs related to data accuracy and integrity. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Shown is a flowchart of the method for correcting hallucinations in the output of the large language model of the present invention in one embodiment;

[0035] Figure 2 Shown is a schematic flowchart of updating the triples of the knowledge graph in the present invention in one embodiment;

[0036] Figure 3 Shown is a schematic structural diagram of the system for correcting hallucinations in the output of the large language model of the present invention in one embodiment;

[0037] Figure 4 Shown is a schematic structural diagram of the electronic device of the present invention in one embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0039] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0040] The technical solutions in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0041] As shown in FIG. 1, in one embodiment, the method for correcting hallucinations in the output of the large language model of the present invention includes steps S1 - S5.

[0042] Step S1: Construct the knowledge graph triples of the product. The knowledge graph triple database is used to store the product name, product features, and feature parameters of the product.

[0043] Specifically, in the present invention, the knowledge graph triples of the product are pre-constructed by collecting product information. Among them, taking "Entity - Attribute - Value" as the data mode for constructing the knowledge graph triples, each detailed product information including "product name", "product features", and their corresponding "feature parameters" is orderly combined into a structured triple. It should be noted that the product features and the feature parameters can be provided by the manufacturer or obtained from public document materials.

[0044] It should be noted that the present invention stores the knowledge graph triples in a storage unit. The storage unit can be a database or a storage device.

[0045] Step S2: Extract the supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and update the knowledge graph triples based on the supplementary product features and the supplementary feature parameters.

[0046] Specifically, since the product features of the product may change or increase or decrease over time. Therefore, it is necessary to update the knowledge graph triples to ensure the accuracy and effectiveness of the subsequent output results of the large language model.

[0047] Among them, the user inputs relevant text about product features and feature parameters to the large language model. The large language model extracts the relevant product features and feature parameters from the user input text, respectively as supplementary product features and supplementary feature parameters, and updates the supplementary product features and supplementary feature parameters to the knowledge graph triples.

[0048] As Figure 2 shown, when updating the knowledge graph triples, the following steps are included:

[0049] 21) Determine whether the supplementary product features are the same as the product features in the knowledge graph triples.

[0050] 22) When the supplementary product features are the same as the product features in the knowledge graph triples, update the corresponding feature parameters in the knowledge graph triples with the supplementary feature parameters.

[0051] Among them, when the supplementary product feature is the same as the product feature in the knowledge graph triple, it indicates that the supplementary product feature is an existing product feature, and only the corresponding feature parameter in the knowledge graph triple needs to be updated with the supplementary feature parameter, so as to ensure the accuracy of the feature parameter.

[0052] 23) When the supplementary product feature is different from the product feature in the knowledge graph triple, add the supplementary product feature and the supplementary feature parameter to the knowledge graph triple.

[0053] Among them, when the supplementary product feature is different from the product feature in the knowledge graph triple, it indicates that the supplementary product feature is a newly added product feature, and the supplementary product feature and the supplementary feature parameter need to be added to the knowledge graph triple.

[0054] Due to the diverse expressions in the user input text, there may be diverse expressions for product features. Therefore, it is necessary to determine whether the supplementary product feature and the product feature in the knowledge graph triple are substantially the same feature. In the present invention, any of the following methods can be used for determination:

[0055] (1) When the vector similarity between the supplementary product feature and the product feature is greater than a preset threshold, it is determined that the supplementary product feature is the same as the product feature in the knowledge graph triple.

[0056] Among them, calculate the vector similarity between the supplementary product feature and the product feature. If the vector similarity is greater than the preset threshold, it is determined that the supplementary product feature is the same as the product feature in the knowledge graph triple. If the vector similarity is less than or equal to the preset threshold, it is determined that the supplementary product feature is different from the product feature in the knowledge graph triple.

[0057] (2) Express the product feature in the knowledge graph triple using the standard product feature field, and map the supplementary product feature to the standard product feature field; when the mapped standard product feature field is the same as the product feature in the knowledge graph triple, it is determined that the supplementary product feature is the same as the product feature in the knowledge graph triple.

[0058] Among them, when constructing the triples of the knowledge graph, standard product feature fields are used to represent product features. For example, "three-dimensional parameters" are used to represent the product volume. Therefore, first, map the supplementary product features to standard product feature fields. For example, map the supplementary product feature "outer dimension" to "three-dimensional parameters". When the obtained standard product feature field is the same as the product feature in the triples of the knowledge graph, it is determined that the supplementary product feature is the same as the product feature in the triples of the knowledge graph. When the obtained standard product feature field is different from the product feature in the triples of the knowledge graph, it is determined that the supplementary product feature is different from the product feature in the triples of the knowledge graph.

[0059] Therefore, through the above judgment method, the integrity and accuracy of the product features in the triples of the knowledge graph are ensured.

[0060] Step S3: Obtain the product feature query statement of the user.

[0061] Specifically, when the user needs to query product features based on the large language model, a product feature query statement is provided. For example: How long is the shelf life of xx product?

[0062] Step S4: Input the product feature query statement into the large language model, and the large language model obtains the feature parameters corresponding to the product feature query statement from the triples of the knowledge graph as the reference reply.

[0063] Specifically, in the present invention, the large language model needs to be queried twice. In the first query, the product feature query statement is input into the large language model, and the large language model obtains the feature parameters corresponding to the product feature query statement from the triples of the knowledge graph as the reference reply. For example, for the product feature query statement "How long is the shelf life of xx product?", the obtained reference reply is: The shelf life of the product is 24 months.

[0064] Step S5: Input the reference reply and the product feature query statement into the large language model to obtain the corrected reply output by the large language model.

[0065] Specifically, to ensure the accuracy of the query results, a query is made to the large language model again based on the benchmark response. When making the second query, it is necessary to construct the input statement of the large language model based on the question guiding field, the benchmark response, and the end-of-sentence prompt question-and-answer field, where the question guiding field is used to indicate the source of the benchmark response; the end-of-sentence prompt question-and-answer field is used to indicate to re-answer the product feature query statement. In one embodiment, the question guiding field is: I learned from the product manufacturer; the benchmark response is: The product shelf life is 24 months; the end-of-sentence prompt question-and-answer field is: Please re-answer how long the shelf life of xx product is. Therefore, based on the benchmark response, the large language model makes a re-query to conform to the settings of the "question guiding field" and the "end-of-sentence prompt question-and-answer field", and then obtains a corrected response to the product feature query statement, such as "The product shelf life is 12 months."

[0066] In one embodiment, the method for correcting hallucinations in the output of the large language model further includes sorting the product features in the knowledge graph based on the query frequency of the product features. Specifically, there may be many product features in the knowledge graph triples. For different product features, users' attentions are different, which naturally leads to different query frequencies. To improve the query efficiency of product features, in the construction of the knowledge graph triples of the present invention, the product features in the knowledge graph are sorted based on the query frequency of the product features. Among them, the higher the query frequency, the higher the ranking; the lower the query frequency, the lower the ranking.

[0067] The protection scope of the method for correcting hallucinations in the output of the large language model described in the embodiments of the present invention is not limited to the execution order of the steps listed in this embodiment. Any solution achieved by adding or subtracting steps of the prior art and replacing steps according to the principles of the present invention is included in the protection scope of the present invention.

[0068] The embodiments of the present invention also provide a system for correcting hallucinations in the output of a large language model. The system for correcting hallucinations in the output of the large language model can implement the method for correcting hallucinations in the output of the large language model described in the present invention. However, the implementation devices of the system for correcting hallucinations in the output of the large language model described in the present invention include but are not limited to the structures of the system for correcting hallucinations in the output of the large language model listed in this embodiment. Any structural deformation and replacement of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0069] As Figure 3 shown, in one embodiment, the system for correcting hallucinations in the output of the large language model of the present invention includes a construction module 31, an update module 32, an acquisition module 33, a primary processing module 34, and a secondary processing module 35.

[0070] The building block 31 is used to build the knowledge graph triples of the product, and the knowledge graph triple database is used to store the product name, product features, and feature parameters of the product.

[0071] The update module 32 is connected to the building block 31 and is used to extract the supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and update the knowledge graph triples based on the supplementary product features and the supplementary feature parameters.

[0072] The acquisition module 33 is used to acquire the product feature query statement of the user.

[0073] The primary processing module 34 is connected to the update module 32 and the acquisition module 33, and is used to input the product feature query statement into the large language model, and the large language model obtains the feature parameters corresponding to the product feature query statement from the knowledge graph triples as the reference reply.

[0074] The secondary processing module 35 is connected to the primary processing module 34, and is used to input the reference reply and the product feature query statement into the large language model to obtain the corrected reply output by the large language model.

[0075] Among them, the structures and principles of the building block 31, the update module 32, the acquisition module 33, the primary processing module 34, and the secondary processing module 35 correspond one by one to the above-mentioned method for correcting hallucinations in the output of the large language model, so they will not be elaborated here.

[0076] In several embodiments provided by the present invention, it should be understood that the disclosed system, device, or method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules / units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or modules or units can be in electrical, mechanical, or other forms.

[0077] The module / unit described as a separate component may or may not be physically separated. The component shown as a module / unit may or may not be a physical module, that is, it may be located in one place, or it may be distributed across multiple network units. Some or all of the modules / units can be selected according to actual needs to achieve the objectives of the embodiments of the present invention. For example, in various embodiments of the present invention, each functional module / unit can be integrated into one processing module, or each module / unit can exist physically alone, or two or more modules / units can be integrated into one module / unit.

[0078] Those of ordinary skill in the art should further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0079] The embodiments of the present invention also provide a computer-readable storage medium. Those of ordinary skill in the art can understand that all or part of the steps in implementing the large language model output hallucination correction method in the above embodiments can be completed by instructing a processor through a program. The program can be stored in a computer-readable storage medium. The storage medium is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof. The above storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid-state disk (SSD)).

[0080] The embodiments of the present invention also provide an electronic device. The electronic device includes a processor and a memory.

[0081] The memory is used to store a computer program.

[0082] The memory includes various media that can store program codes, such as ROM, RAM, magnetic disks, USB flash drives, memory cards, or optical discs.

[0083] The processor is connected to the memory and is configured to execute the computer program stored in the memory, so that the electronic device performs the above-mentioned method for correcting hallucinations in the output of the large language model.

[0084] Preferably, the processor may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0085] As Figure 4 As shown, the electronic device of the present invention is presented in the form of a general computing device. The components of the electronic device may include, but are not limited to: one or more processors or processing units 41, a memory 42, and a bus 43 connecting different system components (including the memory 42 and the processing unit 41).

[0086] The bus 43 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus structures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0087] An electronic device typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0088] The memory 42 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 421 and / or cache memory 422. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 423 may be used for reading and writing non-removable, non-volatile magnetic media (Figure 4 not shown, commonly referred to as a "hard disk drive"). Although Figure 4 not shown in, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) may be provided. In these cases, each drive may be connected to the bus 43 through one or more data medium interfaces. The memory 42 may include at least one program product having a set (such as at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0089] A program / utility 424 having a set (at least one) of program modules 4241 may be stored, for example, in the memory 42. Such program modules 4241 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples. The program modules 4241 generally perform the functions and / or methods in the embodiments described in the present invention.

[0090] The electronic device may also communicate with one or more external devices (such as a keyboard, a pointing device, a display, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device that enables the electronic device to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication may be carried out through an input / output (I / O) interface 44. And, the electronic device may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 45. As Figure 4 shown, the network adapter 45 communicates with other modules of the electronic device through the bus 43. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0091] The above embodiments are only illustrative of the principles and effects of the present invention and are not used to limit the present invention. Any person familiar with this technology may modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for correcting hallucination output of a large language model, characterized by: The method comprises the following steps: Constructing a knowledge graph triple of a product, wherein the knowledge graph triple database is used to store product names, product features, and feature parameters of the product; Extracting supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and updating the knowledge graph triples based on the supplementary product features and the supplementary feature parameters; Get the user's product feature query statement; The product feature query statement is input into a large language model, and the large language model obtains feature parameters corresponding to the product feature query statement from the knowledge graph triples as a reference answer; Inputting the reference answer and the product feature query statement into the large language model to obtain a corrected answer output by the large language model; Updating the knowledge graph triple based on the supplementary product feature and the supplementary feature parameter comprises the following steps: Determine whether the supplementary product feature is the same as the product feature in the knowledge graph triple; When the supplementary product feature is the same as the product feature in the knowledge graph triple, the corresponding feature parameter in the knowledge graph triple is updated using the supplementary feature parameter; When the supplementary product feature is different from the product feature in the knowledge graph triple, adding the supplementary product feature and the supplementary feature parameter to the knowledge graph triple; Determining whether the supplementary product feature is the same as the product feature in the knowledge graph triple includes: when the vector similarity between the supplementary product feature and the product feature is greater than a preset threshold, determining that the supplementary product feature is the same as the product feature in the knowledge graph triple; or, using a standard product feature field to express the product feature in the knowledge graph triple, mapping the supplementary product feature to a standard product feature field; when the mapped standard product feature field is the same as the product feature in the knowledge graph triple, determining that the supplementary product feature is the same as the product feature in the knowledge graph triple; Inputting the benchmark answer and the product feature query sentence into the large language model comprises the following steps: Constructing an input sentence of the large language model based on a question introduction field, a benchmark answer, and a sentence end prompt question answer field, wherein the question introduction field is used to indicate a source of the benchmark answer; the sentence end prompt question answer field is used to indicate a re-answer to the product feature query sentence; Inputting the input sentence into the large language model to obtain the corrected response; It also includes sorting the product features in the knowledge graph based on the query frequency of the product features.

2. The method for correcting hallucination output of a large language model according to claim 1, characterized in that: It also includes storing the knowledge graph triples in a storage unit.

3. A large language model output hallucination correction system, characterized by: The system includes a construction module, an update module, an acquisition module, a primary processing module and a secondary processing module; The construction module is used to construct a knowledge graph triple of a product, and the knowledge graph triple database is used to store product names, product features and feature parameters of the product; The updating module is used to extract the supplementary product features and supplementary feature parameters of the product in the user input text based on the large language model, and update the knowledge graph triples based on the supplementary product features and the supplementary feature parameters; The acquisition module is used to obtain the user's product feature query statement; The primary processing module is used to input the product feature query statement into a large language model, and the large language model obtains feature parameters corresponding to the product feature query statement from the knowledge graph triple as a reference answer; The secondary processing module is used to input the reference answer and the product feature query statement into the large language model to obtain a corrected answer output by the large language model; Updating the knowledge graph triple based on the supplementary product feature and the supplementary feature parameter comprises the following steps: Determine whether the supplementary product feature is the same as the product feature in the knowledge graph triple; When the supplementary product feature is the same as the product feature in the knowledge graph triple, the corresponding feature parameter in the knowledge graph triple is updated using the supplementary feature parameter; When the supplementary product feature is different from the product feature in the knowledge graph triple, adding the supplementary product feature and the supplementary feature parameter to the knowledge graph triple; Determining whether the supplementary product feature is the same as the product feature in the knowledge graph triple includes: when the vector similarity between the supplementary product feature and the product feature is greater than a preset threshold, determining that the supplementary product feature is the same as the product feature in the knowledge graph triple; or, using a standard product feature field to express the product feature in the knowledge graph triple, mapping the supplementary product feature to a standard product feature field; when the mapped standard product feature field is the same as the product feature in the knowledge graph triple, determining that the supplementary product feature is the same as the product feature in the knowledge graph triple; Inputting the benchmark answer and the product feature query sentence into the large language model comprises the following steps: Constructing an input sentence of the large language model based on a question introduction field, a benchmark answer, and a sentence end prompt question answer field, wherein the question introduction field is used to indicate a source of the benchmark answer; the sentence end prompt question answer field is used to indicate a re-answer to the product feature query sentence; Inputting the input sentence into the large language model to obtain the corrected response; It also includes sorting the product features in the knowledge graph based on the query frequency of the product features.

4. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the large language model output hallucination correction method described in any one of claims 1 to 2 is implemented.

5. An electronic device, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory, so that the electronic device performs the large language model output hallucination correction method according to any one of claims 1 to 2.

Images