A method for generating large model hallucination evaluation data

Abstract

This specification discloses a method for generating hallucination evaluation data for a large model. In this method, a structured knowledge graph can be extracted from a pre-built knowledge base for different fields through entity recognition and relationship extraction technology, and then the hallucination evaluation data can be generated based on the connection relationship between the nodes in the knowledge graph and combined with a large language model to evaluate the performance of the model to be tested for non-intuitive or fuzzy information. At the same time, based on the text data corresponding to the original entity in the knowledge base, detailed reference answers are provided to form text pairs. Finally, a comprehensive set of hallucination evaluation data sets is obtained to evaluate the comprehension of the model to be tested and the accuracy of the output information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for generating large-model hallucination evaluation data. Background Art

[0002] At present, large language models (LLMs) are widely used in the field of artificial intelligence due to their powerful context perception and pattern recognition capabilities. However, due to the huge and complex training data used by large language models and the black box nature of large language models themselves, large language models may have hallucination problems, that is, large language models may output content that is inconsistent with the real world, such as fabricating facts, not distinguishing between fiction and reality, believing in rumors and legends, etc., which affects the implementation of large language models in real-world scenarios. Therefore, how to evaluate the hallucination problem of large language models is particularly important.

[0003] Usually, a large amount of evaluation data is required to evaluate the hallucination problem of a large language model, and the quality of the evaluation data is extremely important for the accuracy of the evaluation results of the hallucination problem of the large language model. Among them, the evaluation data here may contain factual errors, for example: "The prosperous situation during Wu Zetian's reign in the Tang Dynasty was called the Zhenguan Reign." Please judge whether there are factual errors in the above description, first answer yes or no, and then explain in detail. At this time, the above evaluation data can be input into the large language model, and then the hallucination problem of the large language model can be evaluated based on the output results of the large language model for the above evaluation data.

[0004] However, since the evaluation dataset relies heavily on manual writing, the production cost of the test dataset is relatively high. Summary of the invention

[0005] This specification provides a method for generating large model hallucination evaluation data to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for generating large model hallucination evaluation data, including:

[0008] Extracting original entities from a pre-built knowledge graph; the knowledge graph is obtained based on entity relationships extracted from text data contained in a pre-built knowledge base; in the knowledge graph, nodes are used to represent entities contained in the text data, and edges are used to represent semantic connections between two entities;

[0009] Extracting an entity combination from the knowledge graph according to a connection relationship between a node corresponding to the original entity and other nodes in the knowledge graph, wherein the entity combination includes the original entity and / or at least one associated entity of the original entity;

[0010] Inputting the entity combination into a preset large language model to obtain an evaluation question, and obtaining reference text data from the text data according to the entity combination;

[0011] Hallucination evaluation data is obtained based on the text pair of the evaluation question and the reference text data.

[0012] Optionally, the method further includes constructing a knowledge base, specifically including:

[0013] Obtaining each candidate text data;

[0014] Perform word segmentation processing on each candidate text data to obtain a semantic unit set of each candidate text data;

[0015] Selecting candidate text data that meets a preset screening condition from each candidate text data according to the similarity between the semantic unit set of each candidate text data and the semantic unit set of other candidate text data;

[0016] Build a knowledge base based on the selected candidate text data.

[0017] Optionally, a knowledge base is constructed based on the selected candidate text data, specifically including:

[0018] According to the classification label of each selected candidate text data, the selected candidate text data is divided into different classification candidate text data sets;

[0019] According to each classification candidate text dataset, each classification knowledge base is constructed.

[0020] Optionally, the classification label is determined based on at least one reference label of the candidate text data, and the reference label includes: a manually annotated label, an original public label, and a model predicted label; wherein the model predicted label is obtained through a pre-trained classification model.

[0021] Optionally, the classification model includes: a text encoder and a fully connected layer;

[0022] The method further includes training a classification model, specifically comprising:

[0023] Get sample text data;

[0024] Inputting the sample text data into the text encoder of the classification model to obtain the encoded feature representation of the sample text data;

[0025] Inputting the encoded feature representation into the fully connected layer to obtain a predicted classification label for the sample text data;

[0026] Determining a target loss according to a difference between the predicted classification label and the true classification label of the sample text data;

[0027] The classification model is trained with minimizing the target loss as the optimization goal to obtain a trained classification model.

[0028] Optionally, the method further includes extracting entity relationships from text data contained in a pre-built knowledge base, specifically including:

[0029] Obtaining first prompt information according to the text data and preset task description information;

[0030] Inputting the first prompt information into the large language model to determine that the text data contains at least one entity;

[0031] According to the text data, entity relationships between the entities are determined.

[0032] Optionally, determining the entity relationship between the entities according to the text data specifically includes:

[0033] Generate second prompt information according to the text data and the entity;

[0034] The second prompt information is input into the large language model to obtain at least one relationship triplet corresponding to the entity, wherein the relationship triplet includes two entities and an entity relationship between the two entities.

[0035] Optionally, extracting an entity combination from the knowledge graph according to a connection relationship between a node corresponding to the original entity in the knowledge graph and other nodes specifically includes:

[0036] Determine at least one associated entity set corresponding to the original entity according to the connection relationship between the node corresponding to the original entity in the knowledge graph and other nodes; wherein the associated entity set is used to store entities corresponding to neighbor nodes of the original entity, and the path lengths between neighbor nodes corresponding to entities stored in different associated entity sets and the nodes corresponding to the original entity are different;

[0037] Randomly extract from each associated entity set to obtain an entity combination.

[0038] Optionally, a random selection is performed from each associated entity set to obtain an entity combination, specifically including:

[0039] Randomly extract from each associated entity set to obtain an entity chain, wherein the entity chain mainly consists of the original entity and associated entities extracted from each associated entity set;

[0040] According to the type of each entity included in the entity chain, at least part of the entities are extracted from the entities included in the entity chain to obtain an entity combination.

[0041] Optionally, obtaining reference text data according to the text data specifically includes:

[0042] Inputting the evaluation questions contained in the hallucination evaluation data into the model to be tested to obtain output information;

[0043] Obtaining detection prompt information according to the output information and the reference text data included in the hallucination evaluation data;

[0044] Inputting the detection prompt information into the large language model to obtain a consistency verification result, wherein the consistency verification result is used to characterize the degree of factual consistency between the output information and the reference text data;

[0045] According to the consistency verification result, the hallucination evaluation result of the model to be tested is obtained.

[0046] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0047] In the large-model hallucination evaluation data generation method provided in the present specification, first, the original entity is extracted from a pre-constructed knowledge graph, wherein the knowledge graph here is obtained based on the entity relationship extracted from the text data contained in the pre-constructed knowledge base. In the knowledge graph, the node is used to represent the entity contained in the text data, and the edge is used to represent the semantic connection between two entities. Then, according to the connection relationship between the node corresponding to the original entity and other nodes in the knowledge graph, an entity combination is extracted from the knowledge graph, and the entity combination contains the original entity and / or at least one associated entity of the original entity. Then, the entity combination is input into a preset large language model to obtain evaluation questions, and reference text data is obtained based on the text data. Finally, the hallucination evaluation data is obtained based on the text pairs of the obtained multiple sets of evaluation questions and the reference text data.

[0048] It can be seen from the above method that based on the direct and indirect connections between entities represented in the knowledge graph, related entities that may be conceptually confused with the original entities can be obtained, and then the original entities can be replaced with related entities, or the original entities and some related entities can be directly used to form an entity combination and input into the large language model. With the help of the logical reasoning and content generation capabilities of the large language model, a series of hallucination evaluation data with difficult-to-identify factual errors and correct hallucination evaluation data can be generated to test the model to be tested, thereby reducing the cost of making the test set. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0050] Figure 1 A flowchart of a method for generating large model hallucination evaluation data provided in this specification;

[0051] Figure 2 A schematic diagram of a method for constructing a knowledge base disclosed in this specification;

[0052] Figure 3 A schematic diagram of the construction process of the knowledge graph provided in this specification;

[0053] Figure 4 A schematic diagram of the process of determining the entity combination;

[0054] Figure 5 A schematic diagram of the hallucination assessment process provided in this manual;

[0055] Figure 6 A schematic diagram of a large-model hallucination evaluation data generation device provided in this specification;

[0056] Figure 7 A method corresponding to the Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0058] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0059] Before the large language model is put into practical use, in order to improve the output quality of the large language model, it is necessary to evaluate the hallucination problem of the large language model. In this process, there are many problems with the evaluation data needed to evaluate the hallucination problem of the large language model, such as: the evaluation data may be directly translated from English evaluation data, lacking China's unique historical and cultural knowledge. The amount of evaluation data is small, and it is difficult to perform a good detection of the hallucination problem of the large language model. Since the evaluation data involves the factor of knowledge accuracy, it is highly dependent on manual writing, which leads to a high cost for the production of evaluation data. The evaluation data may be poor in quality due to the poor knowledge reserve of the writer of the manual writing, which makes the accuracy of the hallucination problem evaluation of the large language model poor, which may have an adverse impact on the development of large language model technology.

[0060] Figure 1 The flowchart of a method for generating large model hallucination evaluation data provided in this specification includes the following steps:

[0061] S101: extracting original entities from a pre-constructed knowledge graph; the knowledge graph is obtained based on entity relationships extracted from text data contained in a pre-constructed knowledge base; in the knowledge graph, nodes are used to represent entities contained in the text data, and edges are used to represent semantic connections between two entities.

[0062] In this specification, the business platform can obtain various text data from different data sources, so that a knowledge base can be built based on the obtained text data, and then the entities of the text data contained in the knowledge base and the relationships between the entities can be extracted, and a knowledge graph can be built based on the entities of the text data contained in the knowledge base and the relationships between the entities.

[0063] The above data sources can be set according to actual needs, such as public Chinese websites, forums, blogs, websites where the government releases information, etc.

[0064] The above text data can cover different fields, such as historical information, literary works, encyclopedia knowledge, etc.

[0065] In the above content, the method for the business platform to build a knowledge base based on the acquired text data is to structure the acquired text data and convert the text content into a unified format (such as JavaScript Object Notation (JSON), comma-separated values ​​(Comma-Separated Values), etc.) to facilitate subsequent operations.

[0066] In addition, since the above-mentioned text data are obtained from different data sources, there may be duplication and redundancy between different text data. Therefore, the business platform can also use the text data obtained from different data sources as candidate text data, and perform word segmentation on each candidate text data to obtain a semantic unit set for each candidate text data. Therefore, based on the similarity between the semantic unit set of each candidate text data and the semantic unit set of other candidate text data, the candidate text data that meets the preset screening conditions can be selected from the candidate text data, and then a knowledge base can be constructed based on the selected candidate text data.

[0067] Among them, for each candidate text data, the semantic unit set of the candidate text data includes each semantic unit obtained after word segmentation processing of the candidate text data. The semantic unit here can be a single character or a single word. For example: if the candidate text data is "I love programming", it can be segmented into "I / love / programming", where "I", "love" and "programming" are the semantic units.

[0068] In the above content, the method for the business platform to determine the similarity between the semantic unit set of each candidate text data and the semantic unit set of other candidate text data can be to determine the encoding value corresponding to each semantic unit contained in each semantic unit set, and then construct k independent hash functions using a random linear function family , the random linear hash function here can refer to the following formula:

[0069]

[0070] In the above formula, represents the hash value of the xth semantic unit in the semantic unit set S determined by the kth hash function, a, b are random integers (a, b are different for each hash function), where a≠0, and P is a prime number

[0071] The business platform can map each semantic unit in the semantic unit set to a fixed hash value through the kth hash function for each semantic unit set, and use the hash value of the semantic unit with the smallest hash value corresponding to each semantic unit in the semantic unit set as the minimum hash value MinHash value of the semantic unit set calculated using the kth hash function. Similarly, each hash function can obtain a minimum hash value corresponding to the semantic unit set. In other words, the semantic unit set has a total of k minimum hash values, and these k minimum hash values ​​together constitute the minimum hash value set of the semantic unit set.

[0072] Furthermore, the business platform may use the Jaccard Similarity Coefficient (Jaccard) similarity between the minimum hash value sets of each two semantic unit sets of candidate text data as the similarity between the two semantic unit sets. For details, refer to the following formula:

[0073]

[0074] In the above formula, A represents the minimum hash value set of the first semantic unit set, and B represents the minimum hash value set of the second semantic unit set. Represents the size of the intersection of two minimum hash value sets, Indicates the size of the union of two minimum hash value sets. If the Jaccard similarity is greater than the preset threshold, the candidate text data corresponding to the first semantic unit set and the candidate text data corresponding to the second semantic unit set are considered to be repeated, and the shorter candidate text data is filtered, and the longer candidate text data is used as the selected candidate text data, and the knowledge base is constructed based on the selected candidate text data.

[0075] Specifically, after filtering and removing duplicates from each text data, the business platform can build a knowledge base based on the remaining text data, and build a knowledge graph based on the text data included in the knowledge base.

[0076] Of course, in order to focus on generating hallucination evaluation data in different fields, the business platform can also build different classification knowledge bases according to the classification labels of each selected candidate text data, as follows: Figure 2 shown.

[0077] Figure 2 A schematic diagram of the method for constructing the knowledge base disclosed in this specification.

[0078] Combination Figure 2It can be seen that the business platform can divide the selected candidate text data into different classification candidate text data sets, and then for each classification candidate text data set, based on the classification candidate text data set, build a classification knowledge base corresponding to the classification candidate text data set.

[0079] Among them, the classification label of each of the above-mentioned candidate text data can be determined based on the reference classification label corresponding to each candidate text data, and the reference classification label here includes: at least one of: manually annotated labels, original public labels (that is, the classification label corresponding to each text data obtained synchronously when each text data is obtained from each data source), and model prediction labels.

[0080] The above-mentioned model prediction label can be obtained through a pre-trained classification model, where the classification model includes: a text encoder and a fully connected layer. The training method of the classification model here can be to obtain sample text data, input the sample text data into the text encoder of the classification model, obtain the encoded feature representation of the sample text data, and then input the encoded feature representation of the sample text data into the fully connected layer of the classification model to obtain the predicted classification label of the sample text data.

[0081] Furthermore, the business platform can determine the target loss based on the difference between the predicted classification label and the actual classification label of the sample text data, and train the classification model with minimizing the target loss as the optimization goal to obtain a trained classification model.

[0082] Among them, the difference between the predicted classification label and the true classification label of the sample text data is positively correlated with the target loss. For details, please refer to the following formula:

[0083]

[0084] In the above formula, is the target loss, C represents the number of all preset classification labels, represents the true classification label of the sample text data, Represents the probability distribution of the predicted class labels.

[0085] It should be noted that the business platform can perform back propagation after determining the target loss, update the model parameters of the classification model, and train the classification model. In the training process, a lower learning rate is used to avoid the situation where the adjustment of the model parameters of the classification model seriously damages the model parameters of the classification model during pre-training.

[0086] Furthermore, after constructing each classification knowledge base, the business platform can construct a knowledge graph corresponding to each classification knowledge base according to each text data contained in the classification knowledge base, as shown in the following example: Figure 3 shown.

[0087] Figure 3 A schematic diagram of the construction process of the knowledge graph provided in this specification.

[0088] Combination Figure 3 It can be seen that the business platform can obtain each first prompt information for each classification knowledge base according to each text data contained in the classification knowledge base and the preset task description information, and then each first prompt information can be input into the large language model to obtain at least one entity contained in the text data.

[0089] Among them, the above-mentioned task description information can be, for example: "Task description: Please identify and label specific types of entities from the text; output format: output a JSON list, in which the information of each entity includes: text content, entity type; input example: sample text; output example: sample JSON list; input: {text data}", where {} is a placeholder used to fill in each text data contained in the classification knowledge base.

[0090] If the above text data is: "Water Melody: When Will the Bright Moon Appear was written by Su Shi in Mizhou in the ninth year of Xining reign of Emperor Shenzong of Song Dynasty.", the first prompt information obtained can be: "Task description: Please identify and label entities of specific types from the text; Output format: Output a JSON list, where the information of each entity includes: text content, entity type; Input example: example text; Output example: example JSON list; Input: Water Melody: When Will the Bright Moon Appear was written by Su Shi in Mizhou in the ninth year of Xining reign of Emperor Shenzong of Song Dynasty.", the output obtained by the large language model according to the above first prompt information can be: "[{"entity": "Su Shi", "type": "Person",}, {" entity ": "The ninth year of Xining reign of Emperor Shenzong of Song Dynasty", "type": "Time"},{" entity ": "Creation of Water Melody: When Will the Bright Moon Appear", "type": "Event"},{" entity": "Mizhou", "type": "Place"}]".

[0091] From the above content, it can be seen that the business platform can extract each entity of different types contained in the text data through the large language model. The types of entities here can be, for example, people, time, events, places, etc.

[0092] It should be noted that when the size of the above-mentioned text data is large, the accuracy of the extraction result obtained by directly inputting the text data into the large language model for entity extraction may be poor. Based on this, the business platform can also segment the text data to obtain segmented text data, so that each segmented text data can be input into the large language model to obtain the entities contained in each segmented text data, and then the entities contained in the text data can be obtained based on the entities contained in each segmented text data.

[0093] In addition, since the business platform extracts only the entities after performing entity recognition on each text data, in order to construct the knowledge graph corresponding to the classification knowledge base, the business platform can also extract the semantic connections between the identified entities.

[0094] Specifically, the business platform may also generate second prompt information according to the text data and entities identified from the text data, and input the second prompt information into the large language model to obtain at least one relation triple.

[0095] Among them, in the above content, the relationship triplet includes two entities and the entity relationship between the two entities, in the form of (entity 1, relationship, entity 2).

[0096] For example: (Su Shi, author, Water Melody Song), the relation triplet here is used to represent the semantic connection between entity 1: Su Shi and entity 2: Water Melody Song, that is, Su Shi is the author of Water Melody Song.

[0097] Another example: (Su Shi, initiated, created "Water Melody: When Will the Bright Moon Appear"), the relationship triplet here is used to represent the semantic connection between entity 2: Create "Water Melody: When Will the Bright Moon Appear" and entity 1: Su Shi, that is, Su Shi is the initiator of the event creation of "Water Melody: When Will the Bright Moon Appear".

[0098] The second prompt information mentioned above can be, for example: "Task description: Given a sentence and the entity information therein, please identify and extract the relationship between the entities in the sentence. Output format: Output a JSON list, represented by a relationship triple of (entity 1, relationship, entity 2). Input example: sample text}, entity information: the corresponding entity information of the sample text; output example: the corresponding relationship extraction JSON list; input: text information {text data}, entity information {entities contained in the text data}".

[0099] Furthermore, after obtaining each relationship triple corresponding to each text data contained in the classification knowledge base, the business platform can construct a knowledge graph corresponding to the classification knowledge base based on each relationship triple. In the knowledge graph corresponding to the classification knowledge base, each node is used to represent each entity in each text data contained in the classification knowledge base, and each edge is used to represent the semantic connection between two entities.

[0100] In this specification, the execution entity used to implement the large model hallucination evaluation data generation method can refer to a designated device such as a server set up on the business platform, or it can refer to a terminal device such as a desktop computer, a laptop computer, etc. For the sake of ease of description, the large model hallucination evaluation data generation method provided in this specification is explained below using the server as the execution entity as an example.

[0101] S102: Extracting an entity combination from the knowledge graph based on the connection relationship between the node corresponding to the original entity and other nodes in the knowledge graph, wherein the entity combination includes the original entity and / or at least one associated entity of the original entity.

[0102] S103: Input the entity combination into a preset large language model to obtain an evaluation question, and obtain reference text data from the text data according to the entity combination.

[0103] Furthermore, the server can generate more complex evaluation questions that are easy to cause hallucinations in the model through a preset large language model based on the connection relationship between entities in the knowledge graph (such as other entities that an entity is directly or indirectly connected to). Among them, the connection relationship in the knowledge graph usually indicates that there is a certain semantic association between entities, and entities with connection relationships often contain interference information. For example: two entities with a connection relationship may share some attributes, but there is no direct relationship.

[0104] Specifically, based on a pre-constructed knowledge graph, the server can extract original entities from the knowledge graph when hallucination evaluation data is needed, and extract at least one associated entity corresponding to the original entity from the knowledge graph based on the connection relationship between the node corresponding to the original entity and other nodes in the knowledge graph, so that the entity combination required to generate hallucination evaluation data can be obtained through a preset large language model based on the extracted original entities and / or the extracted associated entities.

[0105] Among them, the method for the server to extract at least one associated entity corresponding to the original entity from the knowledge graph based on the connection relationship between the node corresponding to the original entity in the knowledge graph and other nodes can be to determine at least one associated entity set of the original entity based on the connection relationship between the node corresponding to the original entity in the knowledge graph and other nodes, and then randomly extract from each associated entity set to obtain each entity chain.

[0106] The above-mentioned associated entity set is used to store the entities corresponding to the neighbor nodes of the original entity, and the path lengths between the neighbor nodes corresponding to the entities stored in different associated entity sets and the nodes corresponding to the original entity are different. The path length here is used to represent the minimum number of edges required to connect two nodes.

[0107] For example: Node A and Node B are directly connected by an edge, then the path length between Node A and Node B is 1.

[0108] Another example: Node A is directly connected to Node B through an edge, Node B is directly connected to Node C through an edge, and there is no direct edge between Node A and Node C. Then the path length between Node A and Node C is 2, that is, two edges need to be connected in sequence to connect Node A and Node C.

[0109] In an actual application scenario, the server may obtain a preset target path length limit value, and then determine a set of associated entities of the original entity corresponding to each path length that is not greater than the target path length value.

[0110] For example: if the target path length limit value is 3, the server needs to obtain the entity corresponding to the node whose path length between the nodes corresponding to the original entity is 1, and constitute the associated entity set corresponding to the path length 1 of the original entity, obtain the entity corresponding to the node whose path length between the nodes corresponding to the original entity is 2, and constitute the associated entity set corresponding to the original entity corresponding to the path length 2, obtain the entity corresponding to the node whose path length between the nodes corresponding to the original entity is 3, and constitute the associated entity set corresponding to the path length 3 of the original entity, and so on.

[0111] In actual application scenarios, there may be a situation where node A is directly connected to node B through one edge, and node A and node B are indirectly connected to node C through two edges. In this case, the entity corresponding to node B may exist in the associated entity set corresponding to the path length 1 of node A, and at the same time, the entity corresponding to node B also exists in the associated entity set corresponding to the path length 2 of node A. Therefore, in order to avoid duplication of entities in different associated entity sets, when determining the associated entity set corresponding to the original entity, the server can also determine whether the entity that needs to be stored in the associated entity set has been stored in other associated entity sets. If not, the entity is stored in the associated entity set.

[0112] In addition, in actual application scenarios, there may be a large number of nodes connected between the nodes corresponding to the original entity. In this case, the server can also randomly select entities corresponding to at least some of the nodes from the nodes whose path length between the nodes corresponding to the original entity is the path length for each path length, as entities in the associated entity set corresponding to the node corresponding to the original entity corresponding to the path length.

[0113] Furthermore, the server may extract a related entity from each related entity set of the acquired original entity to form an entity chain with the original entity.

[0114] For example: {Su Zhe, Su Shi, created "Water Melody: When Will the Bright Moon Appear", the ninth year of Emperor Shenzong of Song's Xining reign}. Here, "Su Shi" and "Su Zhe" are entities of character type, "creating "Water Melody: When Will the Bright Moon Appear"" is an entity of event type, and "the ninth year of Emperor Shenzong of Song's Xining reign" is an entity of time type.

[0115] Then, according to the type of each entity included in the entity chain, at least some entities can be extracted from the entities included in the entity chain to obtain an entity combination, as shown in the following example: Figure 4 shown.

[0116] Figure 4 Schematic diagram of the process of determining entity combinations.

[0117] Combination Figure 4 It can be seen that, for each type of entity included in the entity chain, the server can extract an entity from the entities of the type included in the entity chain to form an entity combination together with the extracted entities of other types.

[0118] For example: {Su Che, composed "Water Melody: When Will the Bright Moon Appear" in the ninth year of Xining reign of Emperor Shenzong of Song Dynasty}.

[0119] Another example: {Su Shi, created "Water Melody: When Will the Bright Moon Appear" in the ninth year of Xining reign of Emperor Shenzong of Song Dynasty}.

[0120] From the above content, it can be seen that through the above method, the server can extract different entity combinations, which include correct entity relationships, such as: When will the bright moon appear when Su Shi created "Water Melody", and also include confusing incorrect entity relationships, such as: When will the bright moon appear when Su Zhe created "Water Melody". Therefore, these entity combinations can be used to detect whether the model can correctly understand the semantic connection between these entities and avoid incorrect reasoning when generating answers.

[0121] It should be noted that in actual application scenarios, not all entity chains are meaningful. For example, the entity chain {Li Bai, Du Fu, Bai Juyi, Li Shangyin}. Although the above entity chain has undergone three hops of expansion, it is a pile of characters and lacks entity information such as places and events. It cannot provide support for the subsequent generation of evaluation questions. It is necessary to filter out such meaningless entity chains.

[0122] Based on this, the server can also determine for each entity chain whether the entity chain meets the preset screening condition. If so, the entity chain is used as the selected entity chain. If not, the entity chain is deleted.

[0123] The above-mentioned screening rules may be set according to actual needs. For example, if the number of entity types contained in the entity chain is higher than a preset type number threshold, the entity chain may be regarded as satisfying the preset screening conditions.

[0124] For another example, if the entity chain contains entities of a specified type, the entity chain may be considered to satisfy a preset screening condition, where the entity of the specified type may be an entity of an event type.

[0125] It should be noted that the above two screening conditions can be used as screening conditions for screening entity chains at the same time. That is, if the entity chain contains entities of a specified type, and the number of entity types contained in the entity chain is higher than a preset type number threshold, the entity chain can be considered to meet the preset screening conditions.

[0126] Furthermore, after acquiring the entity combination, the server may generate third prompt information according to the entity combination and the type of each entity included in the entity combination.

[0127] For example: "Task description: Please rewrite the entity queue and its entity types into a complete and fluent sentence; Output format: Output a complete and fluent sentence; Input example: JSON list of entity combinations and their entity types; Output example: Text sentence; Input: {entity combination and the type of each entity contained in the entity combination}".

[0128] The third prompt information can then be input into the large language model to obtain output information, and hallucination evaluation data can be obtained based on the output information, reference text data and preset question templates.

[0129] For example, the third prompt message is "Task description: Please rewrite the entity queue and its entity type into a complete and fluent sentence; Output format: Output a complete and fluent sentence; Input example: JSON list of entity combination and its entity type; Output example: Text sentence; Input: [{"entity": "Zhenguanzhizhi", "type": "Event",},{"entity": "Tang Dynasty", "type": "Time"},{" entity": "Wu Zetian", "type": "Person"}]", at this time, the output information obtained may be "The prosperous situation during Wu Zetian's reign in the Tang Dynasty is called Zhenguanzhizhi.", which contains factual errors.

[0130] The above output information may be a sentence generated by the large language model according to the input entity combination and the type of each entity included in the entity combination.

[0131] The above question template can be "{output information} Please determine whether there are any factual errors in the above description. First answer yes or no, and then provide detailed explanations." At this time, the obtained hallucination evaluation data can be "{The prosperous situation during Wu Zetian's reign in the Tang Dynasty was called the Zhenguan Reign.} Please determine whether there are any factual errors in the above description. First answer yes or no, and then provide detailed explanations."

[0132] The reference text data mentioned above is text data contained in the text data and used to explain the hallucination evaluation data.

[0133] In actual application scenarios, since the amount of knowledge involved in entities such as people and time included in the entity combination is too large, if all entity knowledge in the entity combination is included in the reference text data, the reference text data will inevitably be too long, thereby introducing greater noise. Therefore, the reference text data can be the knowledge corresponding to the event type entity in the entity combination. For example, the reference text data corresponding to the evaluation question "{The prosperous situation during Wu Zetian's reign in the Tang Dynasty is called the Zhenguan Reign.} Please judge whether there is a factual error in the above description. First answer yes or no, and then explain in detail" is: The Zhenguan Reign is a politically clear, economically revived, and culturally prosperous situation that appeared during the reign of Emperor Taizong of Tang, Li Shimin, in the early Tang Dynasty. Because the reign was "Zhenguan" (627-649), it is known in history as the "Zhenguan Reign."

[0134] It should be noted that the above-mentioned reference text data can be determined based on a predetermined correspondence between entities and text data. The correspondence between entities and text data here may refer to the correspondence between entities saved in the above-mentioned entity extraction process and segmented text data contained in the text data.

[0135] Furthermore, after obtaining the hallucination evaluation data, the server can test the model to be tested according to the obtained hallucination evaluation data, as follows: Figure 5 shown.

[0136] Figure 5 Schematic diagram of the hallucination assessment process provided in this manual.

[0137] Combination Figure 5 It can be seen that the server can input the evaluation questions contained in the hallucination evaluation data into the model to be tested to obtain output information, and obtain detection prompt information based on the output information and reference text data, and then input the detection prompt information into the large language model to obtain consistency verification results, and then obtain the hallucination evaluation results of the model to be tested based on the consistency verification results.

[0138] The above consistency verification result is used to characterize the degree of factual consistency between the output information and the reference text data. The higher the degree of factual consistency here, the higher the degree of consistency between the output information and the facts represented by the reference text data.

[0139] It is worth noting that in actual application scenarios, the server can repeatedly execute the above method to obtain a hallucination evaluation data set, and perform hallucination evaluation on the model to be tested based on the hallucination evaluation data set.

[0140] From the above method, it can be seen that the server can extract structured knowledge graphs from pre-built knowledge bases for different fields through entity recognition and relationship extraction technology, and then generate hallucination evaluation data based on the connection relationship between nodes in the knowledge graph (including: direct connection, indirect connection, etc.) in combination with a large language model to test the performance of the model for non-intuitive or ambiguous information. At the same time, based on the text data corresponding to the original entity in the knowledge base, detailed reference answers are provided to form text pairs in the format of {question, reference text data}. Finally, a comprehensive set of hallucination evaluation data sets is compiled to evaluate and improve the comprehension of large language models and the accuracy of output information.

[0141] The above is one or more implementations of the large model hallucination evaluation data generation method of this specification. Based on the same idea, this specification also provides a corresponding large model hallucination evaluation data generation device, such as Figure 6 shown.

[0142] Figure 6 A schematic diagram of a large model hallucination evaluation data generation device provided in this specification includes:

[0143] The acquisition module 601 is used to extract the original entity from the pre-built knowledge graph; the knowledge graph is obtained based on the entity relationship extracted from the text data contained in the pre-built knowledge base; in the knowledge graph, the node is used to represent the entity contained in the text data, and the edge is used to represent the semantic relationship between two entities;

[0144] A selection module 602 is used to extract an entity combination from the knowledge graph according to a connection relationship between a node corresponding to the original entity and other nodes in the knowledge graph, wherein the entity combination includes the original entity and / or at least one associated entity of the original entity;

[0145] A generating module 603 is used to input the entity combination into a preset large language model to obtain an evaluation question, and obtain reference text data from the text data according to the entity combination;

[0146] The execution module 604 is used to obtain hallucination evaluation data according to the text pair of the evaluation question and the reference text data.

[0147] Optionally, the device further comprises: a construction module 605;

[0148] The construction module 605 is specifically used to obtain each candidate text data; perform word segmentation processing on each candidate text data to obtain a semantic unit set of each candidate text data; select candidate text data that meets preset screening conditions from each candidate text data based on the similarity between the semantic unit set of each candidate text data and the semantic unit sets of other candidate text data; and construct a knowledge base based on the selected candidate text data.

[0149] Optionally, the construction module 605 is specifically used to divide the selected candidate text data into different classification candidate text data sets according to the classification label of each selected candidate text data; and construct each classification knowledge base according to each classification candidate text data set.

[0150] Optionally, the classification label is determined based on at least one reference label of the candidate text data, and the reference label includes: a manually annotated label, an original public label, and a model predicted label; wherein the model predicted label is obtained through a pre-trained classification model.

[0151] Optionally, the classification model includes: a text encoder and a fully connected layer;

[0152] The apparatus further comprises: a training module 606;

[0153] The training module 606 is specifically used to obtain sample text data; input the sample text data into the text encoder of the classification model to obtain the encoded feature representation of the sample text data; input the encoded feature representation into the fully connected layer to obtain the predicted classification label of the sample text data; determine the target loss based on the difference between the predicted classification label and the true classification label of the sample text data; train the classification model with minimizing the target loss as the optimization goal to obtain a trained classification model.

[0154] Optionally, the selection module 602 is specifically used to obtain first prompt information based on the text data and preset task description information; input the first prompt information into the large language model to determine that the text data contains at least one entity; and determine the entity relationship between the entities based on the text data.

[0155] Optionally, the selection module 602 is specifically used to generate a second prompt information based on the text data and the entity; input the second prompt information into the large language model to obtain at least one relationship triple corresponding to the entity, and the relationship triplet contains two entities and the entity relationship between the two entities.

[0156] Optionally, the selection module 602 is specifically used to determine at least one associated entity set corresponding to the original entity based on the connection relationship between the node corresponding to the original entity and other nodes in the knowledge graph; wherein the associated entity set is used to store entities corresponding to neighbor nodes of the original entity, and the path lengths between neighbor nodes corresponding to entities stored in different associated entity sets and nodes corresponding to the original entity are different; and random sampling is performed from each associated entity set to obtain an entity combination.

[0157] Optionally, the selection module 602 is specifically used to randomly extract from each associated entity set to obtain an entity chain, where the entity chain is mainly composed of the original entity and associated entities extracted from each associated entity set; according to the type of each entity included in the entity chain, extract at least part of the entities from each entity included in the entity chain to obtain an entity combination.

[0158] Optionally, the device further includes: an evaluation module 607;

[0159] The evaluation module 607 is specifically used to input the evaluation questions contained in the hallucination evaluation data into the model to be tested to obtain output information; obtain detection prompt information based on the output information and the reference text data contained in the hallucination evaluation data; input the detection prompt information into the large language model to obtain a consistency verification result, and the consistency verification result is used to characterize the degree of factual consistency between the output information and the reference text data; based on the consistency verification result, obtain the hallucination evaluation result of the model to be tested.

[0160] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A method for generating large model hallucination evaluation data is provided.

[0161] This manual also provides Figure 7 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The method for generating large model hallucination evaluation data. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0162] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0163] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0164] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0165] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0166] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0171] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0172] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0173] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0174] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0176] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0177] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for generating large model hallucination evaluation data, characterized in that: include: Extracting original entities from a pre-built knowledge graph; the knowledge graph is obtained based on entity relationships extracted from text data contained in a pre-built knowledge base; in the knowledge graph, nodes are used to represent entities contained in the text data, and edges are used to represent semantic connections between two entities; Extracting an entity combination from the knowledge graph according to a connection relationship between a node corresponding to the original entity in the knowledge graph and other nodes, wherein the entity combination includes the original entity and at least one associated entity of the original entity; The entity combination is input into a preset large language model to obtain an evaluation question, and reference text data is obtained from the text data according to the entity combination; wherein the evaluation question is question text data contained in the hallucination evaluation data for evaluating the large language model, and the reference text data is text data corresponding to the evaluation question for providing a reference answer; Hallucination evaluation data is obtained based on the text pair of the evaluation question and the reference text data.

2. The method according to claim 1, characterized in that The method further includes constructing a knowledge base, specifically comprising: Obtaining each candidate text data; Perform word segmentation processing on each candidate text data to obtain a semantic unit set of each candidate text data; Selecting candidate text data that meets a preset screening condition from each candidate text data according to the similarity between the semantic unit set of each candidate text data and the semantic unit set of other candidate text data; Build a knowledge base based on the selected candidate text data.

3. The method according to claim 2, characterized in that Based on the selected candidate text data, a knowledge base is constructed, including: According to the classification label of each selected candidate text data, the selected candidate text data is divided into different classification candidate text data sets; According to each classification candidate text dataset, each classification knowledge base is constructed.

4. The method according to claim 3, characterized in that The classification label is determined based on at least one reference label of the candidate text data, and the reference label includes: a manually annotated label, an original public label, and a model prediction label; wherein the model prediction label is obtained through a pre-trained classification model.

5. The method according to claim 4, characterized in that The classification model includes: a text encoder and a fully connected layer; The method also includes training a classification model, specifically comprising: Get sample text data; Inputting the sample text data into the text encoder of the classification model to obtain the encoded feature representation of the sample text data; Inputting the encoded feature representation into the fully connected layer to obtain a predicted classification label for the sample text data; Determining a target loss according to a difference between the predicted classification label and the true classification label of the sample text data; The classification model is trained with minimizing the target loss as the optimization goal to obtain a trained classification model.

6. The method according to claim 1, characterized in that The method also includes extracting entity relationships from text data contained in a pre-built knowledge base, specifically including: Obtaining first prompt information according to the text data and preset task description information; Inputting the first prompt information into the large language model to determine that the text data contains at least one entity; According to the text data, entity relationships between the entities are determined.

7. The method according to claim 6, characterized in that Determining entity relationships between the entities according to the text data specifically includes: Generate second prompt information according to the text data and the entity; The second prompt information is input into the large language model to obtain at least one relationship triplet corresponding to the entity, wherein the relationship triplet includes two entities and an entity relationship between the two entities.

8. The method according to claim 1, characterized in that According to the connection relationship between the node corresponding to the original entity in the knowledge graph and other nodes, an entity combination is extracted from the knowledge graph, specifically including: Determine at least one associated entity set corresponding to the original entity according to the connection relationship between the node corresponding to the original entity in the knowledge graph and other nodes; wherein the associated entity set is used to store entities corresponding to neighbor nodes of the original entity, and the path lengths between neighbor nodes corresponding to entities stored in different associated entity sets and nodes corresponding to the original entity are different; Randomly extract from each associated entity set to obtain an entity combination.

9. The method according to claim 8, characterized in that Randomly extract from each associated entity set to obtain entity combinations, including: Randomly extract from each associated entity set to obtain an entity chain, wherein the entity chain mainly consists of the original entity and associated entities extracted from each associated entity set; According to the type of each entity included in the entity chain, at least part of the entities are extracted from the entities included in the entity chain to obtain an entity combination.

10. The method according to claim 1, characterized in that The method further comprises: Inputting the evaluation questions contained in the hallucination evaluation data into the model to be tested to obtain output information; Obtaining detection prompt information according to the output information and the reference text data included in the hallucination evaluation data; Inputting the detection prompt information into the large language model to obtain a consistency verification result, wherein the consistency verification result is used to characterize the degree of factual consistency between the output information and the reference text data; According to the consistency verification result, the hallucination evaluation result of the model to be tested is obtained.

Images