Instruction following data screening method and device for big language model illusion phenomenon

Abstract

The invention discloses an instruction following data screening method and device for a big language model illusion phenomenon, and relates to the field of artificial intelligence. The method comprises the steps that for any instruction, replies corresponding to the instruction are extracted to obtain a reply set; detecting semantic consistency of replies in the reply set to obtain a semantic consistency detection score in the embedded space; clustering replies in the reply set to obtain semantic clusters, and after semantic equivalence detection is performed on a target reply corresponding to the any instruction and the replies in the reply set, determining a target semantic cluster to which the target reply belongs; determining a second score based on the number of the replies generated in the target semantic cluster and the number of all the generated replies; and screening instruction following data corresponding to any instruction based on the semantic consistency detection score and the second score. The instruction following data is screened through internal state consistency detection and semantic equivalence detection, and the screened high-quality instruction following data is obtained.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to a method and device for screening instruction-following data for the hallucination phenomenon of large language models. Background Art

[0002] Alignment is a key process to ensure that large language models (LLMs) follow user instructions. A practical alignment method is to perform instruction fine-tuning on instruction data. However, state-of-the-art LLMs sometimes generate seemingly credible but actually incorrect statements, which is called hallucination. This kind of hallucination may weaken the credibility of LLMs in practical applications.

[0003] Using data containing unknown knowledge to train LLMs during the instruction fine-tuning stage may cause the model to be overconfident and promote hallucination. That is, if the knowledge in the instruction data is not learned during pre-training, the fine-tuned LLMs tend to make more mistakes when generating responses.

[0004] Therefore, there is a dilemma in instruction fine-tuning: on the one hand, LLMs need to learn how to follow user instructions at this stage, which is crucial for user interaction in practical applications; on the other hand, using high-quality data (either manually labeled or generated by other advanced LLMs) for instruction fine-tuning may introduce knowledge that is unfamiliar to LLMs, thus possibly promoting hallucination.

[0005] How to align LLMs during the instruction fine-tuning stage so that they can follow instructions and reduce hallucination? In related technologies, some studies apply reinforcement learning (RL) to teach LLMs to reduce hallucination after the instruction fine-tuning stage. For example, some studies utilize the self-evaluation ability of LLMs and use GPT-3.5-turbo to create preference data, and then align the LLMs through direct preference optimization (DPO). However, these methods require additional corpora and the API cost of closed-source LLMs, making them inefficient. Such reinforcement learning methods ignore the potential impact on the model's ability to follow instructions. Different from RL methods, an intuitive strategy is to screen out unfamiliar instruction data for instruction fine-tuning. These selected high-quality data may present more unknown knowledge to the LLM, thus further promoting hallucination because these data may contain responses with expert-level knowledge and often delve into advanced details. Therefore, the methods in related technologies cannot well solve the hallucination problem of large language models. Summary of the Invention

[0006] The main purpose of the present invention is to provide a method and device for screening instruction-following data for the hallucination phenomenon of large language models to solve the deficiencies in related technologies.

[0007] To achieve the above object, according to the first aspect of the present invention, there is provided a method for screening instruction-following data for the hallucination phenomenon of large language models, including, for any instruction, extracting the replies corresponding to the any instruction to obtain a reply set, wherein the replies corresponding to the any instruction are generated by the large language model based on the any instruction; detecting the semantic consistency of the replies in the reply set to obtain a semantic consistency detection score in the embedding space; clustering the replies in the reply set to obtain semantic clusters, and after performing semantic equivalence detection on the target reply corresponding to the any instruction and the replies in the reply set, determining the target semantic cluster to which the target reply belongs; determining a second score based on the number of replies generated in the target semantic cluster and the total number of generated replies; and screening the instruction-following data corresponding to any instruction based on the semantic consistency detection score and the second score.

[0008] Optionally, for any instruction, extracting the replies corresponding to the any instruction; detecting the semantic consistency of the replies to obtain a semantic consistency detection score in the embedding space includes: for any instruction r, extracting the replies [r′_1, r′_2,..., r′_K] corresponding to the any instruction to obtain a reply set, where K represents the quantity, and [r′_1, r′_2,..., r′_K] are generated based on the large language model; determining the final sentence embedding for the replies, where for the K generated replies and the internal state of the last token of the last layer of the large language model, determining the final sentence embedding E = [e_1, e_2,..., e_K]; and after processing the final sentence embedding into a multivariate Gaussian distribution, determining the differential entropy, where the differential entropy is used as the semantic consistency detection score to measure the semantic consistency in the continuous embedding space.

[0009] Optionally, after processing the final sentence embedding into a multivariate Gaussian distribution, determining the differential entropy includes: for the sentence embedding E, processing it into a multivariate Gaussian distribution E ∼ N(μ, ∑); based on determining the differential entropy, where det(∑) represents the determinant of the covariance matrix Σ, d is the dimension of the sentence embedding; Σ captures the relationship between K different sentence embeddings, expressed as: The differential entropy is simplified to calculate as where λ i represents the i-th eigenvalue of the covariance matrix ∑; G is a constant; q is the any instruction.

[0010] Optionally, clustering the replies in the reply set to obtain semantic clusters includes:

[0011] Clustering the replies through a pre-trained natural language inference model to obtain different semantic clusters.

[0012] Optionally, after detecting the semantic equivalence between the target response corresponding to any one instruction and the responses in the response set, it is determined that the target semantic cluster to which the target response belongs includes: reasoning about the target response and the responses in each semantic cluster based on the natural language inference model to determine whether the target response is semantically equivalent to the responses in the semantic cluster; if they are equivalent, determining the semantic cluster to which the target response belongs based on the majority voting method.

[0013] Optionally, determining the second score based on the number of responses generated in the target semantic cluster and the number of all generated responses includes: using the ratio of the number of responses generated in the target semantic cluster to the number of all generated responses as the second score.

[0014] Optionally, screening the instruction compliance data corresponding to any one instruction based on the semantic consistency detection score and the second score includes: determining the final scoring result based on the ratio of the first score to the second score, where the scoring result is used as the screening basis for the instruction compliance data corresponding to any one instruction.

[0015] Optionally, sorting the final scoring results calculated based on the response sets corresponding to any one instruction extracted in different times, and taking the top N response sets with the highest sorting as the instruction compliance data.

[0016] According to the second aspect of the present invention, there is provided an instruction compliance data screening device for the hallucination phenomenon of a large language model, including a semantic consistency detection unit, configured to extract the responses corresponding to any one instruction to obtain a response set, where the responses corresponding to any one instruction are generated by the large language model based on the any one instruction; detecting the semantic consistency of the responses in the response set to obtain the semantic consistency detection score in the embedding space; a semantic equivalence detection unit, configured to cluster the responses in the response set to obtain semantic clusters, and after detecting the semantic equivalence between the target response corresponding to any one instruction and the responses in the response set, determine the target semantic cluster to which the target response belongs; determining the second score based on the number of responses generated in the target semantic cluster and the number of all generated responses; a screening unit, configured to screen the instruction compliance data corresponding to any one instruction based on the semantic consistency detection score and the second score.

[0017] According to the third aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing the computer to execute the method according to any one of the first aspect.

[0018] The instruction following data screening method and device for the hallucination phenomenon of large language models in this embodiment, wherein the method includes, for any instruction, extracting the replies corresponding to the any instruction to obtain a reply set, wherein the replies corresponding to the any instruction are generated by the large language model based on the any instruction; detecting the semantic consistency of the replies in the reply set to obtain the semantic consistency detection score in the embedding space; clustering the replies in the reply set to obtain semantic clusters, and after detecting the semantic equivalence between the target reply corresponding to the any instruction and the replies in the reply set, determining the target semantic cluster to which the target reply belongs; determining a second score based on the number of replies generated in the target semantic cluster and the total number of generated replies; and screening the instruction following data corresponding to any instruction based on the semantic consistency detection score and the second score. By performing internal state consistency detection and semantic equivalence detection to screen the instruction following data, the screened high-quality following data can be used for the training of large language models on a large scale, so that the trained large language models significantly reduce hallucinations while maintaining strong instruction following capabilities. Brief Description of the Drawings

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a flowchart of the instruction following data screening method for the hallucination phenomenon of large language models in the embodiments of the present invention;

[0021] Figure 2 It is a schematic diagram of an electronic device in the embodiments of the present invention. Detailed Embodiments

[0022] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so as to implement the embodiments of the present invention described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and in combination with the embodiments.

[0025] According to an embodiment of the present invention, there is provided a method for screening instruction-following data for the hallucination phenomenon of large language models, as Figure 1 shown, including the following steps 101 to 103:

[0026] Step 101: For any instruction, extract the replies corresponding to the any instruction to obtain a reply set, where the replies corresponding to the any instruction are generated by a large language model based on the any instruction; detect the semantic consistency of the replies in the reply set to obtain a semantic consistency detection score in the embedding space.

[0027] In this step, the internal consistency detection ICP is used to measure the semantic consistency in the dense sentence embedding space.

[0028] As an optional implementation manner of this embodiment, for any instruction, extract the replies corresponding to the any instruction; detecting the semantic consistency of the replies to obtain a semantic consistency detection score in the embedding space includes: for any instruction r, extract the replies [r′_1, r′_2,..., r′_K] corresponding to the any instruction to obtain a reply set, where K represents the quantity, and [r′_1, r′_2,..., r′_K] is generated based on the large language model; determine the final sentence embedding for the replies, where for the K generated replies and the internal state of the last token of the last layer of the large language model, determine the final sentence embedding E = [e_1, e_2,..., e_K]; and after processing the final sentence embedding into a multivariate Gaussian distribution, determine the differential entropy, where the differential entropy is used as the semantic consistency detection score to measure the semantic consistency in the continuous embedding space.

[0029] In this alternative implementation, for a given instruction data s = (q, r), q represents the instruction and r represents the corresponding target response. For the instruction q, multiple responses [r′_1, r′_2,..., r′_K] are extracted, where K represents the number of samples. These responses are generated by a base LLM. For the generated K responses [r′_1,..., r′_K], the internal state of the last token in the last layer of each response is used as the final sentence embedding E = [e_1, e_2,..., e_K}, as it effectively captures the semantics of the sentence. Further, differential entropy (DE) is utilized to measure the semantic consistency in the continuous embedding space:

[0030] As an alternative implementation of this embodiment, after processing the final sentence embedding into a multivariate Gaussian distribution, determining the differential entropy includes: for the sentence embedding E, processing it into a multivariate Gaussian distribution E ∼ N(μ, ∑); based on determine the differential entropy, where det(∑) represents the determinant of the covariance matrix ∑, and d is the dimension of the sentence embedding; ∑ captures the relationship between K different sentence embeddings, expressed as: The differential entropy is simplified to where λ i represents the i-th eigenvalue of the covariance matrix ∑; G is a constant; and q is any one of the instructions.

[0031] In this alternative implementation, for the sentence embedding E, processing the aforementioned DE into a multivariate Gaussian distribution E ∼ N(μ, ∑), thus the differential entropy can be expressed as:

[0032]

[0033] where det(∑) represents the determinant of the covariance matrix ∑, and d is the dimension of the sentence embedding (for example, the dimension of LLaMA-3-8B is 4096). ∑ captures the relationship between K different sentence embeddings, which can be expressed as This formula is simplified to where λ i represents the i-th eigenvalue of the covariance matrix Σ, which can be easily calculated through singular value decomposition, G is a constant, and finally, the internal state consistency is measured by this formula. For a given instruction q in the data S, it is defined as F ins (q).

[0034] Step 102: Cluster the responses in the response set to obtain semantic clusters, and after performing semantic equivalence detection on the target response corresponding to any one of the instructions and the responses in the response set, determine the target semantic cluster to which the target response belongs; determine a second score based on the number of responses generated in the target semantic cluster and the total number of generated responses.

[0035] In this step, the familiarity of the LLM with the target answer is analyzed at the lexical level. First, attempts are made to cluster the generated answers that express the same content, because instruction data answers are usually in free form, and multiple answers may express the same meaning in different ways.

[0036] As an alternative implementation of this embodiment, clustering the answers in the answer set to obtain semantic clusters includes: clustering the answers through a pre-trained natural language inference model to obtain different semantic clusters.

[0037] In this alternative implementation, a trained natural language inference (NLI) model can be used to test each pair of generated answers, thereby clustering these answers. The NLI model is very suitable for identifying semantic equivalence, because if one sentence can be inferred from another sentence, then the two generated answers mean the same thing. A small NLI model is used as the clustering model, and two answers that can imply each other are regarded as semantically equivalent answers.

[0038] As an alternative implementation of this embodiment, after detecting the semantic equivalence between the target answer corresponding to any instruction and the answers in the answer set, determining the target semantic cluster to which the target answer belongs includes: based on the natural language inference model, reasoning about the target answer and the answers in each semantic cluster to determine whether the target answer is semantically equivalent to the answers in the semantic cluster;

[0039] If they are equivalent, the semantic cluster to which the target answer belongs is determined based on the majority voting method.

[0040] In this alternative implementation, multiple generated answers are clustered using the NLI model to obtain multiple different semantic clusters. Then, pairwise reasoning using the NLI model is performed again between the target answer and the generated answers to determine whether the generated answers and the target answer are semantically equivalent. The majority voting method can be used to decide which semantic cluster the target answer belongs to.

[0041] Step 103: Filter out the instruction-following data corresponding to any instruction based on the semantic consistency detection score and the second score.

[0042] In this step, according to the final data sorting from largest to smallest of the weighted scores, a subset with a specified proportion (for example: the top 10%) can be selected as the final training data.

[0043] As an alternative implementation of this embodiment, determining the second score based on the number of generated responses in the target semantic cluster and the number of all generated responses includes: using the ratio of the number of generated responses in the target semantic cluster to the number of all generated responses as the second score.

[0044] In this alternative implementation, the ratio of the number of generated responses included in the semantic cluster to the number of all generated responses is used as the final score F res (r).

[0045] As an alternative implementation of this embodiment. Screening the instruction compliance data corresponding to any instruction based on the semantic consistency detection score and the second score includes: determining the final scoring result based on the ratio of the first score to the second score, where the scoring result is used as the screening basis for the instruction compliance data corresponding to any instruction.

[0046] As an alternative implementation of this embodiment, the final scoring results calculated based on the response sets corresponding to any instruction extracted in different times are sorted, and the top N response sets with the highest ranking are used as the instruction compliance data.

[0047] In the above alternative implementation, two scores are comprehensively considered, and the weighted score is used as the final scoring result: Using this score as the final sorting criterion, the data set to be screened is sorted from largest to smallest. A specified proportion (e.g., the top 10%) of the subset can be selected as the final training data.

[0048] In this embodiment, the instruction compliance data is screened through internal state consistency detection and semantic equivalence detection. The high-quality compliance data screened by this method can be used for the training of large-scale language models, so that the trained large-scale language model significantly reduces hallucinations while maintaining strong instruction following capabilities.

[0049] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0050] According to an embodiment of the present invention, there is also provided an instruction-following data screening device for the hallucination phenomenon of large language models, including a semantic consistency detection unit, which is used for any instruction, extracting the replies corresponding to the any instruction to obtain a reply set, wherein the replies corresponding to the any instruction are generated by a large language model based on the any instruction; detecting the semantic consistency of the replies in the reply set to obtain a semantic consistency detection score in the embedding space; a semantic equivalence detection unit, which is used for clustering the replies in the reply set to obtain semantic clusters, and after performing semantic equivalence detection on the target reply corresponding to the any instruction and the replies in the reply set, determining the target semantic cluster to which the target reply belongs; determining a second score based on the number of replies generated in the target semantic cluster and the total number of generated replies; a screening unit, which is used for screening the instruction-following data corresponding to any instruction based on the semantic consistency detection score and the second score.

[0051] As an optional implementation manner of this embodiment, for any instruction, extracting the replies corresponding to the any instruction; detecting the semantic consistency of the replies to obtain a semantic consistency detection score in the embedding space includes: for any instruction r, extracting the replies [r′_1, r′_2,..., r′_K] corresponding to the any instruction to obtain a reply set, where K represents the quantity, and [r′_1, r′_2,..., r′_K] are generated based on a large language model; determining the final sentence embedding for the replies, where for the K generated replies and the internal state of the last token of the last layer of the large language model, determining the final sentence embedding E = [e_1, e_2,..., e_K]; and after processing the final sentence embedding into a multivariate Gaussian distribution, determining the differential entropy, where the differential entropy is used as the semantic consistency detection score to measure the semantic consistency in the continuous embedding space.

[0052] As an optional implementation manner of this embodiment, after processing the final sentence embedding into a multivariate Gaussian distribution, determining the differential entropy includes: for the sentence embedding E, processing it into a multivariate Gaussian distribution E ∼ N(μ, Σ); based on determining the differential entropy, where det(Σ) represents the determinant of the covariance matrix Σ, and d is the dimension of the sentence embedding; ∑ captures the relationship between K different sentence embeddings, expressed as: The differential entropy is simplified to calculate as where λ i represents the i-th eigenvalue of the covariance matrix Σ; G is a constant; and q is the any instruction.

[0053] As an optional implementation manner of this embodiment, clustering the replies in the reply set to obtain semantic clusters includes: clustering the replies through a pre-trained natural language inference model to obtain different semantic clusters.

[0054] As an optional implementation manner of this embodiment, after performing semantic equivalence detection on the target reply corresponding to any instruction and the replies in the reply set, it is determined that the target semantic cluster to which the target reply belongs includes: reasoning about the target reply and the replies in each semantic cluster based on the natural language inference model to determine whether the target reply is semantically equivalent to the replies in the semantic cluster; if they are equivalent, then determining the semantic cluster to which the target reply belongs based on the majority voting method.

[0055] As an optional implementation manner of this embodiment, determining the second score based on the number of replies generated in the target semantic cluster and the total number of generated replies includes: using the ratio of the number of replies generated in the target semantic cluster to the total number of generated replies as the second score.

[0056] As an optional implementation manner of this embodiment, screening the instruction compliance data corresponding to any instruction based on the semantic consistency detection score and the second score includes: determining the final scoring result based on the ratio of the first score to the second score, where the scoring result is used as the screening basis for the instruction compliance data of any instruction.

[0057] As an optional implementation manner of this embodiment, sorting the final scoring results calculated based on the reply sets corresponding to any instruction extracted in different times, and taking the top N reply sets with the highest ranking as the instruction compliance data.

[0058] According to an embodiment of the present invention, the present invention also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to at least one processor; wherein, the memory stores instructions executable by at least one processor, and when the instructions are executed by at least one processor, at least one processor is enabled to implement the method described in any of the above embodiments.

[0059] According to an embodiment of the present invention, the present invention also provides a readable storage medium, which stores computer instructions, and when the computer instructions are executed, the method described in any of the above embodiments can be implemented.

[0060] According to an embodiment of the present invention, the present invention also provides a computer program product, and when the computer program is executed by a processor, the method described in any of the above embodiments can be implemented.

[0061] Figure 2FIG. 0 shows a schematic block diagram of an exemplary electronic device 300 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices.

[0062] As Figure 2 shown, the electronic device 300 includes a computing unit 301 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0063] A plurality of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a magnetic disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0064] The computing unit 301 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 executes the various methods and processes described above, such as the object matching method. For example, in some embodiments, the object matching method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the methods described above can be executed.

[0065] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0066] The program code for implementing the methods of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0067] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Claims

1. A method for screening instruction-following data for the hallucination phenomenon of large language models, characterized in that, Including: For any instruction, extract the reply corresponding to the any instruction to obtain a reply set, where the reply corresponding to the any instruction is generated by a large language model based on the any instruction; Detect the semantic consistency of the replies in the reply set to obtain a semantic consistency detection score in the embedding space; Cluster the replies in the reply set to obtain semantic clusters, and after performing semantic equivalence detection on the target reply corresponding to the any instruction and the replies in the reply set, determine the target semantic cluster to which the target reply belongs; determine a second score based on the number of replies generated in the target semantic cluster and the total number of generated replies; Filter out the instruction compliance data corresponding to the any instruction based on the semantic consistency detection score and the second score.

2. The instruction following data screening method for the hallucination phenomenon of large language models according to claim 1, wherein For any instruction, extract the reply corresponding to the any instruction; Detecting the semantic consistency of the reply to obtain a semantic consistency detection score in the embedding space includes: For any instruction r, extract the replies [r′_1, r′_2,..., r′_K] corresponding to the any instruction to obtain a reply set, where K represents the quantity, and [r′_1, r′_2,..., r′_K] is generated based on a large language model; Determine the final sentence embedding for the reply, where for the K generated replies and the internal state of the last token of the last layer of the large language model, determine the final sentence embedding E = [e_1, e_2,..., e_K]; And after processing the final sentence embedding into a multivariate Gaussian distribution, determine the differential entropy, where the differential entropy is used as the semantic consistency detection score to measure the semantic consistency in the continuous embedding space.

3. The instruction-following data screening method for the hallucination phenomenon of large language models according to claim 2, wherein Determining the differential entropy after processing the final sentence embedding into a multivariate Gaussian distribution includes: For the sentence embedding E, process it into a multivariate Gaussian distribution E ∼ N(μ, ∑); Based on the differential entropy is determined, where det(∑) represents the determinant of the covariance matrix ∑, and d is the dimension of the sentence embedding; ∑ captures the relationship between K different sentence embeddings, expressed as: The differential entropy is simplified to be where λ i represents the i-th eigenvalue of the covariance matrix ∑; G is a constant; and q is any one of the instructions.

4. The instruction following data screening method for the hallucination phenomenon of large language models according to claim 1, characterized in that, Clustering the replies in the reply set to obtain semantic clusters includes: Cluster the replies through a pre-trained natural language inference model to obtain different semantic clusters.

5. The instruction-following data screening method for the hallucination phenomenon of large language models according to claim 4, characterized in that, After performing semantic equivalence detection on the target reply corresponding to the any instruction and the replies in the reply set, determining the target semantic cluster to which the target reply belongs includes: Based on the natural language inference model, perform inference on the target reply and the replies in each semantic cluster to determine whether the target reply and the replies in the semantic cluster are semantically equivalent; If they are equivalent, determine the semantic cluster to which the target reply belongs based on the majority voting method.

6. The instruction-following data screening method for the hallucination phenomenon of large language models according to claim 1, characterized in that, Determining a second score based on the number of replies generated in the target semantic cluster and the total number of generated replies includes: taking the ratio of the number of replies generated in the target semantic cluster to the total number of generated replies as the second score.

7. The instruction following data screening method for the hallucination phenomenon of large language models according to claim 1, wherein Filtering out the instruction compliance data corresponding to any instruction based on the semantic consistency detection score and the second score includes: Determine the final scoring result based on the ratio of the first score to the second score, where the scoring result is used as the screening basis for the instruction compliance data of the any instruction.

8. The instruction-following data screening method for the hallucination phenomenon of large language models according to claim 7, characterized in that, Sort the final scoring results calculated based on the reply sets corresponding to any instruction extracted in different times, and take the top N reply sets with the highest ranking as the instruction compliance data.

9. An instruction-following data screening device for the hallucination phenomenon of large language models, characterized in that, Including: A semantic consistency detection unit, configured to extract responses corresponding to any instruction to obtain a response set, wherein the responses corresponding to any instruction are generated by a large language model based on the any instruction; detect the semantic consistency of the responses in the response set to obtain a semantic consistency detection score in the embedding space; A semantic equivalence detection unit, configured to cluster the responses in the response set to obtain semantic clusters, and determine the target semantic cluster to which the target response corresponding to any instruction belongs after performing semantic equivalence detection on the target response corresponding to any instruction and the responses in the response set; determine a second score based on the number of responses generated in the target semantic cluster and the number of all generated responses; A screening unit, configured to screen the instruction compliance data corresponding to any instruction based on the semantic consistency detection score and the second score.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the method according to any one of claims 1-8.

Images