DHR-based security retrieval enhancement method and device

Abstract

The invention relates to a DHR-based security retrieval enhancement method and apparatus. The method comprises the steps of obtaining a user query; performing semantic transformation on the user query for multiple times by using a first large language model (LLM) to obtain a plurality of semantic transformation results, and performing semantic voting on the plurality of semantic transformation results to obtain a security query without inducible semantics of an attacker; and performing backdoor attack detection according to the security query to obtain a backdoor attack detection result, so that compared with the prior art, the security can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a DHR-based security retrieval enhancement method and device. Background Art

[0002] With the rapid development of science and technology, large language models (LLM) have attracted more and more attention. Large language models are deep learning models trained based on massive text data, which can generate natural language text or understand the meaning of language text. Such models can perform a variety of natural language processing (NLP) tasks, including but not limited to text classification, question answering, and dialogue.

[0003] However, a significant limitation of large language models is that their knowledge base is usually static and only covers data at the time of training, which makes the model inadequate when dealing with real-time problems or tasks beyond its knowledge. To overcome this limitation, Retrieval-Augmented Generation (RAG) has become a promising technology. RAG systems combine external knowledge bases (such as vector databases) with generative models to dynamically retrieve relevant information and generate context-rich answers when answering questions.

[0004] Although this approach enables the RAG system to combine real-time data and domain-specific knowledge to provide more accurate and context-relevant answers in areas such as recommendation systems, medical diagnosis, and information retrieval, the core of the RAG system lies in the retrieval and use of the knowledge base, so it also has the problem of low security.

[0005] For example, induced attacks exploit the overfitting characteristics of LLM's language generation process, or the defect that the retrieval module does not filter user query results strictly, and inject malicious queries by constructing special prompts, or gradually guide the model to leak sensitive information in the knowledge base through multiple rounds of interaction. Attackers use LLM's contextual memory and reasoning capabilities to embed induced contextual information in user input, causing the model to expand and expose the content of the knowledge base during the generation phase, and even infer the privacy information of the original data, causing information leakage risks. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a DHR-based security retrieval enhancement method and device, which solves the technical problem of low security in the prior art.

[0008] (2) Technical Solution

[0009] To achieve the above object, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, an embodiment of the present invention provides a method for enhancing secure retrieval based on DHR, including: obtaining a user query; using a first large language model (LLM) to perform multiple semantic transformations on the user query to obtain multiple semantic transformation results, and performing a semantic vote on the multiple semantic transformation results to obtain a secure query without the induced semantics of an attacker; performing a backdoor attack detection based on the secure query to obtain a backdoor attack detection result.

[0011] In a possible embodiment, performing a backdoor attack detection based on the secure query to obtain a backdoor attack detection result includes: determining a target answer result according to the secure query; inputting the secure query into a pre-trained poisoned model to obtain a to-be-compared answer result, where the pre-trained poisoned model is obtained by training using a training set injected with poisoned data; comparing the target answer result and the to-be-compared answer result to obtain a comparison result; if the comparison result is consistent, determining that the backdoor attack detection result is that there is a backdoor attack.

[0012] In a possible embodiment, determining a target answer result according to the secure query includes: inputting the secure query into multiple pre-trained second large language models (LLMs) to obtain multiple intermediate answer results, and performing a semantic vote on the multiple intermediate answer results to obtain a target answer result.

[0013] In a possible embodiment, the structures of each of the multiple second large language models (LLMs) are different.

[0014] In a second aspect, an embodiment of the present invention provides a device for enhancing secure retrieval based on DHR, including: an obtaining module for obtaining a user query; a semantic transformation module for using a first large language model (LLM) to perform multiple semantic transformations on the user query to obtain multiple semantic transformation results, and performing a semantic vote on the multiple semantic transformation results to obtain a secure query without the induced semantics of an attacker; a backdoor attack detection module for performing a backdoor attack detection based on the secure query to obtain a backdoor attack detection result.

[0015] In a possible embodiment, the backdoor attack detection module is specifically configured to: determine a target answer result according to the secure query; input the secure query into a pre-trained poisoned model to obtain a to-be-compared answer result, where the pre-trained poisoned model is obtained by training using a training set injected with poisoned data; compare the target answer result and the to-be-compared answer result to obtain a comparison result; if the comparison result is consistent, determining that the backdoor attack detection result is that there is a backdoor attack.

[0016] In a possible embodiment, the backdoor attack detection module is specifically configured to: input a security query into multiple pre-trained second large language models (LLMs) to obtain multiple intermediate answer results, and perform semantic voting on the multiple intermediate answer results to obtain a target answer result.

[0017] In a possible embodiment, the structures of each of the multiple second large language models (LLMs) are different from each other.

[0018] In a third aspect, an embodiment of the present application provides a storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the method described in the first aspect or any optional implementation manner of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, communication between the processor and the memory is performed through the bus. When the machine-readable instructions are executed by the processor, they execute the method described in the first aspect or any optional implementation manner of the first aspect.

[0020] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a computer, it causes the computer to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0021] (III) Beneficial Effects

[0022] The beneficial effects of the present invention are as follows:

[0023] An embodiment of the present application provides a method and apparatus for enhancing security retrieval based on DHR. By obtaining a user query, and using a first large language model (LLM) to perform multiple semantic transformations on the user query to obtain multiple semantic transformation results, and performing semantic voting on the multiple semantic transformation results to obtain a security query without the induced semantics of an attacker, and performing backdoor attack detection based on the security query to obtain a backdoor attack detection result. Compared with the prior art, it can improve security.

[0024] To make the above objects, features, and advantages to be achieved by the embodiments of the present application more obvious and understandable, the following specific embodiments are given, and in conjunction with the accompanying drawings, detailed descriptions are as follows. Description of the Drawings

[0025] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0026] Figure 1 Shows a schematic structural diagram of a DHR system in the prior art;

[0027] Figure 2 Shows a flowchart of a security retrieval enhancement method based on DHR provided by an embodiment of the present application;

[0028] Figure 3 Shows a schematic diagram of a defense mechanism against induced attacks provided by an embodiment of the present application;

[0029] Figure 4 Shows a block diagram of the structure of a security retrieval enhancement device based on DHR provided by an embodiment of the present application. Detailed implementation manners

[0030] To better explain the present invention for easy understanding, the following will describe the present invention in detail with reference to the accompanying drawings through specific implementation manners.

[0031] To solve the technical problem of low security existing in the prior art, the embodiments of the present application provide a security retrieval enhancement method and device based on DHR, which implement the security protection of DHR in the knowledge entry stage (word embedding or fine-tuning) during knowledge entry and the LLM generation stage after retrieving knowledge, can ensure that the system is protected against knowledge base backdoors and prompt injection attacks, and enhance the security of the RAG system. In addition, in terms of performance and security, security policies can be formulated based on the security requirements of the system environment, and the heterogeneous redundancy mechanisms in the above two stages can be flexibly adjusted to keep a good balance between the performance and security of the system.

[0032] To better understand the above technical solutions, the following will describe the exemplary embodiments of the present invention in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0033] To facilitate the understanding of the embodiments of the present application, the following terms related to the present application are explained as follows:

[0034] "Dynamic Heterogeneous Redundancy System": It is a dynamic active defense system that detects and blocks network attacks based on software and hardware vulnerabilities through system heterogeneity, redundancy, and dynamics. Traditional DHR is applied to active defense in scenarios where information systems inevitably have security vulnerabilities. For example, Figure 1 As shown, the defense mechanism of DHR is that user input is distributed by the input agent to each heterogeneous execution body set in the system that performs the same function. After the execution body set outputs, the voter performs multimodal voting on all outputs and then outputs.

[0035] Moreover, since the entire execution body set is composed of a heterogeneous component set combined through a dynamic selection algorithm, the probability of containing common vulnerabilities is relatively low. When an attacker launches an attack on some components, only a limited execution body set can be affected, and the outputs of these attacked execution bodies will be filtered by the voter. Therefore, active defense can be achieved.

[0036] "Induced Attack": It takes advantage of the overfitting characteristics in the language generation process of the LLM or the defect that the retrieval module has insufficiently strict filtering of user query results. By constructing special prompts to inject malicious queries, or gradually guiding the model to disclose sensitive information in the knowledge base through multiple rounds of interaction. Attackers use the context memory and reasoning capabilities of the LLM to embed induced context information in the user input, causing the model to expand and expose the content of the knowledge base during the generation stage, and even infer the privacy information of the original data, resulting in the risk of information leakage. The following conducts a formal attack surface analysis of the above attack paths:

[0037] System representation: The model receives the user query Q and retrieves the context C from the knowledge base K, and generates the response R, where R = M(Q, C) and C = Retrieve(Q, K). Here, M represents the entire computational process of the LLM receiving the input Q and C to output R, and Retrieve represents retrieval;

[0038] Attack path: The attacker constructs the query Q p , through context control C p , such that the generated result R p contains sensitive information I. Control the retrieval result C p , such that the response generated by the model contains the attacker's target information:

[0039]

[0040] "Backdoor Attack": It exploits the defect that the LLM cannot distinguish between normal data and poisoned data, or the lack of sufficient review in knowledge base updates. By inserting poisoned samples into the training data or tampering with the knowledge base and injecting specific content, it attacks the LLM and the retrieval module, so that the original training data content can be obtained through trigger words during the LLM inference and generation stage, resulting in the risk of data leakage. The following is a formal attack surface analysis of the above attack path:

[0041] System representation: The training dataset is denoted as: D = {d 1 , d 2 ,..., d n}, where each d i is input data. The model M is generated by applying the training function F to the training dataset D, denoted as M = F(D, θ), where θ is the model parameter. The knowledge base is denoted as K = {k 1 , k 2 ,..., k m}, where each k i is a data chunk;

[0042] Attack path: It is introduced from the following three stages: the training data preparation stage, the model training stage, and the model inference and generation stage.

[0043] Among them, in the training data preparation stage: The attacker inserts the poisoned samples d p =(dx p , dy p ) and k p =(kx p , ky p ) into the training dataset D, where dx p and kx p contain the trigger condition T, and dy p and ky p are the sample constructs corresponding to the output I expected by the attacker. The training data and knowledge base after injecting the poisoned data are denoted as D' = {d 1 , d 2 ,..., dx p ,..., d n} and K' = {k 1 , k 2 ,..., kx p ,..., k m};

[0044] Model training stage: During training, the following constraints are met: M(d, K') ≈ y, where y represents the output of the LLM model for the poisoned training data d and the poisoned knowledge base K' during training;

[0045] Model inference generation stage: After training, the attacker constructs (dx p , dy p ) to obtain the output I: M((dx p , dy p ), K) = I.

[0046] Please refer to Figure 2 , Figure 2 , which shows a flowchart of a security retrieval enhancement method based on DHR provided by an embodiment of the present application. As Figure 2 shown, the security retrieval enhancement method can be executed by a security retrieval enhancement device based on DHR, and the specific device of the security retrieval enhancement device can be set according to actual needs, and the embodiments of the present application are not limited thereto. For example, the security retrieval enhancement device can be a computer or a server, etc. Specifically, the security retrieval enhancement method includes:

[0047] Step S210, obtaining a user query. Among them, the user query can be used to ask questions.

[0048] Step S220, using a first large language model LLM to perform multiple semantic conversions on the user query to obtain multiple semantic conversion results, and performing semantic voting on the multiple semantic conversion results to obtain a secure query without the induced semantics of the attacker.

[0049] That is to say, in the relevant descriptions of previous induced attacks (i.e., prompt-induced, injection attacks), the key to the attacker's successful attack is that the attacker successfully constructs (Q p , C p ) to obtain the sensitive information I. Therefore, the core of the defense method is to cut off the path from (Q p , C p ) to obtain the sensitive information I.

[0050] Specifically, as Figure 3 shown, when the attacker inputs a query request (Q p , C p ), use the first large language model LLM to perform semantic conversion on (Q p , C p ), and only extract the semantic tasks that need to be completed by the system's final LLM in (Q p , C p ). At the same time, perform the extraction operation n times repeatedly to obtain multiple semantic conversion results, that is, (RQ p1 , RC p1 ), (RQ p2 , RC p2 ),..., (RQ pn , RC pn)。Among them, n is a preset positive integer. Since there is randomness in the discriminant LLM (the randomness of generation is controlled by specifying the Temprature parameter of the LLM), the generated (RQ pi ,RC pi ) is different each time it is extracted. Therefore, the above-mentioned pairs of queries Q and contexts C are heterogeneous redundant. When the number of re-extraction times n reaches a certain scale, the pairs of queries Q and contexts C no longer have the induced semantics of the attacker. Finally, in the voting stage, based on semantics, multimodal voting is performed on all the above-mentioned pairs of queries Q and contexts C, and finally a secure query (RQ p ,RC p ) without the induced semantics of the attacker is obtained.

[0051] It should be understood that the specific algorithm used for semantic voting can be set according to actual needs, and the embodiments of the present application are not limited thereto. For example, the algorithm used for semantic voting can be the binary data similarity analysis method, the word embedding semantic analysis method, the clustering analysis method, etc.

[0052] Step S230, perform backdoor attack detection based on the secure query to obtain a backdoor attack detection result.

[0053] It should be understood that the specific process of performing backdoor attack detection based on the secure query to obtain a backdoor attack detection result can be set according to actual needs, and the embodiments of the present application are not limited thereto.

[0054] Specifically, in the previous description of backdoor attacks, the key for the attacker to succeed in the attack is that the attacker successfully constructs (dx p ,dy p ) to obtain the output I. Therefore, the core of the defense method is to cut off the path from (dx p ,dy p ) to obtain the output I. Since the cost and difficulty of making heterogeneous redundancy for both the corpus D used for model fine-tuning and the knowledge base K are relatively large, the present application places the timing of cutting off the attack path in the final LLM model generation stage.

[0055] That is to say, the answer generation task is assigned to multiple second large language models LLM in the LLM model pool, and the structures of each second large language model LLM in the multiple second large language models LLM are different, and the multiple second large language models LLM are trained based on different corpora. At the same time, all second large language models LLM are associated with the knowledge base, and the outputs of all second large language models LLM are intermediate answer results M i ((dx i ,dy i), K), where \(i\in[1, m]\). Moreover, after inputting the security query into multiple pre-trained second large language models (LLMs) to obtain multiple intermediate answer results, the semantic voting is performed on all the intermediate answer results output by the models through a voter to obtain the target answer result.

[0056] In addition, the security query is also input into a pre-trained poisoning model to obtain the answer result to be compared. Among them, the pre-trained poisoning model is obtained by training with a training set injected with poisoned data in the poisoning corpus, and the output of the poisoning model is the answer result \(M\) to be compared. p ((dx p , dy p ), K), and the poisoning model is also associated with the knowledge base.

[0057] Moreover, the target answer result and the answer result to be compared are compared to obtain a comparison result. If the comparison result is inconsistent, it is determined that there is no backdoor attack in the backdoor attack detection result; if the comparison result is consistent, it is determined that there is a backdoor attack in the backdoor attack detection result. Subsequently, the attack can be identified or an alarm can be issued, so as to perform subsequent defense strategies, such as intercepting requests, re-outputting by converting semantics, etc.

[0058] Therefore, by means of the above technical solution, the present application realizes the security protection of DHR in the knowledge entry stage (word embedding or fine-tuning) during knowledge entry and the LLM generation stage after retrieving knowledge, which can ensure that the system is protected from the knowledge base backdoor and prompt injection attacks, and enhance the security of the RAG system. In addition, in terms of performance and security, a security policy can be formulated based on the security requirements of the system environment, and the heterogeneous redundancy mechanism of the above two stages can be flexibly adjusted, so that the system maintains a good balance between performance and security.

[0059] It should be understood that the above security retrieval enhancement method based on DHR is only exemplary, and those skilled in the art can make various deformations according to the above method, and the deformed solutions also belong to the protection scope of the present application.

[0060] Please refer to Figure 4 , Figure 4The structural block diagram of a security retrieval enhancement device 400 provided by an embodiment of the present application is shown. It should be understood that the security retrieval enhancement device 400 can execute each step in the above method embodiment. The specific functions of the security retrieval enhancement device 400 can refer to the descriptions above. To avoid repetition, the detailed descriptions are appropriately omitted here. The security retrieval enhancement device 400 includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the security retrieval enhancement device 400. Specifically, the security retrieval enhancement device 400 includes:

[0061] An acquisition module 410, configured to acquire a user query;

[0062] A semantic transformation module 420, configured to perform multiple semantic transformations on the user query by using a first large language model (LLM) to obtain multiple semantic transformation results, and perform semantic voting on the multiple semantic transformation results to obtain a secure query without attacker-induced semantics;

[0063] A backdoor attack detection module 430, configured to perform backdoor attack detection based on the secure query to obtain a backdoor attack detection result.

[0064] In a possible embodiment, the backdoor attack detection module 430 is specifically configured to: determine a target answer result according to the secure query; input the secure query into a pre-trained poisoned model to obtain a to-be-compared answer result, where the pre-trained poisoned model is obtained by training using a training set injected with poisoned data; compare the target answer result and the to-be-compared answer result to obtain a comparison result; if the comparison result is consistent, determine that the backdoor attack detection result is that there is a backdoor attack.

[0065] In a possible embodiment, the backdoor attack detection module 430 is specifically configured to: input the secure query into multiple pre-trained second large language models (LLMs) to obtain multiple intermediate answer results, and perform semantic voting on the multiple intermediate answer results to obtain a target answer result.

[0066] In a possible embodiment, the structures of each of the multiple second large language models (LLMs) are different.

[0067] Since the device described in the above embodiments of the present invention is the device adopted for implementing the method in the above embodiments of the present invention, based on the method described in the above embodiments of the present invention, those skilled in the art can understand the specific structure and variations of the device, and thus will not be elaborated here. Any device adopted for the method in the above embodiments of the present invention belongs to the scope of protection of the present invention.

[0068] The present application provides a storage medium, on which a computer program is stored, and when the computer program is run by a processor, it executes the method described in the embodiments.

[0069] The present application also provides a computer program product, and when the computer program product runs on a computer, it causes the computer to execute the method described in the method embodiments.

[0070] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0071] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions.

[0072] It should be noted that in the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware including several different components and by means of a suitably programmed computer. In the claims listing several devices, several of these devices can be embodied by the same piece of hardware. The use of the terms first, second, third, etc. is only for convenience of expression and does not indicate any order. These terms can be understood as part of the component name.

[0073] In addition, it should be noted that in the description of this specification, the descriptions of the terms "an embodiment", "some embodiments", "embodiments", "examples", "specific examples", or "some examples", etc. refer to the specific features, structures, materials, or characteristics described in connection with the embodiment or example being included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0074] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications after learning the basic creative concept. Therefore, the claims should be construed to cover the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0075] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention should also cover these modifications and variations.

Claims

1. A method for enhancing secure retrieval based on DHR, characterized in that, it includes: Obtain a user query; Use a first large language model (LLM) to perform multiple semantic transformations on the user query to obtain multiple semantic transformation results, and perform semantic voting on the multiple semantic transformation results to obtain a secure query without the induced semantics of the attacker; Perform backdoor attack detection based on the secure query to obtain a backdoor attack detection result.

2. The method for enhancing secure retrieval according to claim 1, characterized in that, the performing backdoor attack detection based on the secure query to obtain a backdoor attack detection result includes: Determine a target answer result according to the secure query; Input the secure query into a pre-trained poisoned model to obtain a to-be-compared answer result; wherein, the pre-trained poisoned model is obtained by training using a training set injected with poisoned data; Compare the target answer result and the to-be-compared answer result to obtain a comparison result; If the comparison result is consistent, determine that the backdoor attack detection result is that there is a backdoor attack.

3. The method for enhancing secure retrieval according to claim 2, characterized in that, the determining a target answer result according to the secure query includes: Input the secure query into multiple pre-trained second large language models (LLMs) to obtain multiple intermediate answer results, and perform semantic voting on the multiple intermediate answer results to obtain the target answer result.

4. The method for enhancing secure retrieval according to claim 3, characterized in that, the structures of each of the multiple second large language models (LLMs) are different.

5. A device for enhancing secure retrieval based on DHR, characterized in that, it includes: An acquisition module for acquiring a user query; A semantic transformation module for using a first large language model (LLM) to perform multiple semantic transformations on the user query to obtain multiple semantic transformation results, and performing semantic voting on the multiple semantic transformation results to obtain a secure query without the induced semantics of the attacker; A backdoor attack detection module for performing backdoor attack detection based on the secure query to obtain a backdoor attack detection result.

6. The device for enhancing secure retrieval according to claim 5, characterized in that, the backdoor attack detection module is specifically used for: determining a target answer result according to the secure query; inputting the secure query into a pre-trained poisoned model to obtain a to-be-compared answer result; wherein, the pre-trained poisoned model is obtained by training using a training set injected with poisoned data; comparing the target answer result and the to-be-compared answer result to obtain a comparison result; if the comparison result is consistent, determining that the backdoor attack detection result is that there is a backdoor attack.

7. The device for enhancing secure retrieval according to claim 6, characterized in that, The backdoor attack detection module is specifically configured to: input the security query into multiple pre-trained second large language models (LLMs) to obtain multiple intermediate answer results, and perform semantic voting on the multiple intermediate answer results to obtain the target answer result.

8. The security retrieval enhancement device according to claim 7, wherein, the structures of each of the multiple second large language models (LLMs) are different.

9. A storage medium, on which a computer program is stored, wherein, when the computer program is run by a processor, it executes the DHR-based security retrieval enhancement method according to any one of claims 1-4.

10. An electronic device, including a processor, a memory, and a computer program stored on the memory, wherein, the processor executes the computer program to implement the DHR-based security retrieval enhancement method according to any one of claims 1-4.

Images