Security defense method based on counterfactual semantic interference detection

Abstract

The application relates to the technical field of computer security protection, and discloses a security defense method based on counterfactual semantic interference detection, which is used for detecting and blocking semantic attacks without relying on predefined attack samples. The security defense method based on counterfactual semantic interference detection comprises the following steps: obtaining original query text input by a user; performing semantic interference on the original query text to generate a plurality of counterfactual query texts which are semantically equivalent but different in expression form; performing knowledge retrieval enhancement processing on the original query text and the plurality of counterfactual query texts to obtain an original response result and a plurality of counterfactual response results respectively; calculating semantic similarity and output entropy difference between the original response result and the plurality of counterfactual response results, and determining a comprehensive difference index based on the semantic similarity and the output entropy difference; and according to the relationship between the comprehensive difference index and a preset risk threshold, executing a corresponding security defense strategy.

Description

Technical Field

[0001] This invention relates to the field of computer security protection technology, and in particular to a security defense method based on counterfactual semantic interference detection. Background Technology

[0002] In a Search Augmentation (RAG) system, the query text entered by the user is the core instruction driving the system to perform knowledge retrieval and content generation. To prevent malicious users from using specially crafted query text to carry out attacks such as unauthorized searches, data leaks, or the generation of harmful content, deploying appropriate security defense mechanisms has become a necessary prerequisite.

[0003] In existing technologies, the various security defense mechanisms set up in RAG systems typically operate independently and as a post-processing mechanism. Taking rule-based filtering, attack sample classifier-based detection, and output content review as examples, these three common defense layers rely on keyword matching, known attack pattern identification, and post-event evaluation of the generated results, respectively. Because these mechanisms are fragmented in their technical principles and triggering timing, security operations personnel need to deploy, optimize, and monitor their status separately at different stages of the system, resulting in a complex defense system, delayed response, and high operational costs. Whether using a single detection method or simply connecting multiple methods, operations personnel must configure policies and thresholds for each defense layer individually, or perform manual correlation analysis of various alarms. This not only leads to cumbersome defense processes and poor adaptability to new and variant attacks, but also, during the independent evaluation of each defense layer, it is prone to missed or false alarms due to limited detection perspectives and inconsistent evaluation standards. This fails to efficiently and accurately meet the need for real-time and reliable blocking of covert semantic attacks in high-security scenarios. Summary of the Invention

[0004] This invention provides a security defense method based on counterfactual semantic interference detection to solve the problems in the prior art, which are caused by over-reliance on predefined attack samples and static rules, resulting in lagging detection of new and variant semantic attacks, high false negative rates, and difficulty in blocking attacks in real time before they take effect.

[0005] The first aspect of this invention provides a security defense method based on counterfactual semantic interference detection, comprising: acquiring original query text input by a user; performing semantic interference on the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression; performing knowledge retrieval enhancement processing on the original query text and the multiple counterfactual query texts to obtain original response results and multiple counterfactual response results respectively; calculating the semantic similarity and output entropy difference between the original response results and the multiple counterfactual response results, and determining a comprehensive difference index based on the semantic similarity and the output entropy difference; and executing a corresponding security defense strategy according to the relationship between the comprehensive difference index and a preset risk threshold.

[0006] In one feasible implementation, the step of semantically interfering with the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression includes: generating corresponding perturbed query text variants based on the semantic representation vector of the original query text using multiple preset semantic interference strategies; calculating the semantic similarity between each perturbed query text variant and the original query text, selecting variants with semantic similarity higher than a first preset threshold as candidate counterfactual query texts, and obtaining a candidate counterfactual query text set; calculating the pairwise semantic similarity between each variant in the candidate counterfactual query text set, and selecting a set of variants whose average pairwise similarity is lower than a second preset threshold based on the similarity matrix, thereby obtaining multiple counterfactual query texts.

[0007] In one feasible implementation, the step of performing knowledge retrieval enhancement processing on the original query text and the multiple counterfactual query texts to obtain the original response result and multiple counterfactual response results includes: under the same retrieval context and model generation parameter configuration, submitting the original query text and the multiple counterfactual query texts as a set of parallel query tasks to the RAG engine simultaneously; having the RAG engine perform vectorized retrieval on each query task to obtain the corresponding reference document fragments from the associated knowledge base; and inputting each query task and its corresponding reference document fragments into the same generative model to synchronously generate and output the original response result corresponding to the original query text and the counterfactual response result corresponding to each counterfactual query text.

[0008] In one feasible implementation, the step of calculating the semantic similarity and output entropy difference between the original response result and the plurality of counterfactual response results, and determining a comprehensive difference index based on the semantic similarity and the output entropy difference, includes: calculating the semantic similarity between the original response result and each counterfactual response result, and determining a semantic consistency index for difference analysis based on all calculated semantic similarity values; obtaining the probability distribution of the corresponding output sequence of the original response result and each counterfactual response result during the generation process, and calculating an entropy difference index for measuring response stability based on the probability distribution; and weighting and fusing the semantic consistency index and the entropy difference index based on preset weight coefficients to generate the comprehensive difference index.

[0009] In one feasible implementation, determining the semantic consistency index for difference analysis based on all calculated semantic similarity values ​​includes: determining the minimum value among all semantic similarity values ​​as a reference value representing semantic consistency; and determining the minimum value or a value obtained by transforming the minimum value as the semantic consistency index.

[0010] In one feasible implementation, the step of calculating the entropy difference index for measuring response stability based on the probability distribution includes: calculating the generation entropy of the original response result and each counterfactual response result according to the output sequence probability distribution of the result corresponding to each response; calculating the difference between the generation entropy of the original response result and the generation entropy of each counterfactual response result; and determining the entropy difference index by taking the average or maximum value of all the calculated differences.

[0011] In one feasible implementation, the step of executing a corresponding security defense strategy based on the relationship between the comprehensive difference index and a preset risk threshold includes: comparing the comprehensive difference index with at least two incremental preset risk thresholds, and determining the risk level corresponding to the current request based on the numerical range it falls into; matching and triggering a corresponding security response action from a preset policy mapping table based on the determined risk level; processing the content of the original response result or intervening in the user request based on the triggered security response action, and recording the judgment and processing results in the security audit log.

[0012] A second aspect of the present invention provides a security defense device based on counterfactual semantic interference detection, comprising: an acquisition module for acquiring original query text input by a user; a generation module for semantically interfering with the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression; a processing module for performing knowledge retrieval enhancement processing on the original query text and the multiple counterfactual query texts to obtain an original response result and multiple counterfactual response results respectively; a calculation module for calculating the semantic similarity and output entropy difference between the original response result and the multiple counterfactual response results, and determining a comprehensive difference index based on the semantic similarity and the output entropy difference; and an execution module for executing a corresponding security defense strategy according to the relationship between the comprehensive difference index and a preset risk threshold.

[0013] In one feasible implementation, the generation module is specifically used to: generate corresponding perturbed query text variants based on the semantic representation vector of the original query text, using multiple preset semantic perturbation strategies; calculate the semantic similarity between each perturbed query text variant and the original query text, select variants with semantic similarity higher than a first preset threshold as candidate counterfactual query texts, and obtain a candidate counterfactual query text set; calculate the semantic similarity between each pair of variants in the candidate counterfactual query text set, and select a group of variants whose average pairwise similarity is lower than a second preset threshold based on the similarity matrix, to obtain multiple counterfactual query texts.

[0014] In one feasible implementation, the processing module is specifically used to: submit the original query text and the multiple counterfactual query texts as a set of parallel query tasks to the RAG engine under the same retrieval context and model generation parameter configuration; have the RAG engine perform vectorized retrieval on each query task to obtain the corresponding reference document fragments from the associated knowledge base; and input each query task and its corresponding reference document fragments into the same generative model to synchronously generate and output the original response results corresponding to the original query text and the counterfactual response results corresponding to each counterfactual query text.

[0015] In one feasible implementation, the calculation module includes: a processing unit, configured to calculate the semantic similarity between the original response result and each counterfactual response result, and determine a semantic consistency index for difference analysis based on all calculated semantic similarity values; a calculation unit, configured to obtain the probability distribution of the corresponding output sequences of the original response result and each counterfactual response result during the generation process, and calculate an entropy difference index for measuring response stability based on the probability distribution; and a generation unit, configured to weight and fuse the semantic consistency index and the entropy difference index based on preset weight coefficients to generate the comprehensive difference index.

[0016] In one feasible implementation, the processing unit is specifically used to: determine the minimum value among all semantic similarity values ​​as a reference value representing semantic consistency; and determine the minimum value or the value obtained by transforming the minimum value as the semantic consistency index. In one feasible implementation, the calculation unit is specifically used to: calculate the generation entropy of the original response and each counterfactual response result according to the probability distribution of the output sequence corresponding to each response result; calculate the difference between the generation entropy of the original response result and the generation entropy of each counterfactual response; and determine the entropy difference index based on all the calculated differences by taking the average or the maximum value.

[0017] In one feasible implementation, the execution module is specifically used to: compare the comprehensive difference index with at least two incremental preset risk thresholds, and determine the risk level corresponding to the current request based on the numerical range it falls into; match and trigger the corresponding security response action from a preset policy mapping table based on the determined risk level; process the content of the original response result or intervene in the user request based on the triggered security response action, and record the judgment and processing results to the security audit log.

[0018] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the electronic device to execute the security defense method based on counterfactual semantic interference detection described above.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the security defense method based on counterfactual semantic interference detection described above.

[0020] The technical solution provided by this invention involves: acquiring the original query text input by the user; semantically interfering with the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression; performing knowledge retrieval enhancement processing on the original query text and the multiple counterfactual query texts to obtain the original response result and multiple counterfactual response results respectively; calculating the semantic similarity and output entropy difference between the original response result and the multiple counterfactual response results, and determining a comprehensive difference index based on the semantic similarity and the output entropy difference; and executing a corresponding security defense strategy based on the relationship between the comprehensive difference index and a preset risk threshold. In this embodiment of the invention, by actively semantically interfering with the original user query, multiple semantically equivalent but differently expressed counterfactual queries are generated and subjected to knowledge retrieval enhancement processing. Then, the differences in the system's response to the original query and the counterfactual query are compared and analyzed. This achieves a direct and real-time assessment of the system's robustness to semantic perturbations from the system behavior level. This method does not rely on a predefined attack sample library or keyword rules, and can effectively identify hidden malicious queries that aim to induce the model to produce inconsistent or uncertain outputs. Thus, it can accurately block and classify the attack before it triggers unauthorized retrieval or generates harmful content, significantly improving the RAG system's proactive defense capability and overall security level against zero-sample and variant semantic attacks. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of an embodiment of the security defense method based on counterfactual semantic interference detection in this invention. Figure 2 This is a schematic diagram of another embodiment of the security defense method based on counterfactual semantic interference detection in this invention. Figure 3 This is a schematic diagram of an embodiment of a security defense device based on counterfactual semantic interference detection according to the present invention; Figure 4 This is a schematic diagram of another embodiment of the security defense device based on counterfactual semantic interference detection in this invention. Figure 5This is a schematic diagram of one embodiment of the electronic device in this invention. Detailed Implementation

[0022] This invention provides a security defense method based on counterfactual semantic interference detection. By constructing semantically equivalent comparison query groups and analyzing the differences in system responses, it can identify and classify hidden semantic attacks in real time without prior samples.

[0023] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] It is understood that the executing entity of this invention can be a security defense device based on counterfactual semantic interference detection, or it can be a terminal or a server; the specific implementation is not limited here. This embodiment of the invention will be described using a server as an example.

[0025] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the security defense method based on counterfactual semantic interference detection in this invention includes: 101. Obtain the original query text input by the user; The system captures natural language queries entered by users in text form through the front-end interactive interface. These queries are transmitted to the back-end server via the network. After basic format verification and encoding processing, they are extracted into the original query text to be processed.

[0026] 102. Perform semantic interference on the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression; By utilizing large-scale pre-trained language models or specialized thesaurus, core nouns, verbs, and adjectives in the original query text are replaced with synonyms, replacing them with words that have the same meaning but different expressions while ensuring that the core semantics of the original sentence remain unchanged. Variations are generated by adjusting the sentence's grammatical structure, such as changing active voice to passive voice, declarative sentences to interrogative sentences, or reasonably splitting and merging long sentences. Furthermore, without affecting the logic, the positions of adverbs, attributives, and other modifying elements in the sentence can be rearranged, or conjunctions and transitional phrases that do not change the substantive meaning can be added or deleted. Through one or more combinations of these text conversion techniques, a series of counterfactual query texts can be systematically generated that differ significantly from the original query text in vocabulary, syntax, and word order, but are highly consistent with the user intent and semantic content.

[0027] 103. Perform knowledge retrieval enhancement processing on the original query text and multiple counterfactual query texts to obtain the original response results and multiple counterfactual response results respectively; A task queue containing all pending query texts is constructed, including one original query text entry and multiple counterfactual query text entries. Then, using a concurrent programming framework or distributed task executor, an independent processing thread or coroutine is created for each query text in the queue. These threads or coroutines synchronously call the RAG engine's query interface, each call carrying an independent query text as input. Internally, the RAG engine independently executes its complete retrieval and generation process for each concurrent request, including retrieving relevant document fragments from the knowledge base and synthesizing the final natural language response based on the retrieval results. All concurrent query requests are initiated and executed almost simultaneously, and the system continuously listens for and collects the results returned by each thread. Finally, after all parallel tasks have completed, the system integrates the output to obtain an original response result corresponding to the original query text, and a counterfactual response result corresponding to each counterfactual query text.

[0028] 104. Calculate the semantic similarity and output entropy difference between the original response result and multiple counterfactual response results, and determine the comprehensive difference index based on the semantic similarity and output entropy difference; Semantic similarity calculation is typically based on semantic vector models. For example, pre-trained models such as Sentence-BERT are used to convert each text response into a vector representation in a high-dimensional semantic space. Then, the cosine similarity between the original response vector and each counterfactual response vector is calculated, and the average or median of these similarity values ​​is taken as the overall semantic similarity index. Secondly, calculating the output entropy difference requires first performing a probabilistic analysis on each response text. A pre-trained language model is used to calculate the generation probability of each word in the response result, thereby obtaining the entropy value of the entire response result sequence to measure its uncertainty. The entropy value of the original response result and the average of the entropy values ​​of all counterfactual response results are calculated separately, and the difference between the two yields the output entropy difference. Finally, a comprehensive difference index is determined based on these two indicators. A weighted fusion approach can be used, assigning a configurable weight coefficient to both semantic similarity and output entropy difference. The normalized semantic similarity value and the output entropy difference are linearly combined according to the weights to obtain a scalarized comprehensive difference index. This index comprehensively reflects the overall deviation between the original response result and the counterfactual response result group in terms of both semantic consistency and deterministic fluctuation.

[0029] 105. Based on the relationship between the comprehensive difference index and the preset risk threshold, implement the corresponding security defense strategy.

[0030] One or more risk thresholds are pre-defined to classify different risk levels, such as safe, suspicious, and dangerous. When the calculated comprehensive difference index is below the minimum risk threshold, the system determines the current query is safe and allows the original response result to be returned directly to the user. If the comprehensive difference index is between the low-risk and high-risk thresholds, the system determines it as a suspicious query and triggers a medium-strength defense strategy, such as adding a system-generated warning statement about the potential uncertainty of the answer while returning the original response result to the user. When the comprehensive difference index exceeds the maximum risk threshold, the system determines it as a high-risk malicious query and immediately triggers the highest level of defense measures, including but not limited to completely blocking the response result of the query, not returning any content generated by RAG to the user, and instead returning a preset security prompt or error message. At the same time, the abnormal event is logged and reported to the security audit system. This tiered response mechanism can effectively defend against potential semantic attacks while ensuring a normal user experience and system availability.

[0031] In this embodiment of the invention, by actively semantically interfering with the original user query, multiple semantically equivalent but differently expressed counterfactual query texts are generated and subjected to knowledge retrieval enhancement processing. Then, the differences in the system's response to the original query and the counterfactual query are compared and analyzed. This achieves a direct and real-time assessment of the system's robustness to semantic perturbations from the system behavior level. This method does not rely on a predefined attack sample library or keyword rules, and can effectively identify hidden malicious queries that aim to induce the model to produce inconsistent or uncertain outputs. Thus, it can accurately block and classify the attack before it triggers unauthorized retrieval or generates harmful content, significantly improving the RAG system's proactive defense capability and overall security level against zero-sample and variant semantic attacks.

[0032] Please see Figure 2 Another embodiment of the security defense method based on counterfactual semantic interference detection in this invention includes: 201. Obtain the original query text input by the user; 202. Perform semantic interference on the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression; Based on the semantic representation vector of the original query text, various preset semantic interference strategies are used to generate corresponding interference-based query text variants. The semantic similarity between each interference-based query text variant and the original query text is calculated. Variants with semantic similarity higher than a first preset threshold are selected as candidate counterfactual query texts, resulting in a candidate counterfactual query text set. The semantic similarity between each pair of variants in the candidate counterfactual query text set is calculated. Based on the similarity matrix, a set of variants with an average pairwise similarity lower than a second preset threshold is selected to obtain multiple counterfactual query texts.

[0033] Based on the original query text, a high-dimensional semantic representation vector is generated using a pre-trained language model. Subsequently, various pre-defined semantic interference strategies are applied to the original query text to generate a series of perturbed query text variants. These strategies include, but are not limited to, synonym replacement, which uses a domain-knowledge-enhanced thesaurus to accurately replace key entities, verbs, and adjectives in the sentence, ensuring that the core concepts remain unchanged; sentence reconstruction, which uses dependency parsing trees to transform the sentence structure, such as switching active and passive voice, shifting adverbials, and converting clauses and phrases into equivalents; paraphrasing, which calls a finely tuned text paraphrasing model to directly restate the entire sentence; and controlled noise injection, which adds a small amount of Gaussian noise to the semantic vector at the embedding level and then reconstructs it into surface text through a decoder, thereby introducing slight and unpredictable changes in expression. To ensure that the generated variants are semantically strictly aligned with the original intent, the cosine similarity between the semantic vector of each perturbed query text variant and the original query text is calculated. Variants with similarity values ​​higher than a first preset threshold, such as 0.85, are selected to form an initial candidate set of counterfactual query texts. This step aims to filter out invalid variants that have semantically deviated due to excessive perturbation. Then, to ensure sufficient structural diversity among the finally selected counterfactual query texts and avoid excessive clustering in the semantic space, the candidate set undergoes redundancy removal. Specifically, the semantic similarity between all pairs of variants in the candidate set is calculated to form a symmetric similarity matrix. Based on this similarity matrix, an optimized selection algorithm is used, such as the maximum difference selection method based on a greedy strategy. The goal is to select a preset subset of variants, such as three to five, where the average pairwise similarity between all variant pairs within this subset is lower than a second preset threshold, such as 0.7. In this way, the final selected set of variants are not only highly semantically equivalent to the original query text, but also have significant differences in expression and syntactic structure.

[0034] 203. Perform knowledge retrieval enhancement processing on the original query text and multiple counterfactual query texts to obtain the original response results and multiple counterfactual response results respectively; Under the same retrieval context and model generation parameter configuration, the original query text and multiple counterfactual query texts are submitted to the RAG engine as a set of parallel query tasks. The RAG engine performs vectorized retrieval on each query task to obtain the corresponding reference document fragments from the associated knowledge base. Each query task and its corresponding reference document fragment are input into the same generative model to synchronously generate and output the original response results corresponding to the original query text and the counterfactual response results corresponding to each counterfactual query text.

[0035] A unified, isolated query execution environment is constructed to ensure that the original query text and all filtered counterfactual query texts are processed under completely consistent context and parameter conditions. This includes locking the RAG engine's retrieval configuration, such as using the same vectorization model to encode knowledge base documents and queries, and fixing the retrieval algorithm, such as using the exact same approximate nearest neighbor search algorithm and the number of returned Top-K documents, to ensure that the retrieval context encountered by each parallel query is highly comparable. Simultaneously, a set of model parameters is pre-set and fixed for the subsequent generation stage, including using the same generative model instance and setting completely consistent decoding parameters, such as the same temperature value, repetition penalty coefficient, and maximum generation length, thereby eliminating interference from output fluctuations caused by parameter randomness or configuration differences. Subsequently, the prepared query texts—one original query text and multiple counterfactual query texts—are encapsulated into a set of parallel query tasks. These tasks are submitted to the RAG engine's query interface almost simultaneously via an asynchronous framework. Upon receiving these tasks, the RAG engine's internal retrieval module performs vectorized retrieval operations for each independent query text, encoding each query text into a query vector and using this vector vector to perform parallel searches in the knowledge base's vector index, retrieving several reference document fragments most relevant to the specific query semantics. Then, each query task, along with its retrieved, unique reference document fragments, is packaged together to form a complete generation request. All these generation requests are fed in parallel into the same parameter-locked generative model for inference computation. The model independently performs forward propagation for each request, synchronously generating natural language responses based on the "query-reference context." Finally, the outputs of all parallel tasks are collected to accurately obtain an original response result that strictly corresponds to the original query text, and a set of counterfactual response results corresponding one-to-one with each counterfactual query text. The core of this method lies in ensuring, through strict parallelization and parameter control, that the differences between all response results can be attributed to the semantic structure changes of the input query text itself to the greatest extent, rather than external random factors in the retrieval or generation process, thus making the subsequent difference analysis conclusions more reliable and effective.

[0036] 204. For the original response result and each counterfactual response result, calculate the semantic similarity between them, and determine the semantic consistency index for difference analysis based on all the calculated semantic similarity values. The specific steps for determining the semantic consistency index for difference analysis based on all calculated semantic similarity values ​​are as follows: From all semantic similarity values, determine the minimum value as a reference value representing semantic consistency; determine the minimum value or the value obtained by transforming the minimum value as the semantic consistency index.

[0037] For each counterfactual response, a comparison with the original response is performed, calculating the cosine similarity between their corresponding semantic vectors. This value ranges from -1 to 1, with higher values ​​indicating greater semantic similarity. After all pairings are calculated, a set of semantic similarity values, containing multiple values, is obtained, specific to the current query task. Subsequently, to construct an indicator that conservatively and sensitively reflects potential semantic deviations, the system does not use the average or median, but instead determines the minimum value from this set. This design is based on a safety-first principle: if any counterfactual response deviates significantly semantically from the original response, regardless of how consistent other responses are, it indicates that the RAG engine's generation process may have been affected by changes in the input format, revealing a vulnerability in its robustness. Therefore, this minimum value is selected as the key reference value characterizing the overall semantic consistency for this query. Finally, this minimum value is determined as the semantic consistency index. Alternatively, to accommodate the standardization requirements for subsequent weighted fusion with other indices (such as entropy difference), a linear scaling transformation is applied to this minimum value, for example, mapping it from the interval [-1, 1] or [0, 1] to a unified, unitless scoring interval such as [0, 100]. Whether used directly or after transformation, this finalized semantic consistency index carries the same core information: it represents the weakest semantic connection point between the original response and all counterfactual response groups. A decrease in its value directly and strongly indicates that the generated content has experienced unstable semantic drift risks that could be maliciously exploited when faced with semantically equivalent perturbations.

[0038] 205. Obtain the probability distribution of the original response result and the corresponding output sequence of each counterfactual response result during the generation process, and calculate the entropy difference index to measure the stability of the response based on the probability distribution. The specific steps for calculating the entropy difference index for measuring response stability based on probability distribution are as follows: Calculate the generation entropy of the original response result and each counterfactual response result according to the probability distribution of the output sequence corresponding to each response result; calculate the difference between the generation entropy of the original response result and the generation entropy of each counterfactual response result; and determine the entropy difference index by taking the average or maximum value of all calculated differences.

[0039] During the generation phase of the RAG engine, as the language model generates the original response and each counterfactual response word by word, it simultaneously records the complete probability distribution of the model's vocabulary at each generation step. For each complete response, the generation entropy is calculated by first obtaining the conditional probability of the true output tokens predicted by the model at each step when generating the response sequence. Then, the natural logarithm of the conditional probabilities for all steps is taken and negative. Finally, these negative logarithmic values ​​are summed and divided by the total sequence length to obtain the average generation entropy of the response. This entropy value quantifies the average uncertainty or confidence level of the model when generating this specific content. Subsequently, the absolute difference between the generation entropy of the original response and the generation entropy of each counterfactual response is calculated. After completing all paired difference calculations, to obtain a comprehensive entropy difference index, one can choose to calculate the arithmetic mean of all differences to reflect the overall volatility level; or, to capture the stability break in the worst case, the maximum value among all differences can be selected. The higher the value of this entropy difference index, the greater the difference between the generation process of the counterfactual response and the original response at the confidence level. This suggests that the model output may be more susceptible to perturbations from subtle changes in the input form, revealing a potential risk of instability.

[0040] 206. Based on preset weighting coefficients, the semantic consistency index and the entropy difference index are weighted and fused to generate a comprehensive difference index. Both the semantic consistency index and the entropy difference index are normalized to map them to the same numerical range, such as between zero and one, to ensure dimensional consistency and comparability. The normalized semantic consistency index is typically positively correlated with security, meaning a higher value indicates greater security; while the normalized entropy difference index is positively correlated with risk, meaning a higher value indicates greater risk. Therefore, before weighted fusion, the semantic consistency index usually needs to be reverse-processed, for example, by subtracting the value from one, converting it into a semantic risk index positively correlated with risk. Then, the converted semantic risk index is multiplied by its corresponding weight coefficient, and the normalized entropy difference index is multiplied by its corresponding weight coefficient. These two weighted results are then summed, and the sum is the final comprehensive difference index. This comprehensive difference index is a single scalar value; its value directly reflects the overall risk level that the system determines the current query request may pose, with a higher value indicating higher risk. 207. Based on the relationship between the comprehensive difference index and the preset risk threshold, implement the corresponding security defense strategy.

[0041] The comprehensive difference index is compared with at least two incremental preset risk thresholds, and the risk level corresponding to the current request is determined based on the numerical range it falls into. Based on the determined risk level, the corresponding security response action is matched from the preset policy mapping table and triggered. Based on the triggered security response action, the content of the original response is processed or the user request is intervened, and the judgment and processing results are recorded in the security audit log.

[0042] At least two incrementally increasing values ​​are pre-set as risk thresholds, such as a low-risk threshold and a high-risk threshold, thus dividing the range of the comprehensive difference index into three consecutive intervals, corresponding to three risk levels: safe, suspicious, and dangerous. When the comprehensive difference index calculated in real time is lower than the low-risk threshold, the system determines the current request to be at the safe level; if the index is between the low-risk and high-risk thresholds, it is determined to be at the suspicious level; if the index is higher than the high-risk threshold, it is determined to be at the dangerous level. The system maintains a pre-set policy mapping table, which clearly defines the specific security response actions corresponding to each risk level. For example, a direct release action is matched to the safe level, an additional warning action is matched to the suspicious level, and an interception and replacement action is matched to the dangerous level. Once the risk level is determined, the system immediately triggers the corresponding action in the mapping table. These actions will substantially process the original response result or intervene in the user request, such as adding a system warning prefix to the returned original response result for the suspicious level, and completely abandoning the original response result and returning a pre-set security prompt page for the dangerous level. The entire judgment process, the final risk level, the specific actions triggered, and the original request and response information will all be structured and recorded in the system's security audit log to form a complete traceability chain.

[0043] In this embodiment of the invention, by constructing multiple semantically equivalent counterfactual inputs, parallel behavioral stress tests are performed on the original query. This enables real-time security assessment based on the consistency and stability of the system's own behavior without relying on any predefined attack samples or keyword rules. This method constructs a unified comprehensive risk index by quantifying the dual differences in model response at both the semantic and generation confidence levels. This index can identify various covert semantic attacks aimed at inducing inconsistent or uncertain model outputs, including zero-shot and variant attacks. Finally, based on the risk level, a tiered defense strategy ranging from content anonymization and human-machine questioning to complete rejection can be automatically triggered. This achieves precise, proactive, and explainable security blocking before the attack triggers harmful retrieval or generation, significantly improving the overall robustness and security controllability of the RAG system in the face of complex semantic attacks.

[0044] The above describes the security defense method based on counterfactual semantic interference detection in the embodiments of the present invention. The following describes the security defense device based on counterfactual semantic interference detection in the embodiments of the present invention. Please refer to [link / reference]. Figure 3 One embodiment of the security defense device based on counterfactual semantic interference detection in this invention includes: The acquisition module 301 is used to acquire the original query text input by the user; The generation module 302 is used to semantically interfere with the original query text and generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression. Processing module 303 is used to perform knowledge retrieval enhancement processing on the original query text and multiple counterfactual query texts to obtain the original response result and multiple counterfactual response results respectively; The calculation module 304 is used to calculate the semantic similarity and output entropy difference between the original response result and multiple counterfactual response results, and to determine a comprehensive difference index based on the semantic similarity and output entropy difference. The execution module 305 is used to execute corresponding security defense strategies based on the relationship between the comprehensive difference index and the preset risk threshold.

[0045] Beneficial effects are observed in the embodiments of the present invention.

[0046] Please see Figure 4 Another embodiment of the security defense device based on counterfactual semantic interference detection in this invention includes: The acquisition module 301 is used to acquire the original query text input by the user; The generation module 302 is used to semantically interfere with the original query text and generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression. Processing module 303 is used to input the original query text and multiple counterfactual query texts into the RAG engine in parallel to obtain the original response result and multiple counterfactual response results respectively; The calculation module 304 is used to calculate the semantic similarity and output entropy difference between the original response result and multiple counterfactual response results, and to determine a comprehensive difference index based on the semantic similarity and output entropy difference. The execution module 305 is used to execute corresponding security defense strategies based on the relationship between the comprehensive difference index and the preset risk threshold.

[0047] Optionally, the generation module 302 can be specifically used for: Based on the semantic representation vector of the original query text, various preset semantic interference strategies are used to generate corresponding interference-based query text variants. The semantic similarity between each interference-based query text variant and the original query text is calculated. Variants with semantic similarity higher than a first preset threshold are selected as candidate counterfactual query texts, resulting in a candidate counterfactual query text set. The semantic similarity between each pair of variants in the candidate counterfactual query text set is calculated. Based on the similarity matrix, a set of variants with an average pairwise similarity lower than a second preset threshold is selected to obtain multiple counterfactual query texts.

[0048] Optionally, the processing module 303 can also be specifically used for: Under the same retrieval context and model generation parameter configuration, the original query text and multiple counterfactual query texts are submitted to the RAG engine as a set of parallel query tasks. The RAG engine performs vectorized retrieval on each query task to obtain the corresponding reference document fragments from the associated knowledge base. Each query task and its corresponding reference document fragment are input into the same generative model to synchronously generate and output the original response results corresponding to the original query text and the counterfactual response results corresponding to each counterfactual query text.

[0049] Optionally, the calculation module 304 includes: The processing unit 3041 is used to calculate the semantic similarity between the original response result and each counterfactual response result, and to determine the semantic consistency index for difference analysis based on all the calculated semantic similarity values. The calculation unit 3042 is used to obtain the probability distribution of the original response result and the corresponding output sequence of each counterfactual response result during the generation process, and calculate the entropy difference index to measure the stability of the response based on the probability distribution. The generation unit 3043 is used to weight and fuse the semantic consistency index and the entropy difference index based on preset weight coefficients to generate a comprehensive difference index.

[0050] Optionally, the processing unit 3041 may be specifically used for: From all semantic similarity values, the minimum value is determined as a reference value to represent semantic consistency; the minimum value or the value obtained by transforming the minimum value is determined as the semantic consistency index. Optionally, the computing unit 3042 can be specifically used for: Based on the probability distribution of the output sequence corresponding to each response result, calculate the generation entropy of the original response result and each counterfactual response result respectively; calculate the difference between the generation entropy of the original response result and the generation entropy of each counterfactual response result; based on all the calculated differences, determine the entropy difference index by taking the average or the maximum value.

[0051] Optionally, execution module 305 can be specifically used for: The comprehensive difference index is compared with at least two incremental preset risk thresholds, and the risk level corresponding to the current request is determined based on the numerical range it falls into. Based on the determined risk level, the corresponding security response action is matched from the preset policy mapping table and triggered. Based on the triggered security response action, the content of the original response is processed or the user request is intervened, and the judgment and processing results are recorded in the security audit log.

[0052] In this embodiment of the invention, by constructing multiple semantically equivalent counterfactual inputs, parallel behavioral stress tests are performed on the original query. This enables real-time security assessment based on the consistency and stability of the system's own behavior without relying on any predefined attack samples or keyword rules. This method constructs a unified comprehensive risk index by quantifying the dual differences in model response at both the semantic and generation confidence levels. This index can identify various covert semantic attacks aimed at inducing inconsistent or uncertain model outputs, including zero-shot and variant attacks. Finally, based on the risk level, a tiered defense strategy ranging from content anonymization and human-machine questioning to complete rejection can be automatically triggered. This achieves precise, proactive, and explainable security blocking before the attack triggers harmful retrieval or generation, significantly improving the overall robustness and security controllability of the RAG system in the face of complex semantic attacks.

[0053] above Figure 3 and Figure 4 The security defense device based on counterfactual semantic interference detection in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The electronic device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0054] See Figure 5 As shown, the electronic device includes a processor 500 and a memory 501. The memory 501 stores machine-executable instructions that can be executed by the processor 500. The processor 500 executes the machine-executable instructions to implement the aforementioned security defense method based on counterfactual semantic interference detection.

[0055] Furthermore, Figure 5 The electronic device shown also includes a bus 502 and a communication interface 503. The processor 500, the communication interface 503 and the memory 501 are connected via the bus 502.

[0056] The memory 501 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 502 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0057] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the method steps of the aforementioned embodiment.

[0058] The present invention also provides an electronic device, the computer device including a memory and a processor, the memory storing computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the security defense method based on counterfactual semantic interference detection in the above embodiments.

[0059] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the security defense method based on counterfactual semantic interference detection.

[0060] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0061] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A security defense method based on counterfactual semantic interference detection, characterized in that, The security defense method based on counterfactual semantic interference detection includes: Obtain the original query text input by the user; The original query text is semantically perturbed to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression; The original query text and the multiple counterfactual query texts are subjected to knowledge retrieval enhancement processing to obtain the original response result and multiple counterfactual response results respectively; Calculate the semantic similarity and output entropy difference between the original response result and the multiple counterfactual response results, and determine a comprehensive difference index based on the semantic similarity and the output entropy difference; Based on the relationship between the comprehensive difference index and the preset risk threshold, corresponding security defense strategies are implemented.

2. The security defense method based on counterfactual semantic interference detection according to claim 1, characterized in that, The process of semantically interfering with the original query text to generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression includes: Based on the semantic representation vector of the original query text, various preset semantic interference strategies are used to generate corresponding interference-enhanced query text variants. Each variant of the disturbed query text is semantically similar to the original query text. Variants with semantic similarity higher than a first preset threshold are selected as candidate counterfactual query texts to obtain a set of candidate counterfactual query texts. Calculate the semantic similarity between each pair of variants in the candidate counterfactual query text set, and select a group of variants whose average pairwise similarity is lower than a second preset threshold based on the similarity matrix to obtain multiple counterfactual query texts.

3. The security defense method based on counterfactual semantic interference detection according to claim 1, characterized in that, The step of performing knowledge retrieval enhancement processing on the original query text and the multiple counterfactual query texts to obtain the original response result and multiple counterfactual response results includes: Under the same retrieval context and model generation parameter configuration, the original query text and the multiple counterfactual query texts are submitted to the RAG engine as a set of parallel query tasks. The RAG engine performs vectorized retrieval for each query task to obtain the corresponding reference document fragments from the associated knowledge base; Each query task and its corresponding reference document fragment are input into the same generative model, and the original response results corresponding to the original query text and the counterfactual response results corresponding to each counterfactual query text are generated and output synchronously.

4. The security defense method based on counterfactual semantic interference detection according to claim 1, characterized in that, The calculation of semantic similarity and output entropy difference between the original response result and the multiple counterfactual response results, and the determination of a comprehensive difference index based on the semantic similarity and output entropy difference, includes: For the original response result and each counterfactual response result, the semantic similarity between the two is calculated, and a semantic consistency index for difference analysis is determined based on all the calculated semantic similarity values. The probability distributions of the original response result and the corresponding output sequence of each counterfactual response result during the generation process are obtained respectively, and the entropy difference index used to measure the stability of the response is calculated based on the probability distributions. Based on preset weighting coefficients, the semantic consistency index and the entropy difference index are weighted and fused to generate a comprehensive difference index.

5. The security defense method based on counterfactual semantic interference detection according to claim 4, characterized in that, The step of determining the semantic consistency index for difference analysis based on all calculated semantic similarity values ​​includes: From all semantic similarity values, the minimum value is determined as a reference value representing semantic consistency; The minimum value or the value obtained by transforming the minimum value is determined as the semantic consistency index.

6. The security defense method based on counterfactual semantic interference detection according to claim 4, characterized in that, The calculation of the entropy difference index for measuring response stability based on the probability distribution includes: Based on the probability distribution of the output sequence corresponding to each response result, calculate the generation entropy of the original response result and each counterfactual response result respectively; Calculate the difference between the generation entropy of the original response result and the generation entropy of each counterfactual response result; Based on all the calculated differences, the entropy difference index is determined by taking the average or the maximum value.

7. The security defense method based on counterfactual semantic interference detection according to any one of claims 1-6, characterized in that, The step of executing corresponding security defense strategies based on the relationship between the comprehensive difference index and the preset risk threshold includes: The comprehensive difference index is compared with at least two incremental preset risk thresholds, and the risk level corresponding to the current request is determined based on the numerical range in which it falls. Based on the determined risk level, the corresponding security response action is matched and triggered from the preset policy mapping table; Based on the triggered security response action, the content of the original response result is processed or the user request is intervened, and the judgment and processing result is recorded in the security audit log.

8. A security defense device based on counterfactual semantic interference detection, characterized in that, The security defense device based on counterfactual semantic interference detection includes: The acquisition module is used to acquire the original query text input by the user; The generation module is used to semantically interfere with the original query text and generate multiple counterfactual query texts that are semantically equivalent but have different forms of expression. The processing module is used to perform knowledge retrieval enhancement processing on the original query text and the multiple counterfactual query texts to obtain the original response result and multiple counterfactual response results respectively; The calculation module is used to calculate the semantic similarity and output entropy difference between the original response result and the multiple counterfactual response results, and to determine a comprehensive difference index based on the semantic similarity and the output entropy difference; The execution module is used to execute corresponding security defense strategies based on the relationship between the comprehensive difference index and the preset risk threshold.

9. An electronic device, characterized in that, The electronic device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the electronic device to execute the security defense method based on counterfactual semantic interference detection as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the security defense method based on counterfactual semantic interference detection as described in any one of claims 1-7.

Images