Text steganography method based on retrieval enhanced large language model

Abstract

The invention discloses a text steganography method based on a retrieval enhanced large language model. The text steganography method comprises the steps that a sender and a receiver set a carrier problem set in advance; the sender generates a target answer corresponding to the secret information through a preset embedding sequence and a target answer mapping table according to the secret information to be transmitted; generating a steganographic text containing the target answer based on the target answer, and injecting the steganographic text into a corpus of a retrieval enhancement generation system; and the receiver generates a reply containing the target answer by inputting the carrier question, and recovers the secret information through reverse mapping according to the reply containing the target answer. According to the method, hidden information transmission in the retrieval enhancement generation process is realized through corpus injection.

Description

Technical Field

[0001] This invention relates to the field of computer security steganography technology based on artificial intelligence, and in particular to a text steganography method based on a retrieval-enhanced large language model. Background Technology

[0002] Steganography, as a classic method of covert communication, relies on common and readily available communication media as cover to embed secret information within seemingly ordinary content, thereby achieving secure information transmission. With the rapid development of digital media, steganographic carriers have expanded from traditional text and images to multimedia data, including naturally collected images and audio, as well as content generated by intelligent algorithms. In this process, steganography needs to strike a balance between concealment, embedding capacity, and robustness: concealment requires the carrier to remain visually or aurally natural, and its statistical characteristics must be able to resist detection by steganalysis algorithms; robustness, on the other hand, is not about preventing active attacks, but rather about dealing with the unavoidable lossy processing such as recompression and format conversion in real-world channels like social media, ensuring the survival of steganographic information during transmission. However, existing steganography algorithms still have limitations in simultaneously achieving high concealment, large-capacity embedding, and broad applicability.

[0003] In recent years, the rapid development of artificial intelligence, especially Large Language Models (LLMs), has brought revolutionary changes to text generation and processing, and has also provided new ideas for steganography. Some studies have begun to explore the direct use of LLMs to generate text containing secret information. Although these methods can generate seemingly natural text, their core mechanisms mostly rely on manipulating the generation probability or sampling process within the model to encode information. The generated text may still have subtle differences in statistical distribution compared to natural human writing, posing a risk of being detected by targeted analysis models. Furthermore, these methods often lack robust design for the transmission and processing of the carrier text in real-world application scenarios (such as interaction through question-and-answer systems).

[0004] On the other hand, the Retrieval-Augmented Generation (RAG) framework, as an important architecture for improving the accuracy and timeliness of knowledge in large language models, has been widely used. This framework retrieves relevant text fragments from external knowledge bases and provides them as context to large language models, thereby guiding the models to generate more reliable answers. However, existing research largely focuses on the security vulnerabilities of RAG systems, namely, attackers could maliciously inject misleading corpora to "poison" the knowledge base, thereby affecting or manipulating the model's output. These studies only consider "corpus injection" as a security threat that needs to be defended against.

[0005] In summary, existing text steganography techniques still have limitations in adapting to the next generation of intelligent interactive environments and achieving highly covert, high-capacity communication. Furthermore, current understanding of the "corpus injection" characteristic of RAG systems is limited to its negative security impact. Currently, there is no research or solution that actively and systematically transforms the core operating mechanism of RAG systems—especially the dynamic injectability of its corpus and the retrieval-generation coupling characteristics—into a controllable, reliable, and highly covert communication channel. How to utilize this architectural feature to design a novel steganography method that can seamlessly integrate into everyday question-and-answer interactions while effectively resisting statistical analysis detection and unintentional channel interference has become an important direction worthy of exploration. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention proposes a text steganography method based on a retrieval-enhanced large language model, ensuring the accuracy of extracted information.

[0007] To achieve the above objectives, this invention provides a text steganography method based on a retrieval-enhanced large language model, comprising: The sender and receiver pre-set a set of carrier questions; The sender generates a target answer corresponding to the secret information based on the secret information to be transmitted, using a preset embedding sequence and target answer mapping table; Based on the target answer, generate steganographic text containing the target answer, and inject the steganographic text into the corpus of the retrieval enhancement generation system; The receiver inputs the carrier question, generates a response containing the target answer, and recovers the secret information through reverse mapping based on the response containing the target answer.

[0008] Preferably, the embedding sequence and target answer mapping table constructs a target answer set for each carrier question, and each target answer corresponds to a unique embedding sequence, realizing a bidirectional one-to-one mapping between secret information and target answers; The embedded sequence and the target answer mapping table are represented as follows: ; In the formula, For embedding sequences and mapping tables to target answers; The length of the secret message that can be sent in carrier-based communication; Let be the set of target answers, where each target answer ( ).

[0009] Preferably, each answer in the target answer set must satisfy semantic rationality, and any two answers must be distinguishable in terms of semantics or expression.

[0010] Preferably, generating steganographic text containing the target answer based on the target answer includes: Generate initial steganographic text containing the target answer related to the carrier question using a large language model; The initial steganographic text is optimized to obtain steganographic text containing the target answer; The optimization process includes: The initial stegtext is iteratively optimized by minimizing a joint loss function, which is constructed by combining retrieval priority loss, answer similarity loss, text quality loss, and steganalysis loss. The retrieval priority loss is used to obtain a higher retrieval ranking by increasing the semantic similarity between the steganographic text and the carrier problem; The answer similarity loss is used to guide the output answer by improving the semantic similarity between the response generated by the large language model based on the steganographic text and the target answer.

[0011] Preferably, the text quality loss is used to control the change in natural language quality of the optimized stegtext relative to the initial stegtext; The steganalysis loss is optimized based on the steganalysis model. By reducing the probability that the steg text is identified as a steganalytic carrier, it ensures that the features of the steg text and the normal text converge.

[0012] Preferably, calculating the answer similarity loss includes: Candidate text fragments are extracted from the responses generated by the large language model, and the semantic vector similarity between each candidate fragment and the target answer is obtained. The highest semantic similarity is selected as the matching score, and the answer similarity loss is calculated based on the matching score.

[0013] Preferably, the optimization process is performed using a text adversarial optimization algorithm based on HotFlip, which obtains steganographic text that meets the optimization objective by iteratively replacing text lexical units.

[0014] Preferably, the receiver generates a response containing the target answer by inputting the carrier question, and recovers the secret information through reverse mapping based on the response containing the target answer, including: By sequentially inputting the carrier question into the retrieval enhancement generation system, the corresponding large language model output text is obtained; For each output text, a two-layer answer extraction process is performed: first, an attempt is made to directly identify the preset target answer from the output text through substring matching; if the matching fails, the candidate segment that is semantically closest to the target answer is extracted from the output text as the extracted answer through semantic similarity calculation. The extracted answer is matched with all candidate target answers in a pre-stored mapping table to determine the final target answer; Based on the mapping table, the final target answer is reverse-mapped into the corresponding embedding sequence; By sequentially concatenating the embedded sequences corresponding to all carrier problems to form a complete ciphertext sequence and then decrypting it, the original secret information can be obtained.

[0015] Preferably, the candidate segment that is semantically closest to the target answer is extracted from the output text through semantic similarity calculation as the extracted answer, including: The output text is segmented and entity-recognized using natural language processing tools to obtain several candidate text fragments. Each candidate text fragment and the target answer are encoded into a semantic vector; By calculating cosine similarity, candidate text segments that are closest to the semantic vector of the target answer are selected as the extracted answer.

[0016] Preferably, determining the final target answer includes: If the extracted answer is obtained through substring matching, then the corresponding extracted answer is directly used as the final target answer; If the extracted answer is obtained through semantic similarity calculation, then the candidate target answer with the highest semantic similarity is determined as the final target answer by calculating the semantic similarity between the corresponding extracted answer and each candidate target answer in the mapping table.

[0017] Compared with the prior art, the present invention has the following advantages and technical effects: (1) This invention realizes a new type of steganography based on RAG and large model by constructing a mapping table between secret sequence and target answer and injecting corpus into the carrier problem. It is suitable for the current development and application environment of LLM.

[0018] (2) This invention proposes a steganography corpus injection algorithm. The optimization of this algorithm includes two levels. First, it defines the retrieval priority distortion by calling LLM to optimize the question prompts and target answer content of the steganography text, thereby improving the retrieval priority ranking. Second, it defines the target answer generation accuracy distortion by further optimizing the steganography text to ensure that the LLM accurately includes the target answer when regenerating the answer based on the steganography text, thus ensuring the accuracy of information extraction. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a text steganography method based on a retrieval-enhanced large language model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the StegoRAG system architecture according to an embodiment of the present invention. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0022] First, the relevant knowledge mentioned in this embodiment will be explained: RAG related knowledge: Existing research has shown that RAG systems' corpora are vulnerable to text injection vulnerabilities. Attackers can inject misleading, biased, or malicious content into the knowledge base through various means, thereby affecting the LLM's generation results and even inducing its output to conform to the attacker's intentions. For example, when the knowledge database contains millions of texts collected from Wikipedia, attackers can inject malicious text by maliciously modifying Wikipedia pages; if the knowledge database originates from the Internet, attackers can also inject malicious text by publishing fake news; internal personnel can also inject malicious text into a company's private knowledge database. The StegoRAG system proposed in this embodiment actively utilizes this vulnerability of RAG system corpus injection, transforming it into a carrier for achieving covert communication.

[0023] The RAG steganography system (StegoRAG) designed in this embodiment includes a search engine, a Large Language Model (LLM), and a dynamically updated independent corpus (the corpus text is sourced from Wikipedia). The system uses everyday question-and-answer queries as a public cover to achieve normal information exchange. The steganography sender has the authority to actively inject text into the RAG corpus and can also access the search engine in a white-box manner—this access is a prerequisite for steganography generation and optimization. The receiver only needs LLM access permissions to extract the steganographic information.

[0024] In the StegoRAG system architecture, the core publicly disclosed functionality is the LLM question-answering interface, while the RAG search engine and independent corpus remain confidential. The system faces the following threats: 1. Steganography Analyst: By calling the system's public interface to perform normal queries, collect multiple sets of "question-answer" sample pairs, and attempt to determine whether the system is engaging in text steganography by analyzing non-natural semantic features, logical contradictions, and other anomalies in the output text.

[0025] 2. RAG System Maintainer: To ensure the stability and accuracy of the StegoRAG system's publicly available search functions and improve the search adaptability of the corpus text in diverse question scenarios, maintenance personnel may perform operations such as rewriting definitions, replacing synonyms, and supplementing contextualized content on existing texts in the corpus. Such routine maintenance may disrupt the structural integrity or semantic relevance of the steganographic text, leading to distorted steganographic information extraction or even complete inability to parse it.

[0026] Based on the aforementioned threats, StegoRAG's design must meet two requirements to ensure the feasibility and reliability of covert communication under the threat model: 1. Concealment To address the threat posed by steganalysis, the system must ensure that stegtext is indistinguishable from normal text at the semantic, syntactic, and logical levels, effectively evading detection after being injected into the corpus. Specifically, stegtext must simultaneously satisfy three key characteristics: semantic naturalness, logical consistency, and statistical indistinguishability. To evaluate the system's concealment effectiveness, multiple analytical methods were employed in the experiment: perplexity was used to measure the fluency and semantic consistency of the text; and concealment detection was performed based on mainstream steganalysis models, thus achieving a multi-dimensional assessment of the concealment effectiveness of stegtext.

[0027] 2. Robustness To address routine operational threats from system maintainers, the system must ensure that steganographic text can still effectively convey information even after the corpus is dynamically updated. Specifically, this includes: (1) Text structure and semantic robustness: It can resist the destruction of the integrity of the steganographic text structure and semantic relevance by operations such as paraphrasing and synonym replacement, and maintain the guiding signal of the steganographic text to the target answer unchanged.

[0028] (2) Retrieval robustness: It can resist the impact of knowledge expansion, text optimization and other operations on retrieval competitiveness, and ensure that the steganographic text can still be stably recalled by the retrieval machine in the updated corpus, so as to enter the context of LLM.

[0029] If steganographic text loses its search competitiveness due to maintenance operations, subsequent answer generation and message extraction processes will directly fail. Therefore, retrieval robustness is a fundamental prerequisite for covert communication. To address this robustness requirement, this solution constructs and optimizes a joint objective function based on HotFlip to simultaneously enhance the retrieval effectiveness and answer guidance capabilities of steganographic text, thereby improving its robustness in dynamic corpora.

[0030] This embodiment proposes a text steganography method based on a retrieval-enhanced large language model, including: The sender and receiver pre-set a set of carrier questions; The sender generates a target answer corresponding to the secret information based on the secret information to be transmitted, using a preset embedding sequence and target answer mapping table; Based on the target answer, generate steganographic text containing the target answer, and inject the steganographic text into the corpus of the retrieval enhancement generation system; The receiver inputs the carrier question, generates a response containing the target answer, and recovers the secret information through reverse mapping based on the response containing the target answer.

[0031] Specifically, such as Figure 2 The StegoRAG framework leverages the retrieval and generation capabilities of the RAG system to achieve covert communication. Both the sender and receiver pre-set a carrier question set derived from the system's publicly available corpus and construct an "Embedding sequence - Target answer" mapping table (ES2TA-Map). The sender encrypts the original secret message to obtain the sequence to be embedded and, based on this, the target answer to the carrier question. StegoRAG then generates steganography containing the target answer for the carrier question and performs dual-objective optimization on the text: priority retrieval ensures that the steganography, after being injected into the corpus, can be reliably retrieved by the retrieval system to the relevant text set corresponding to the carrier question; the latter ensures that when the system's LLM answers the question based on this context, the output text contains the target answer. Finally, the optimized steganography is injected into the RAG corpus, completing the covert embedding of the secret information. The receiver sequentially inputs carrier questions, triggering the RAG system to retrieve and guide the LLM to output text containing the target answer. The output text is then matched against the target answers in the ES2TA-Map for similarity. The embedding sequence corresponding to the answer with the highest matching degree is the extracted sequence. This sequence is then decrypted to obtain the secret message, completing the covert communication process. The algorithm flowchart is shown below. Figure 1 As shown.

[0032] Furthermore, the embedding sequence and target answer mapping table constructs a target answer set for each carrier question, and each target answer corresponds to a unique embedding sequence, realizing a one-to-one bidirectional mapping between secret information and target answers.

[0033] Specifically, before establishing a covert communication link, the two parties need to negotiate and synchronize communication parameters through a secure channel, including: (1) a shared RAG system; (2) a key for encrypting the original plaintext; (3) an ordered set of carrier questions as the basis for triggering retrieval; and (4) a one-to-one mapping table between the sequence to be embedded and the target answer for the carrier questions.

[0034] Mapping table construction includes: (1) Selection of carrier: To ensure the flexibility and covertness of steganographic communication, the carrier issue... Prioritize selecting open-ended questions from the knowledge base (such as "someone's profession" or "the genre of a certain movie"). This facilitates the subsequent construction of a large-capacity set of semantically natural and independent target answers. Arrange the selected questions in order to form an ordered set: .

[0035] (2) Construction of the target answer set: For each carrier problem ( ), construct its target answer set Each target answer ( ), must satisfy: 1) Semantic rationality: as 1) Natural and reasonable response; 2) Discrete and distinguishable: any two answers and ( Clearly distinguish between them in terms of semantics or expression.

[0036] (3) Mapping table generation: Based on the carrier problem and its target answer set Construct a mapping table , The length of the secret message that can be sent based on this carrier-based problem communication. binary index sequence and target answer set It is a bidirectional, one-to-one correspondence. The formal definition of the mapping table is as follows: (1); In the formula, For embedding sequences and mapping tables to target answers; The length of the secret message that can be sent in carrier-based communication; Let be the set of target answers, where each target answer ( ).

[0037] against The problem of constructing a carrier And it implements steganography, allowing for message lengths of [length to be specified]. That is, the total embedding amount of steganographic communication is Bit.

[0038] Suppose the secret message to be transmitted is a binary sequence. key The ciphertext sequence is obtained by encrypting with a stream cipher. ,Right now: (2); In the formula, Represents binary XOR operation, ciphertext length .

[0039] Based on ciphertext length Number of carrier-related issues and the number of bits carried by each question , ciphertext sequence Sequentially divided into A sequence of non-overlapping sub-ciphertexts : (3); (4); (5); In the formula, || represents the binary sequence concatenation operation; For the first A sequence of ciphertexts, by It is divided into segments.

[0040] For each sub-ciphertext sequence Through the corresponding mapping table Find the corresponding target answer: (6); In the formula, For the carrier problem The corresponding number One target answer; This answer will be used for subsequent generation of steganographic text and information embedding.

[0041] Furthermore, generating steganographic text containing the target answer based on the target answer includes: Generate initial steganographic text containing the target answer related to the carrier question using a large language model; The initial steganographic text is optimized to obtain steganographic text containing the target answer; The optimization process includes: The initial stegtext is iteratively optimized by minimizing a joint loss function, which is constructed by combining retrieval priority loss, answer similarity loss, text quality loss, and steganalysis loss. The retrieval priority loss is used to obtain a higher retrieval ranking by increasing the semantic similarity between the steganographic text and the carrier problem; The answer similarity loss is used to guide the output answer by improving the semantic similarity between the response generated by the large language model based on the steganographic text and the target answer.

[0042] Specifically, each steganographic text The carrier issue needs to be included. The corresponding target answer To increase the probability of steganographic text being recalled during the retrieval phase, thereby improving the success rate of covert communication, multiple pieces of steganographic text need to be generated and injected for a single carrier problem. .

[0043] Steganographic text constraints: (1) Search criteria: Suppose the search engine searches Individual and The most relevant text serves as the context for the LLM, steganography. The issue of carrier needs to be addressed in the search engine. The top-k search result set must be selected; otherwise, it cannot participate in the context construction of LLM. The output should contain the target answer. The reply. Therefore, steganographic text. With carrier issues The embedding vectors need to have sufficiently high semantic similarity to ensure that the retrieval machine can correctly identify them. To achieve precise recall, namely: (7); In the formula, and These are the question encoder and text encoder in the retrieval system, respectively. To be responsible for generating the embedding vector for the problem, Generate a corresponding vector for each text in the corpus. Depending on the retrieval configuration, the two encoders may be the same or different. Represents the original corpus and Related text.

[0044] The corpus after injecting steganographic text is defined as follows: (8); In the formula, To inject steganographic text into the target corpus, The original corpus without injected steganographic text; This is a collection of steganographic text.

[0045] The search engine targets after injecting steganographic text. The returned Top-k related text set is defined as follows: (9); In the formula, the input problem Corpus Number of recalls Output and Most relevant a collection of texts (Sorted by relevance in descending order).

[0046] The retrieval tool retrieves information from the knowledge base. In the context of acquisition and carrier issues The most similar sets of related texts are used as the context for input to the LLM.

[0047] Steganographic text must satisfy the constraint of being included in the Top-k set: (10).

[0048] (2) Generation conditions: when In After obtaining the Top-k search results, it is necessary to further ensure that LLM prioritizes generating results similar to those obtained from the Top-k search. Semantically equivalent output text. Its core logic lies in: steganographic text. It needs to be highly guiding, specifically including two requirements: 1) Single text guidance: When only using... When used as the input context of an LLM, the LLM should be able to stably generate [data / features]. Semantically equivalent output text; 2) Mixed text guidance: when When LLM is used in conjunction with raw text and other steganographic text in a corpus to form its context, it can still maintain a high probability of success. Generation and Semantically equivalent output text.

[0049] This objective can be achieved by maximizing the probability of LLM outputting this type of text. For simplicity, the following will use... Representative and Semantically equivalent output text: (11); (12); (13); Formula (11) indicates that after inputting the carrier problem and steganographic text into the LLM, the LLM generates and... Semantically equivalent output text. Formula (12) represents the carrier problem and the retrieval machine targeting... From the corpus The relevant text set retrieved in the search After inputting LLM, LLM generation and Semantically equivalent output text. Equation (13) represents maximizing the following probability: the retriever targets... From the corpus The relevant text set retrieved in the search After being input into the LLM context, the LLM generates and The probability of semantically equivalent output text.

[0050] (3) Concealment conditions: The core objective of steganography is to achieve secure and covert transmission of information. Therefore, in addition to meeting the aforementioned retrieval and generation conditions, it is also essential to ensure that the text itself does not reveal any steganographic behavior. This embodiment imposes constraints from two levels: naturalness of language and statistical traces. The specific conditions are as follows: Text perplexity hidden conditions: To prevent the embedding of secret information or optimization of retrieval relevance from resulting in awkward or unnatural text that could arouse human suspicion, PPL distortion loss is used as a constraint, while also requiring the absolute perplexity of the steganographic text. It falls within a reasonable range of normal text. This ensures that the steganographic text is indistinguishable from the ordinary carrier text in terms of sentence structure, grammar, and semantic fluency, thus avoiding censorship risks from the perspective of language perception.

[0051] Statistical feature concealment conditions: Even if the text reads naturally, its underlying statistical features may still deviate from their normal distribution due to information embedding, thus making it detectable by advanced steganalysis models. Therefore, a steganalysis distortion loss is introduced for control, requiring that this loss not exceed a preset threshold: .

[0052] This constraint ensures that the stegtext remains consistent with the normal text in the deep feature space, thereby effectively evading detection by steganalysis tools and achieving statistical concealment.

[0053] Initial steganography generation: LLM (Library Locator) is used to generate stegtext by providing instructions to generate text containing both the carrier question and the target answer. By introducing a non-zero temperature coefficient to the LLM, generation randomness is achieved, allowing the same prompt to produce stegtexts with varying expressions. These texts maintain semantic relevance to the carrier question and guide the LLM to generate stegtexts that are consistent with the target answer. Under the premise of semantically equivalent content, they possess different sentence structures and expression styles.

[0054] Targeted question and answer pair The system calls the LLM API interface and sends a generation request through preset prompt words (see appendix for details), explicitly requesting the LLM to generate steganographic text consisting of question-related text and answer guidance text. The relevant part of the question is: Synonyms, the answer guidance section implicitly guides LLM output. Semantic clues; and at the same time, require steganographic text Satisfying two major constraints—using it as context and When using a common input LLM, the model output must include the same information as the input LLM. The generated text must be semantically equivalent, and the total length of the steganographic text must not exceed a preset threshold V. If the generated text does not meet any of the above conditions, the system will trigger a regeneration mechanism. This process will be repeated. This process ultimately yields a set of initial steganographic text. .

[0055] (14).

[0056] To ensure that stegtext can be reliably recalled during the retrieval phase and accurately guide the LLM to output the target answer during the generation phase, this embodiment establishes a joint text quality constraint triggering and optimization mechanism for two scenarios. The text optimization process is triggered when the initial stegtext meets either of the following conditions: Search similarity not met: The semantic similarity between the initial steganographic text and the carrier question did not reach the preset threshold. At this time, even if the text contains the target answer, it cannot be stably recalled to the Top-k result set by the RAG search engine, which leads to the interruption of the subsequent information extraction process. Answer guidance is inadequate: The semantic similarity between the initial steganographic text and the carrier question has met the standard, but the similarity with the target answer has not reached the corresponding threshold. At this point, although the text can be retrieved, it cannot effectively guide the LLM to generate output containing secret information, resulting in information transmission failure.

[0057] Text quality is substandard: The fluency or naturalness of the initial steganographic text is lower than that of normal text (the level of confusion exceeds a reasonable range), which may attract the attention of human reviewers and affect the confidentiality of communication.

[0058] Statistical concealment is not up to standard: The statistical characteristics of steganographic text deviate significantly from the distribution of normal text, which may be detected and identified by steganographic analysis models (such as SALT-LS), leading to the exposure of steganographic behavior.

[0059] Once any of the above conditions is triggered, an iterative update process based on multi-objective optimization will be initiated until stegtext that simultaneously satisfies the requirements of retrieval recall, generation guidance, natural language, and statistical concealment is generated.

[0060] The optimization goals of this stage are to simultaneously meet the triple requirements of stable retrieval recall, accurate information extraction, and text naturalness: (1) improve the retrieval similarity with the carrier question to ensure that the steganography is stably recalled by the retrieval system (Formula 10); (2) strengthen the answer guidance capability for LLM (Formula 11) to ensure that the recipient can accurately extract secret information; (3) introduce text naturalness (perplexity level PPL) as a basic constraint to prevent text semantic distortion or decreased fluency during the optimization process; (4) ensure statistical concealment to ensure that the deep statistical features of the text are consistent with the distribution of normal text. In the previous stage, we have already prepared for the question-answering pair Generated Initial Steganographic Text Let the core parameters of the RAG search engine be... (That is, return the Top-k search results), and based on this, construct an optimization objective with dual constraints: (1) High search priority constraint: Steganographic text Need and carrier issues To maintain high semantic relevance and ensure stable recall to the Top-k result set by the retrieval system, the retrieval priority loss is defined as the complement of the cosine similarity between the steganographic text and the semantic embedding of the carrier question, as shown in the following formula: (15); In the formula, This is the cosine similarity function. The core logic of search optimization is to minimize... Essentially, this is equivalent to maximizing the semantic similarity between the steganographic text and the carrier problem, thereby improving the stability of retrieval and recall.

[0061] (2) High answer similarity constraint: When steganographic text When used as context input for an LLM, it is necessary to ensure that it can effectively guide the generation of the LLM. Semantically equivalent output text: (16); To quantify the guiding ability of steganographic text on large language models, this embodiment employs a two-layer answer extraction mechanism: substring matching as the primary method and semantic matching as a secondary method. This mechanism aims to address common expression uncertainties in LLM output, such as paraphrasing, sentence transformation, and information redundancy. Specifically, the system prioritizes judging the consistency between the steganographic text and the target answer based on substring matching; if the match fails, a semantic similarity-based answer extractor is activated as a candidate solution. During the optimization phase, this mechanism is used to calculate the similarity between the LLM output and the target answer; at the receiving end, it is used to extract the content that best matches the semantics of the target answer from the output. This design balances matching efficiency and semantic fault tolerance, thereby ensuring the robustness and accuracy of answer extraction throughout the entire process.

[0062] The specific process for optimizing answer similarity is as follows: Candidate fragment extraction: When a substring match fails, the Spacy natural language processing tool is used to... Structured parsing is performed, sequentially completing sentence segmentation and named entity recognition (nER). Based on syntactic analysis, complete sentences, named entities, and core noun phrases are extracted to construct a candidate fragment set. ,in, The number of candidate segments, Representing the 1 candidate fragment.

[0063] Semantic vector computation: The candidate fragment set is obtained through the Sentence-BERT model. All elements in the target answer Each segment is encoded into a fixed-dimensional semantic vector. Let candidate segments be denoted as... The semantic vector is Target answer The semantic vector is .

[0064] Match score calculation: Calculate each candidate fragment With the target answer The cosine similarity is used as the maximum value. With the target answer Final matching score : (17); (18); Based on the extraction results, the answer similarity loss is defined as the complement of the final matching score, in order to achieve the optimization goal of "maximizing the matching score and strengthening the answer guidance ability". The formula is as follows: (19); In the formula, Indicates that alone The output text generated by LLM in context is used to evaluate the answer similarity loss; From The set of candidate fragments extracted from; This is a semantic encoding function based on Sentence-BERT, which encodes text into a fixed-dimensional semantic vector; Calculate candidate fragments With the target answer The similarity between semantic vectors.

[0065] The loss function is designed to optimize the guidance performance of steganographic text by maximizing the semantic similarity between candidate fragments and the target answer: when When it approaches 1, A value close to 0 indicates that the steganographic text successfully guided the model to output content containing the target answer; conversely, when... When it approaches 0, A value approaching 1 indicates a boot failure. Therefore, by minimizing... The optimization process can directly enhance the ability of steganographic text to guide LLM in generating relevant expressions for the target answer.

[0066] Furthermore, text quality loss is used to control for changes in the natural language quality of the optimized stegtext relative to the initial stegtext. Specifically, to ensure the quality of steganographic text and avoid detection and identification by steganalysis technology due to low text naturalness and semantic distortion, a normalized loss term based on perplexity (PPL) is introduced in the gradient optimization process. This term is used to quantify the degree of change in the quality of the optimized text relative to the initial text and is incorporated into the total loss function for joint optimization. This achieves the steganographic goal while strictly controlling the text quality.

[0067] The normalized relative PPL loss is calculated using the following formula: (20); In the formula, The perplexity of the original text (the initial text containing the target answer) reflects the natural language fluency of the original text; the lower the value, the more natural the text. To optimize the perplexity of the steganographic text and reflect the quality of the optimized text; The maximum acceptable perplexity threshold (i.e., tolerance limit) preset for the task corresponds to the maximum tolerable increase in PPL. .

[0068] like (If the optimized text has a higher or equal level of naturalness than the original text), then let (Loss is 0); like (If the original text exceeds the acceptable limit), then let .

[0069] Furthermore, the steganalysis loss is optimized based on the steganalysis model. By reducing the probability that stegtext is identified as a steganalysis carrier, it ensures that the features of stegtext and normal text converge, effectively avoiding the risk of steganalysis detection.

[0070] Specifically, to resist machine learning-based steganalysis, the statistical features of stegtext must be consistent with those of normal text. To this end, this embodiment introduces an adversarial loss based on information entropy, forcing the stegtext to align with the distribution of normal text in the feature space, making it difficult for the analysis model to distinguish them. This loss aims to maximize the uncertainty of the model's classification of stegtext, and can be formalized as: (twenty one); in, and These represent the confidence levels of the model in determining the input (steganographic text) as normal text and steganographic text, respectively. By minimizing the normalized entropy loss, statistical concealment is achieved, effectively evading detection.

[0071] Furthermore, the optimization process is performed using a text adversarial optimization algorithm based on HotFlip, which iteratively replaces text words to obtain steganographic text that meets the optimization objective.

[0072] Specifically, to comprehensively achieve retrieval priority optimization, target answer similarity guidance, and high-quality text constraints, this embodiment constructs the following joint loss function, aiming to generate the optimal stegtext set that simultaneously satisfies steganalysis requirements and natural language quality: : (twenty two); In the formula, , , , To balance hyperparameters (satisfy) This hyperparameter is used to dynamically balance the optimization weights of retrieval priority, answer guidance capability, text quality constraints, and statistical concealment. It can adaptively adjust based on the retrieval difficulty, LLM model characteristics, text quality requirements, and steganalysis risk level in actual communication scenarios, thereby achieving the optimal balance between steganalysis performance, text naturalness, and statistical concealment.

[0073] HotFlip (Greedy Coordinate Gradient, GCG) is selected as the core optimization algorithm. The steganography optimization algorithm based on GCG takes the initial steganography as input and aims to minimize the joint loss function. To optimize the target, the candidate text is continuously updated through iterative word-level greedy search and replacement. The convergence condition is reaching the maximum number of iterations. Alternatively, it may meet the preset early stopping conditions (both retrieval similarity and answer similarity reach the corresponding preset thresholds). Output steganographic text. This ultimately forms a collection of steganographic text. .

[0074] (twenty three); in, Optimize the function for GCG.

[0075] The GCG method does not require complex hyperparameter tuning, has low computational cost per iteration, and balances optimization quality with generation efficiency, making it suitable for real-time requirements in covert communication scenarios.

[0076] Furthermore, the receiver generates a response containing the target answer by inputting the carrier question, and recovers the secret information through reverse mapping based on the response containing the target answer, including: By sequentially inputting the carrier question into the retrieval enhancement generation system, the corresponding large language model output text is obtained; For each output text, a two-layer answer extraction process is performed: first, an attempt is made to directly identify the preset target answer from the output text through substring matching; if the matching fails, the candidate segment that is semantically closest to the target answer is extracted from the output text as the extracted answer through semantic similarity calculation. The extracted answer is matched with all candidate target answers in a pre-stored mapping table to determine the final target answer; Based on the mapping table, the final target answer is reverse-mapped into the corresponding embedding sequence; By sequentially concatenating the embedded sequences corresponding to all carrier problems to form a complete ciphertext sequence and then decrypting it, the original secret information can be obtained.

[0077] Specifically, to improve the system's robustness to representational noise in the LLM output, the receiver introduces a two-layer processing strategy of substring matching + semantic similarity fuzzy matching before performing the reverse mapping operation of the secret message. This strategy, combined with the previously designed two-layer answer extractor, achieves robust recovery of the secret message. The specific process is as follows: Output answer text extraction: The receiver-based pre-shared ordered carrier problem group In the agreed-upon RAG system, the questions are entered sequentially according to a preset order. The system then returns a Top-k context set containing the target steganographic text through a retrieval mechanism. After inference by the large language model, the corresponding ordered output text is output. For each output text, a two-layer answer extractor is invoked to extract the core content: (1) Prioritize substring matching: Corresponding carrier problem The target answer candidate set is subjected to substring matching. If a match is successful, the matching result is directly used as the output answer text. ; (2) Semantic matching candidates: If the substring matching fails, extract sentences, named entities and core noun phrases from the LLM output text to construct a candidate fragment set; then calculate the similarity between each candidate fragment and the corresponding carrier question, and, combined with the premise that the injected text contains only a unique target answer, select the candidate fragment with the highest similarity as the output answer text. .

[0078] (twenty four).

[0079] Semantic fuzzy matching: For each output answer text Perform differentiated processing: (1) If The result obtained by matching the substring is directly used as the final matching result. ; (2) If If obtained through semantic matching (answer extractor), then the Sentence-BERT encoder is used to match it with the corresponding carrier question. All target answers in the target answer mapping table Encode the target answer as a semantic vector, calculate the cosine similarity between each pair of targets, and select the target answer with the highest similarity as the matching result for that round. : (25); Reverse mapping of sub-ciphertext sequences and ciphertext concatenation: The receiver uses a pre-shared "target answer – embedded sequence" mapping table to assign each matched answer... The reverse mapping is to the corresponding sub-ciphertext sequence. All sub-ciphertext sequences are concatenated sequentially according to the input order of the carrier problem to form a complete ciphertext sequence. .

[0080] (26); (27); Plain text recovery: The original secret message is obtained after the ciphertext sequence is decrypted.

[0081] (28); This process employs a layered processing mechanism to effectively tolerate various noises in LLM output caused by representational uncertainty. The semantic fuzzy matching step only applies to text where substring matching fails, achieving robust adaptation to representational noise and ultimately ensuring the complete and accurate recovery of secret messages in complex output scenarios.

[0082] The core objective of this embodiment is to ensure the accurate transmission of secret messages through the StegoRAG framework, meaning that the message sequence ultimately recovered by the receiver is completely consistent with the original secret message. To this end, the system's correctness is verified below by simulating a complete covert communication process, focusing on both retrieval effectiveness and message recovery accuracy.

[0083] 1. Experimental setup: Data and Model: The corpus was built based on the HotpotQA dataset, injecting m steganographic texts into each carrier question while retaining relevant native texts to form a hybrid corpus. The Contriever retrieval tool was used, returning the top-k context texts for each carrier question. Steganographic text generation employed GPT-4o, with the model temperature parameter set to 0.1. During generation, the maximum number of trials L=20 and the maximum text length V=90 were set.

[0084] Communication parameters: 6 bits of binary plaintext are randomly generated for each experiment and XORed with a fixed key '101010' to generate different ciphertext sequences; the carrier problem selects occupational classification questions from the dataset and establishes a "question-target answer" relationship based on a pre-built mapping table of 64 target answers (carrying 6 bits).

[0085] Experimental environment: The experiment was conducted on a hardware environment equipped with a RAX4090 graphics card. The experimental code was implemented using the Python 3.10 programming language, and the deep learning framework used was PyTorch 1.13.0 (with CUDA version 11.7).

[0086] 2. Evaluation Indicators: (1) Search validity indicators: False Detection Rate (FDR): Measures the degree of interference from non-steganographic text in the search results. It is calculated as the percentage of non-steganographic text in the Top-k search results for the carrier question (FDR = number of non-steganographic texts in Top-k / k). If the FDR is close to 0, it indicates that almost all of the Top-k results are steganographic text, proving that HotFlip's similarity optimization strategy can effectively improve the search ranking of steganographic text, allowing it to stably enter the LLM context and provide necessary steganographic information support for subsequent answer generation.

[0087] Recall Rate (RcR): Measures the retrieval coverage of stegtext. After injecting N stegtexts into the search database, it represents the actual number of stegtexts included in the "Top-N Search Results" (RCR = number of stegtexts in Top-N / N). If RcR is close to 1, it indicates that the injected stegtexts can be retrieved at a high rate, proving that the HotFlip-optimized stegtexts have not lost their relevance due to perturbation and can effectively support the subsequent LLM answer generation.

[0088] (2) Message recovery accuracy index: For each experiment (using a single carrier problem), the receiver recovers the ciphertext sequence from the LLM output and calculates its Hamming distance to the original ciphertext. When the Hamming distance is 0, it is determined that the message recovery is accurate in a single experiment (i.e., the single accuracy rate is 100%). The overall message recovery accuracy rate (AccuracyRate, AR) of multiple independent experiments is calculated using the formula: AR = Number of successful experiments with a Hamming distance of 0 / Total number of experiments. The closer the AR value is to the theoretical upper limit (100%), the higher the reliability and accuracy of StegoRAG in ciphertext transmission under real communication environments.

[0089] 3. Example of a complete communication process: In the complete communication process presented in Table 1, the experiment pre-set the retrieval similarity threshold to be higher than the highest similarity of the original documents in the knowledge base. In this embodiment, the initial steganographic text already contained the target answer, and its retrieval similarity, ppl, and steganalysis anomaly confidence all met the preset conditions, so the optimization process was not triggered. At this time, the Top-5 retrieval results all pointed to the steganographic text, providing sufficient contextual support for the generation process of the Large Language Model (LLM). Finally, the output content of the LLM successfully covered the target answer, and the receiver extracted data from the output text according to the preset rules to complete the information recovery.

[0090] Table 1 4. Verification of stegtext retrieval coverage capability: This experiment verifies the effectiveness of steganography retrieval, providing a foundation for subsequent experiments in interference scenarios. The experiment employs a controlled variable design: the value of k in "Top-k retrieval" is fixed to be equal to the number of steganography injections n (i.e., Top-N retrieval). n is set to different injection scales of 4, 8, 16, 32, and 64, with 30 independent experiments conducted for each scale. Recall (RcR) is used as the core evaluation metric to measure the retrieval coverage of the injected steganography.

[0091] The experimental results are shown in Table 2. The recall rate reached 100% for all injection scales. This result shows that under the Top-n retrieval setting, all steganographic text injected by the sender can be accurately recalled by the retrieval system without any omissions, thus clarifying the feasibility of "the steganographic text injected by the sender can be completely retrieved".

[0092] Table 2 5. Steganographic text retrieval priority verification: This experiment verifies whether the retrieval priority of stegtext is higher than that of the original corpus text. The experimental design is as follows: a fixed number of stegtext injections n is used to construct a scenario where the number of stegtexts covers the number of retrieval slots by adjusting the total number of retrieval results k (i.e., n≥k, ensuring that the number of stegtexts is sufficient), and the proportion of non-stegtext text under different k values ​​is evaluated; 30 independent and repeated experiments are conducted for each group (n,k), with the false detection rate (FDR) as the core indicator.

[0093] The experimental results are shown in Table 3. The false detection rate (FDR) was 0% under all experimental configurations. This result indicates that as long as the amount of steganographic text injected is sufficient (i.e., n≥k), it can occupy all search results and exclude relevant text from the original corpus. Therefore, the proposed optimization strategy can effectively elevate the retrieval priority of steganographic text above that of the original text.

[0094] Table 3 6. Verification of covert communication performance under interference environment This experiment verifies the actual communication performance of the scheme in a real interference scenario—that is, whether it can accurately recover secret messages when the search results are mixed with a large amount of non-steganographic text. The experimental design is as follows: a fixed total number of search results k is used to simulate an interference scenario where "some of the search results are steganographic text and the rest are irrelevant original text". Different steganographic text injection ratios are set, and 30 independent repeated experiments are conducted for each configuration. The message recovery accuracy rate is used as an indicator to quantify the communication reliability.

[0095] The experimental results are shown in Table 4. This result shows that even if the search results are diluted by irrelevant text, the proposed scheme can still recover the complete secret message from a limited and impure search context; and under the constraint of a fixed number of searches k, reliable communication can be achieved by injecting only about 50% of the steganographic text, verifying the efficiency and robustness of the scheme in real interference environments.

[0096] Table 4 The quality of steganographic text and system behavior are key to resistance to detection. This experiment verifies these aspects from two perspectives: text quality and system resistance to analysis.

[0097] Text quality check: Steganographic text quality is a fundamental prerequisite for undetectability. To ensure that stegographic text is semantically and syntactically indistinguishable from native text and to avoid detection due to quality anomalies, PPL loss (perplexity loss) is introduced in the generation optimization process, and the following detection is performed: 1. Text quality inspection based on perplexity: The perplexity (PPL) of steganographic text is calculated using a GPT-2 pre-trained language model. A lower PPL value indicates more coherent semantic logic and better adherence to natural language rules. This approach incorporates the PPL loss into the optimization objective of steganographic text generation, and iteratively optimizes and controls the PPL value of the steganographic text to ensure that it maintains high text quality throughout the optimization process of embedding steganographic information.

[0098] 2. Duplicate text detection and deduplication: Text repetition can disrupt the original statistical features of a corpus, creating detectable repetition patterns and thus exposing steganography injection traces. To ensure the diversity of steganographic text, the experiment added a duplicate text detection and deduplication step before injecting steganographic text into the corpus: a unique hash value (SHA-256) was calculated for each generated steganographic text, and the hash values ​​were compared to determine if the text was duplicated. If a hash collision was detected, only a single valid text was retained, and the remaining duplicate texts were regenerated until no duplicate texts were found.

[0099] By eliminating the repetition of steganographic text, abnormally repetitive text patterns are avoided in the corpus, maintaining the original content diversity statistical characteristics of the corpus, preventing attackers from capturing steganographic traces through text repetition rate analysis, and further enhancing the quality and concealment of steganographic text.

[0100] Text antisteganography analysis: The resistance to detection of steganographic text is a core guarantee for covert communication. To ensure that steganographic text is statistically indistinguishable from normal text, the SALT-LS steganalysis model is selected as the detection tool. This model can distinguish steganographic text from normal text by mining underlying statistical patterns such as word frequency distribution and semantic feature entropy. During iterative optimization, by reducing the statistical feature loss, the statistical features of steganographic text are driven to approximate the distribution of normal text, reducing the probability that the model will judge steganographic text as abnormal and enhancing the resistance to detection of steganographic text in the statistical dimension.

[0101] System's resistance to steganalysis: Even if the text quality meets the standards, analysts can still detect it through behavioral patterns. StegoRAG constructs the following dual anti-analysis barriers at the system level: (1) Difficulty in exhaustively listing carriers: Steganalists typically collect question-and-answer pairs from systems in bulk through public interfaces, attempting to uncover unusual patterns to pinpoint the steganographic carrier. However, StegoRAG steganographic carriers originate from a massive and diverse RAG corpus, and are selected randomly. Analysts cannot exhaustively identify all possible carriers, nor can they pinpoint the actual communication content through limited sampling, thus preventing the carrier from tracing its origin.

[0102] (2) The confusing nature of LLM hallucinations: One of the key principles of steganalysis is identifying anomalous features in the output text that contradict the facts or contain logical contradictions, and attributing them to steganography. However, the inherent "factual illusion" in large language models can generate erroneous content. When the output contains factual biases, analysts cannot determine whether the anomaly stems from steganography or a defect in the model itself. This confusion greatly increases the difficulty of detection and effectively masks the traces of steganography.

[0103] In summary, StegoRAG achieves its undetectability goal through high-quality steganographic text combined with system-level barriers that prevent the exhaustiveness of carriers and the illusion of model confusion.

[0104] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A text steganography method based on a retrieval-enhanced large language model, characterized in that, include: The sender and receiver pre-set a set of carrier questions; The sender generates a target answer corresponding to the secret information based on the secret information to be transmitted, using a preset embedding sequence and target answer mapping table; Based on the target answer, generate steganographic text containing the target answer, and inject the steganographic text into the corpus of the retrieval enhancement generation system; The receiver inputs the carrier question, generates a response containing the target answer, and recovers the secret information through reverse mapping based on the response containing the target answer.

2. The text steganography method based on a retrieval-enhanced large language model according to claim 1, characterized in that, The embedding sequence and target answer mapping table constructs a set of target answers for each carrier question, and each target answer corresponds to a unique embedding sequence, realizing a bidirectional one-to-one mapping between secret information and target answers; The embedded sequence and the target answer mapping table are represented as follows: ； In the formula, For embedding sequences and mapping tables to target answers; The length of the secret message that can be sent in carrier-based communication; Let be the set of target answers, where each target answer ( ).

3. The text steganography method based on a retrieval-enhanced large language model according to claim 2, characterized in that, Each answer in the target answer set must satisfy semantic rationality, and any two answers must be semantically or expressively distinguishable.

4. The text steganography method based on a retrieval-enhanced large language model according to claim 1, characterized in that, Generate steganographic text containing the target answer based on the target answer, including: Generate initial steganographic text containing the target answer related to the carrier question using a large language model; The initial steganographic text is optimized to obtain steganographic text containing the target answer; The optimization process includes: The initial stegtext is iteratively optimized by minimizing a joint loss function, which is constructed by combining retrieval priority loss, answer similarity loss, text quality loss, and steganalysis loss. The retrieval priority loss is used to obtain a higher retrieval ranking by increasing the semantic similarity between the steganographic text and the carrier problem; The answer similarity loss is used to guide the output answer by improving the semantic similarity between the response generated by the large language model based on the steganographic text and the target answer.

5. The text steganography method based on a retrieval-enhanced large language model according to claim 4, characterized in that, The text quality loss is used to control the change in natural language quality of the optimized steganographic text relative to the initial steganographic text. The steganalysis loss is optimized based on the steganalysis model. By reducing the probability that the steg text is identified as a steganalytic carrier, it ensures that the features of the steg text and the normal text converge.

6. The text steganography method based on a retrieval-enhanced large language model according to claim 4, characterized in that, Calculating the answer similarity loss includes: Candidate text fragments are extracted from the responses generated by the large language model, and the semantic vector similarity between each candidate fragment and the target answer is obtained. The highest semantic similarity is selected as the matching score, and the answer similarity loss is calculated based on the matching score.

7. The text steganography method based on a retrieval-enhanced large language model according to claim 4, characterized in that, The optimization process is performed using a text adversarial optimization algorithm based on HotFlip, which iteratively replaces text words to obtain steganographic text that meets the optimization objective.

8. The text steganography method based on a retrieval-enhanced large language model according to claim 1, characterized in that, The receiver inputs the carrier question, generates a response containing the target answer, and recovers the secret information through reverse mapping based on the response containing the target answer, including: By sequentially inputting the carrier question into the retrieval enhancement generation system, the corresponding large language model output text is obtained; For each output text, a two-layer answer extraction process is performed: first, an attempt is made to directly identify the preset target answer from the output text through substring matching; if the matching fails, the candidate segment that is semantically closest to the target answer is extracted from the output text as the extracted answer through semantic similarity calculation. The extracted answer is matched with all candidate target answers in a pre-stored mapping table to determine the final target answer; Based on the mapping table, the final target answer is reverse-mapped into the corresponding embedding sequence; By sequentially concatenating the embedded sequences corresponding to all carrier problems to form a complete ciphertext sequence and then decrypting it, the original secret information can be obtained.

9. The text steganography method based on a retrieval-enhanced large language model according to claim 8, characterized in that, The candidate segments that are semantically closest to the target answer are extracted from the output text using semantic similarity calculation as the extracted answer, including: The output text is segmented and entity-recognized using natural language processing tools to obtain several candidate text fragments. Each candidate text fragment and the target answer are encoded into a semantic vector; By calculating cosine similarity, candidate text segments that are closest to the semantic vector of the target answer are selected as the extracted answer.

10. The text steganography method based on a retrieval-enhanced large language model according to claim 8, characterized in that, Determining the final target answer includes: If the extracted answer is obtained through substring matching, then the corresponding extracted answer is directly used as the final target answer; If the extracted answer is obtained through semantic similarity calculation, then the candidate target answer with the highest semantic similarity is determined as the final target answer by calculating the semantic similarity between the corresponding extracted answer and each candidate target answer in the mapping table.

Images