A natural language generation-based Chinese steganography method and system

Abstract

The present application relates to a kind of natural language generation-based Chinese steganography method and system, it is related to information security, natural language processing field, the system framework includes: embedding covert information corresponding packet check code;Based on Chinese generation model and covert information generation steganography;Through Chinese generation model, steganography is decoded into bit stream set;Using check algorithm determines steganography extraction result.For the word segmentation problem of Chinese steganography, the present application proposes a series of innovations such as lightweight check mechanism, so that the receiver can correctly extract the effective covert information written in the text by the steganography party, ensure the robustness and computational feasibility of system;The present application realizes intelligent natural language generation by combining the Chinese pre-training language model based on large-scale corpus, with the characteristics of rich and diverse corpus, natural and smooth writing, effectively improves the embedding rate and concealment of steganography.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information security and natural language processing, and particularly relates to a Chinese steganography method and system based on natural language generation. BACKGROUND

[0002] With the advancement of global information sharing process, information security and data protection are associated with increased risk of major information leakage, which brings more serious economic losses to individuals, society and even industrial sectors. Information hiding and steganography, as a key technology of information security system, aims to ensure the security of information transmission without the awareness of the third party. Text-based information steganography has important security value and practical significance in network activities.

[0003] At present, the most common in the field of generative text steganography is English steganography. However, Chinese is one of the most commonly used languages in the world, and Chinese text steganography has high application value in global information security transmission. At present, most Chinese text steganography methods are based on traditional steganography methods such as content modification or format modification, which are difficult to resist advanced steganalysis technology. Although there are Chinese text generation models based on artificial intelligence technology, there are still the following technical challenges in directly applying the encoding and decoding process of generative English text steganography to Chinese: limited by the word segmentation of Chinese, the word segmentation method of the generated text by the steganographer and the extracted text by the receiver may be different, which may cause the receiver to make a wrong word segmentation and further extract the hidden text.

[0004] In order to expand the use scenarios of Chinese text steganography and cope with the challenge of information security transmission in non-controllable channels, and fully utilize the high embedding rate advantage of Chinese steganography, the present application proposes a Chinese steganography method and system based on natural language generation. The combination of Chinese text steganography and generative language model solves the word segmentation confusion problem of Chinese text steganography, and a series of innovations such as lightweight verification mechanism are proposed to ensure that the receiver can correctly extract the effective steganographic information written in the text by the steganographer. The present application realizes intelligent natural language generation based on a large-scale corpus of Chinese pre-trained language models, improves the low embedding rate phenomenon of traditional modified steganography methods, and overall improves the corpus richness, writing fluency and anti-steganalysis ability of the hidden text. SUMMARY

[0005] The purpose of the present application is to propose a Chinese steganography method and system based on natural language generation, which overcomes the problem that the existing Chinese steganography method cannot be effectively decoded, and solves the problems of poor semantic correlation, low embedding rate and weak text fluency of traditional steganography methods, and uses the language model trained by a large-scale corpus and the verification mechanism to overcome the uncertainty of Chinese hidden text decoding word segmentation, and improve the robustness and concealment of the steganography system.

[0006] To achieve the above purpose, the present application provides the following scheme:

[0007] A Chinese steganography method based on natural language generation, the method comprising the following steps:

[0008] (1) The steganographer trains a Chinese language generation model through a large-scale corpus;

[0009] (2) The steganographer converts the covert information to be embedded into a bit stream form, groups the bit stream according to a specific rule, adds a corresponding check code at the end of each group, and obtains a grouped bit stream with a check function;

[0010] (3) The steganographer gradually generates phrases according to the specified initial context, the Chinese language generation model obtained in step (1), and the bit stream obtained in step (2). When a phrase is generated, a phrase candidate pool and the probability distribution of each candidate phrase are first constructed, and a phrase is selected from the candidate pool according to the encoding rule to generate text until the termination condition is reached, forming a complete steganography.

[0011] Optionally, the selection of the Chinese language generation model in step (1) includes but is not limited to a language generation model based on an attention mechanism, a recurrent neural network, and a Markov chain. The selection of the pre-trained Chinese corpus includes but is not limited to essays, social media, and news corpora.

[0012] Optionally, the grouping and checking in step (2) inserts a check code (n i bits) after the i i th bit stream (m m bits) in turn; wherein the selection of the check algorithm includes but is not limited to checksum, XOR check, cyclic redundancy check (Cyclic Redundancy Check, CRC), message-digest algorithm (Message-Digest Algorithm, MD), secure hash algorithm (Secure Hash Algorithm, SHA), etc.

[0013] Optionally, in the process of generating a single phrase in step (3), a candidate pool CP = [word1, word2, …, word m ] of size m and a probability distribution P = [p1, p2, …, p m ] corresponding to each phrase in the candidate pool are generated word by word according to the context E, a salt value s (used to enhance the randomness of natural language generation), and the Chinese language generation model. According to the encoding method, a phrase word j that matches the encoding state is selected from CP as the next word of the steganography, and the context E is updated to increase the new text formed by word j .

[0014] Wherein, the step (3) encoding mode can be selected from local encoding or global encoding; wherein, the local encoding includes but is not limited to complete binary tree encoding, Huffman encoding, etc., and the global encoding includes but is not limited to arithmetic encoding, interval encoding, etc.

[0015] Further, the encoding state of the local encoding is determined by the remaining bit stream, the way to change the encoding state is to advance the bit stream traversal, and the termination condition is no remaining bit stream; the encoding state of the global encoding is determined by the encoding interval, the way to change the encoding state is to narrow the encoding interval, and the termination condition is that the current encoding interval can be estimated as a complete bit stream.

[0016] The application also provides a Chinese steganography system based on natural language generation, the system comprising:

[0017] (1) steganography generation module: for generating Chinese steganography embedded with hidden information, wherein the Chinese steganography is generated by the Chinese steganography method based on natural language generation of claim 1;

[0018] (2) steganography extraction module: for receiving party to decode the steganography into a bit stream set;

[0019] (3) steganography verification module: for screening out the only bit stream that meets the verification, the receiving party uses the verification algorithm used by module (1) to verify the bit stream set generated by module (2) one by one to determine the steganography bit stream.

[0020] According to the specific embodiments provided by the application, the following technical effects are disclosed:

[0021] (1) strong robustness, the application proposes a lightweight grouping verification mechanism, which can overcome decoding errors caused by Chinese word segmentation uncertainty through low occupancy rate verification code, and can also resist decoding errors caused by active attacks on uncontrolled channels, ensuring that the receiving party can quickly screen and correctly extract valid information.

[0022] (2) high embedding rate, the application embeds hidden information through a word-level generative Chinese model, and embeds several bits of hidden information in the generation process of each word, so that the information embedding rate is much higher than that of the modification type Chinese steganography method based on text image, format and content.

[0023] (3) strong concealment, the application realizes intelligent natural language generation by using a Chinese pre-trained language model, and is based on a large-scale Chinese public corpus, which has the characteristics of rich and diverse corpus, natural and fluent writing, strong generalization ability, etc., effectively improves the text quality and concealment of Chinese steganography, and enhances its ability to resist steganalysis algorithm investigation and human subjective evaluation investigation. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings described below are only some embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort on the basis of these drawings.

[0025] Figure 1 A flowchart of the Chinese steganography method based on natural language generation provided in Embodiment 1 of the present application (based on arithmetic coding);

[0026] Figure 2 A flowchart of the Chinese steganography method based on natural language generation provided in Embodiment 2 of the present application (based on Huffman coding);

[0027] Figure 3 A structural schematic diagram of the Chinese steganography system based on natural language generation provided in Embodiment 3 of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort belong to the scope of protection of the present application.

[0029] The purpose of the present application is to provide a Chinese steganography method and system based on natural language generation, which improves the concealment of the transmission of the steganography and the robustness of the system by combining Chinese steganography with natural language generation model and lightweight verification mechanism.

[0030] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0031] Embodiment 1

[0032] This embodiment is used to provide a Chinese steganography method based on natural language generation (based on arithmetic coding), as shown in Figure 1 The method comprises the following steps:

[0033] (1) The steganography party trains the Chinese language generation model through a large-scale corpus;

[0034] In this embodiment, the Chinese language model described in step (1) can be the large-scale Chinese pre-trained model CPM. This model is the first large-scale Chinese pre-trained model since the release of GPT-3. Compared with other Chinese pre-trained models, this model has many advantages such as strong learning ability, rich and diverse corpus, and natural and fluent writing.

[0035] Furthermore, the model uses over 200,000 complete essays written by humans as its training corpus;

[0036] Specifically, the training details include: the optimizer is Adam, and the learning rate is initially set to 1.5 × 10⁻⁶. -4 The batch size is set to 16, and the number of training epochs is 50.

[0037] (2) The steganography party converts the hidden information to be embedded into a bit stream, groups the bit stream according to specific rules, adds corresponding check codes to the end of each group, and obtains a grouped bit stream with check function.

[0038] Specifically, the total length of the hidden information to be embedded in the form of a bit stream is 240 bits. The grouping rule is a fixed-length group with a group length of 60 bits. The verification algorithm uses CRC-4 with a check code length of 4 bits. That is, the corresponding calculated CRC check code (4 bits) is inserted after each sub-bit stream (60 bits).

[0039] Furthermore, it is necessary to calculate the check codes of the four sub-bit streams and insert them sequentially. At this point, the length of the bit stream with check function to be embedded is 4×(60+4)=256 bits.

[0040] (3) The steganography party generates word groups step by step according to the specified initial context, the Chinese language generation model obtained in step (1) and the bit stream obtained in step (2). When generating a word group, it first constructs a word group candidate pool and the probability distribution of each candidate word group, and selects word groups in the candidate pool according to the encoding rules to generate text until the termination condition is met, forming a complete steganography.

[0041] Specifically, based on the given context E and salt value s, the CPM generation model generates candidate pool word groups CP and their corresponding probability distributions P word by word. CP = [word1, word2, ..., word m ], P = [p1, p2, ..., p m ], e i This refers to the i-th word group in the preceding clause. i It is the i-th word group in the candidate pool, p i It is a word i The probability of word selection, i.e.

[0042] Furthermore, the probability distribution described in step (3) is adjusted to adjust the embedding rate (bit / word group) by introducing the hyperparameter τ of the softmax function:

[0043]

[0044] Increasing the hyperparameter τ reduces the variance of the probability distribution of pool words, which can accelerate the convergence speed of arithmetic coding, reduce the number of steganographic words, and ultimately lead to an increase in the steganographic embedding rate (bits / words). Conversely, decreasing the hyperparameter τ leads to a decrease in the steganographic embedding rate (bits / words).

[0045] Furthermore, arithmetic coding is used as the encoding rule, which is an entropy coding method. This ensures that the embedding of hidden information does not cause embedding bias, thus guaranteeing high text quality and high concealment of the steganography.

[0046] Furthermore, since arithmetic coding is a global coding method, the initial arithmetic interval needs to be set as D0 = [lower0, upper0] = [0, 2]. l ], where lower0 is the lower bound of the initial arithmetic interval, upper0 is the upper bound of the initial arithmetic interval, and l is the bit stream b to be embedded with verification function. EMD The number of bits, i.e., D0 = [lower0, upper0] = [0, 2]. 256 As the initial encoding state, the cumulative weight of each word group in the candidate pool

[0047] Furthermore, the word groups in the candidate pool are sequentially... i Determine words based on phrase interval judgment formula i Does it match the encoding status, i.e., word? i The arithmetic interval contains the target value, and the valid conditions for the phrase interval determination formula are as follows:

[0048]

[0049] Where Tar get is the bitstream to be embedded, b EMD The decimal value;

[0050] If the phrase interval is determined to be invalid, then the validity of the next phrase in the candidate pool is checked; if the phrase interval is determined to be valid, then the word is... i As the selected phrase, the arithmetic interval is narrowed to [s(word)]. i-1 ),s(sord i )], and word i Add to text E above.

[0051] Further, the termination condition of the arithmetic coding in the generated steganography is that the current coding interval can be estimated as a complete bit stream, that is, the lower bound of the arithmetic interval is rounded up to be the same as the Target value, that is, ceiling(lower last )=Tar get.

[0052] Embodiment 2:

[0053] This embodiment is used to provide a natural language generation-based Chinese steganography method (based on Huffman coding), as shown in Figure 2 , the method comprises the following steps:

[0054] (1) The steganography party trains a Chinese language generation model through a large-scale corpus;

[0055] Among them, the Chinese language model in step (1) can adopt a large-scale Chinese pre-training model CPM in this embodiment, and the corpus and training details are the same as in embodiment 1.

[0056] (2) The steganography party converts the hidden information to be embedded into a bit stream form, groups the bit stream according to a specific rule, adds a corresponding check code at the end of each group, and obtains a grouped bit stream with a check function;

[0057] Similarly to embodiment 1, the total length of the hidden information to be embedded in the bit stream form is 240 bits, the grouping rule is fixed-length grouping, the grouping length is 60 bits, the check algorithm uses CRC-4, and the check code length is 4 bits, that is, the corresponding CRC check code (4 bits) is inserted after each segment of sub-bit stream (60 bits) is calculated;

[0058] (3) The steganography party gradually generates phrases according to the specified initial context, the Chinese language generation model obtained in step (1), and the bit stream obtained in step (2), and when each phrase is generated, first constructs a phrase candidate pool and a probability distribution of each candidate phrase, and selects a phrase from the candidate pool according to the coding rule to generate text, until the termination condition is reached, and a complete hidden text is formed;

[0059] Specifically, according to the given context E, the salt value s, the candidate pool phrase CP and the corresponding probability distribution P are generated word by word through the CPM generation model;

[0060] Among them CP=[word1,word2,…,word m ], P=[p1,p2,…,p m ], e i represents the i-th phrase in the pre-sentence, word i is the i-th phrase of the candidate pool phrase, and p i is the probability of wordi word probability, i.e.

[0061] Further, the probability distribution of step (3) is adjusted by introducing a hyperparameter τ of softmax function to the embedding rate (bit / word group):

[0062]

[0063] Wherein, the increase of hyperparameter τ reduces the variance of probability distribution of pool word group, which can accelerate the convergence speed of arithmetic coding, reduce the number of word groups of hidden text, and finally lead to the improvement of steganographic embedding rate (bit / word group); on the contrary, the decrease of hyperparameter τ leads to the decrease of steganographic embedding rate (bit / word group).

[0064] Wherein, the coding rule is selected as Huffman coding, which is a kind of entropy coding method. Although the embedding of hidden information will cause embedding bias, Huffman coding is a lightweight and easy-to-implement steganographic coding method, which takes into account the algorithm complexity and the quality of generated text.

[0065] Further, as a local coding method, Huffman coding needs to set the initial remaining bit stream B as the bit stream b with check function to be embedded EMD , each word group word i in the candidate pool is encoded according to its word probability p i to obtain H i .

[0066] Further, the word groups word i of the candidate pool are sequentially encoded according to whether they match the prefix of the remaining bit stream B to determine whether word i matches the coding state. The effective condition of word group interval determination formula is as follows:

[0067]

[0068] If the word group interval determination is invalid, the effectiveness of the next word group in the candidate pool is determined. If the word group interval determination is valid, the word i is selected as the selected word group, and the H i prefix of the original remaining bit stream B is deleted, i.e. B=B[n:-1] and the word i is added to E above.

[0069] Further, the termination condition of Huffman coding in the generated steganography is no remaining bit stream, i.e.

[0070] Embodiment 3:

[0071] This embodiment is used to provide a Chinese steganography system based on natural language generation, such as Figure 3As shown, the system comprises:

[0072] (1) steganography generation module: for generating Chinese steganography embedded with covert information, wherein the Chinese steganography is generated by the Chinese steganography method based on natural language generation of claim 1;

[0073] Specifically, the covert information to be embedded is converted into a bit stream with a total length of 240 according to embodiment 1 or 2, and the check algorithm and grouping rule adopted are the same as those of embodiment 1 or 2, that is, 4-bit CRC check code is added after every 60-bit bit stream;

[0074] Further, the encoding mode can be selected from arithmetic encoding or Huffman encoding and other encoding modes, and the generated steganography is sent to the receiver through a public channel.

[0075] (2) steganography extraction module: for decoding the steganography into a bit stream set by the receiver;

[0076] Specifically, the steganography extraction of module (2) uses the same Chinese language generation model to parse word by word according to the steganography method of module (1), and in each round of candidate pool, selects the word group that can be used as the prefix of the remaining steganography, tries to use each prefix word group as the next word, changes the current encoding state and moves the steganography position backward; the above process is recursive and multi-branch until all branches complete the steganography traversal; finally, all branch decoding results are constructed into a bit stream set;

[0077] Wherein, if all word groups in the candidate pool cannot match the prefix of the remaining steganography during the decoding process, the extraction process can be terminated in advance;

[0078] Optionally, if the encoding mode adopted is Huffman encoding or other local encoding mode, after each 64-bit bit stream is parsed, the verification is carried out, and if the segment verification fails, the extraction process can be terminated in advance.

[0079] (3) steganography check module: for screening the only bit stream that meets the check, the receiver uses the check algorithm used by module (1) to check the bit stream set produced by module (2) one by one to determine the steganography bit stream;

[0080] Specifically, the receiver needs to screen the bit stream (256 bits) with matching length from the set, and the remaining bit streams are regarded as invalid error decoding;

[0081] Further, the receiver needs to check the length-matching bit stream set in turn;

[0082] In which, each bit stream needs to be checked in the unit of sub-bit stream (60+4 bits) divided by packet rule, in fact, only one bit stream can pass all (4) packet checks; finally, the receiver deletes all the check codes embedded in the only valid bit stream as the final extraction result.

[0083] The principles and implementation manners of the present application are described herein by using specific examples, and the above examples are only used for helping to understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the field, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the present description should not be understood as the limitation of the present application.

Claims

1. A Chinese steganography method based on natural language generation, characterized in that, Includes the following steps: (1) The steganography team trains the Chinese language generation model using a large-scale corpus; (2) The steganography party converts the hidden information to be embedded into a bit stream, groups the bit stream according to specific rules, adds corresponding check codes to the end of each group, and obtains a grouped bit stream with check function; (3) The steganography party generates word groups step by step based on the specified initial context, the Chinese language generation model obtained in step (1), and the bit stream obtained in step (2). When generating each word group, it first constructs a word group candidate pool and the probability distribution of each candidate word group, and selects word groups from the candidate pool according to the encoding rules to generate text, until the termination condition is met, forming a complete steganography, where: In the process of generating a single phrase, based on the preceding text E, the salt value s, and the size generated word-by-word by the Chinese language generation model, the following values ​​are used: candidate pool and the probability distribution of each phrase in the candidate pool The salt value 's' is used to enhance the randomness of natural language generation, depending on the encoding method. The phrase that matches the encoded state is selected as the next word in the hidden text, and the phrase is added after updating the previous text E. The resulting new text; The encoding state of local encoding is determined by the remaining bit stream. The way to change the encoding state is to advance the bit stream traversal, and the termination condition is that there is no remaining bit stream. The encoding state of global encoding is determined by the encoding interval. The way to change the encoding state is to shrink the encoding interval, and the termination condition is that the current encoding interval is estimated to be a complete bit stream.

2. The Chinese steganography method based on natural language generation according to claim 1, characterized in that, The selection of Chinese language generation models in step (1) includes language generation models based on attention mechanisms, recurrent neural networks, and Markov chains. The selection of pre-trained Chinese corpora includes essays, social media, and news corpora.

3. The Chinese steganography method based on natural language generation according to claim 1, characterized in that, Step (2) group verification is performed sequentially in the 1st... A checksum is inserted after the segment bit stream; the choice of checksum algorithm includes checksum, XOR check, cyclic redundancy check, message digest algorithm, and secure hash algorithm.

4. The Chinese steganography method based on natural language generation according to claim 1, characterized in that, Step (3) uses either local coding or global coding. Local coding includes complete binary tree coding and Huffman coding, while global coding includes arithmetic coding and interval coding.

5. A Chinese steganography system based on natural language generation, implementing the method as described in claim 1, characterized in that, Includes the following modules: (1) Hidden text generation module: used to generate Chinese hidden text with embedded hidden information, wherein the Chinese hidden text is generated by the Chinese steganography method based on natural language generation as described in claim 1; (2) Hidden text extraction module: used by the receiver to decode the hidden text into a set of bit streams; (3) Hidden text verification module: used to filter out the unique bit stream that meets the verification. The receiver uses the verification algorithm used by module (1) to verify the set of bit streams generated by module (2) one by one to determine the hidden text bit stream.

6. A Chinese steganography system based on natural language generation according to claim 5, characterized in that, Module (2) extracts the hidden text based on the steganography method described in Module (1). It uses the same Chinese language generation model to parse word by word. In each round of the candidate pool, it selects the word groups that are the prefixes of the remaining hidden text and tries to use each prefix word group as the next word. It changes the current encoding state and moves the hidden text position to the right. The above process is recursive and has multiple branches until all branches have completed the hidden text traversal. Finally, it constructs a bit stream set from the decoding results of all branches. The extraction process is based on exchanging the same initial context E, salt value s and received hidden text with the steganography party.

7. A Chinese steganography system based on natural language generation according to claim 5 or 6, characterized in that, The local coding-based group verification mechanism performs both hidden text extraction and verification processes in module (2). When the bit stream with extracted hidden text reaches the current segment length, verification is performed in multiple candidate bit streams. For the bit stream that passes the group verification, the receiver deletes the check code in the bit stream as the hidden text extraction result.

8. A Chinese steganography system based on natural language generation according to claim 5 or 6, characterized in that, The global coding-based block check mechanism requires sequential verification of the bit stream set generated by module (2); each bit stream needs to be checked in groups based on the sub-bit streams divided by the grouping rules; for bit streams that pass all group checks, the receiver deletes the check code in the bit stream as the hidden text extraction result.

Images