A generative steganalysis method across corpora across algorithms

Abstract

The application discloses a cross-corpus cross-algorithm generative text steganalysis method, which comprises the following steps: inputting a text segment into a pre-established and trained text steganalysis detection model to realize discrimination of natural text and steganographic text; the text steganalysis detection model comprises a word importance semantic coding module, a word correlation multi-scale perception module and a ridge regression classification model connected in sequence; the word importance semantic coding module is used for extracting semantic features of the text segment; the word correlation multi-scale perception module is used for acquiring word correlation features from the semantic features; and the ridge regression classification model is used for realizing discrimination of natural text and steganographic text according to the word correlation features. The method of the application can still maintain accurate discrimination of natural text and steganographic text in a small sample scene, and the performance index is higher than that of the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information security and deep learning, and particularly relates to a cross-corpus and cross-algorithm generative text steganalysis method. BACKGROUND

[0002] Steganography is a technique that can embed secret information in digital media such as images, sounds, and texts without causing significant changes. With the help of steganography, the modified carrier can be sent to the receiver through a public channel without arousing the suspicion of the supervisor, and then the receiver can completely reconstruct the secret information from the modified carrier to achieve covert communication. As the main communication medium in daily life, text can be an important information hiding carrier. Text steganography mainly includes two types: modification-based and generative-based. Modification-based text steganography refers to modifying the text format or content based on the existing text carrier to embed secret information, and the embedding capacity is not high. Generative-based text steganography refers to learning the statistical distribution of text words based on a large-scale corpus by means of a language model, and then generating steganographic text according to secret information. This method allows embedding more secret information, thereby causing greater information security threats. Therefore, in order to prevent illegal criminals from using generative text steganography to carry out activities that endanger public information security, it is necessary to develop a generative text steganography analysis algorithm with excellent performance.

[0003] Generative text steganalysis methods are mainly divided into two categories: feature extraction-based methods and deep learning-based methods. The basic idea of the feature extraction-based method is to manually extract text features and then input them into a traditional classifier such as a support vector machine to achieve classification. The disadvantage of this method is that it requires a lot of domain knowledge and the separation of feature extraction and classifier training leads to low detection performance, making it difficult to deal with high-quality steganographic texts generated by emerging generative text steganography methods. The deep learning-based method can automatically extract features through a specific network structure and train in an end-to-end manner. Compared with the feature extraction-based method, the deep learning-based method overcomes the limitations of introducing prior knowledge and achieves better detection results.

[0004] The stego texts generated under the same training corpus and the same steganography algorithm belong to the same stego domain. When the generative text steganography method generates a stego text, firstly, a language model pre-trained in a large corpus is used to obtain a candidate word pool, and then a steganography algorithm is used to establish a mapping relationship between the secret information bit stream and the candidate words, so that the secret information is embedded while ensuring the naturalness of the generated text. In this process, when different training corpora are used, the generated stego texts will have great differences, and in addition, different steganography algorithms will also bring different stego texts. Therefore, the stego text is mainly affected by the training corpus and the steganography algorithm. The existing steganalysis methods based on deep learning mostly need to meet two prerequisites: a large amount of supervised data for training and the test set and the training set belong to the same stego domain. When the amount of supervised data is small (few samples) or the test set and the training set belong to different stego domains (cross-domain), the detection performance of these methods will decrease sharply. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the prior art and provide a generative text steganalysis method that can cross corpora and algorithms.

[0006] In order to achieve the above purpose, the present application provides a generative text steganalysis method that can cross corpora and algorithms, which comprises:

[0007] inputting the text segment into a pre-established and trained text steganography detection model to realize the discrimination of natural text and stego text;

[0008] The text steganography detection model comprises a word importance semantic encoding module, a word correlation multi-scale perception module and a ridge regression classification model connected in sequence; wherein,

[0009] The word importance semantic encoding module is used to extract semantic features of the text segment;

[0010] The word correlation multi-scale perception module is used to obtain word correlation features from the semantic features;

[0011] The ridge regression classification model is used to discriminate natural text and stego text according to the word correlation features.

[0012] As an improvement of the above method, the word importance semantic encoding module comprises a pre-trained language representation model RoBERTa and a word importance mining structure; wherein,

[0013] The pre-trained language representation model RoBERTa comprises a word embedding layer and a stacked 12-layer Transformer;

[0014] The word importance mining structure includes average pooling, maximum pooling and a convolution layer with a kernel size of 2*1.

[0015] As an improvement of the above method, the processing process of the pre-trained language representation model RoBERTa specifically includes:

[0016] For an input text segment T = {t1, t2, …, t j ,…,t len}, where len represents the length of the sentence, t j represents the jth word, the word embedding layer converts the words in T into word vectors, and introduces segment vectors and position vectors to obtain an input matrix E = {e1, e2, …, e j ,…,e len}, where e j represents the embedding vector of the jth word, and E is obtained after passing through a 12-layer Transformer to obtain a text word embedding representation P = {p1, p2, …, p j ,…,p len}, where p j ∈[1,L WE ] is the word embedding representation of the jth word, and L WE is the word embedding length.

[0017] As an improvement of the above method, the processing process of the word importance mining structure specifically includes:

[0018] The average pooling and maximum pooling are used to obtain text information in the text word embedding representation P from different perspectives, and the two are spliced to obtain double-angle text information C. The double-angle text information C is information fused by a convolution kernel to obtain word-level importance information G. The weight coefficient G representing the word importance is obtained by a sigmoid activation function, and the word embedding representation is updated using the weight coefficient G to find the semantic feature V of the stego-sensitive word. The calculation formula is as follows:

[0019] C = [MaxPool(P), AvgPool(P)]

[0020]

[0021] V = P o G

[0022] Where MaxPool and MaxPool are maximum pooling and average pooling operations, Conv f is a convolution layer with a kernel size of (2, 1), V ∈ [len, L SF ], L SF is the word semantic feature length, and o represents matrix point-by-point multiplication.

[0023] As an improvement of the above method, the inter-word correlation multi-scale perception module comprises a feature extraction module with different receptive fields and an attention module with a compression-excitation structure; wherein,

[0024] The feature extraction module is used for extracting inter-word relationship features of different scales from semantic features V of steganographic sensitive words in a targeted manner, and comprises three branches, each branch comprising two-dimensional convolution of a plurality of convolution kernels and maximum pooling; the three branches are then spliced to satisfy the following formula:

[0025] O = [MaxPool(Conv1(V)), MaxPool(Conv2(V)), MaxPool(Conv3(V))]

[0026] Wherein, Conv1, Conv2, Conv3 respectively contain d convolution kernels with sizes (3, L SF ), (4, L SF ), (5, L SF ) respectively;

[0027] The attention module comprises two linear layers with different sizes, uses compression operation to obtain global description of features, and then uses excitation operation to find mutual dependence between different features, to obtain weight values Q representing importance of different features, and then uses dot multiplication operation to realize fusion of weight information and inter-word relationship features O, to obtain multi-scale correlation perception features M, satisfying the following formula:

[0028]

[0029] M = Q O

[0030] Wherein, and are weight vectors and bias of the two linear layers respectively, sigmoid represents excitation operation, Q e (3d), M e (3d).

[0031] As an improvement of the above method, the method further comprises a training step of a text steganographic detection model; specifically comprising:

[0032] Step 1) Divide the data set, and divide the steganographic texts generated by using different training corpus or different steganographic algorithms with different embedding rates into different categories, and divide the labeled data into source domain metadata set S source , and the unlabeled data into target domain data set S target .

[0033] Step 2) Using meta-learning strategy, the source domain metadata set S source and the target domain data set S target obtained in step 1) are trained to obtain a text steganographic detection model.source source set query set and their corresponding labels are y s , y q , respectively target target set

[0034] Step 3) The text segment set obtained in step 2) is input into a word importance semantic encoding module to obtain semantic features n , n s , n q and n t , respectively

[0035] Step 4) The text features n s , n q and n t obtained in step 3) are input into an inter-word association multi-scale perception module to obtain inter-word association features m s , m q and m t , respectively

[0036] Step 5) The text features m s , m q and m t obtained in step 4) are input into a ridge regression model trained using the inter-word association features m s , and the ridge regression model is used to predict the inter-word association features m q to obtain a classification loss The maximum mean difference and conditional mean difference of m q and m t are calculated to obtain a domain adaptation loss

[0037] Step 6) The classification loss and the domain adaptation loss obtained in step 5) are added together to obtain a total loss The model parameters are updated according to the total loss , and the iteration is repeated until the optimal parameter combination is trained

[0038] Step 7) A trained text steganographic detection model is obtained based on the optimal parameter combination.

[0039] As an improvement of the above method, the step 2) specifically comprises:

[0040] Step 2-1) Extract N categories from the source domain meta-training set, N is less than the total number of categories in the source domain meta-data set, and the extraction needs to ensure that both natural text and steganographic text are included, K samples are extracted for each category to form a support set, and Q samples are extracted to form a query set

[0041] Step 2-2) Extract N categories from the target domain dataset, N is less than the total number of categories in the target domain dataset, and ensure that natural text and steganographic text are included during extraction, and extract Q samples from each category to form a target set;

[0042] Step 2-3) For the support set, the query set and the target set, the maximum text length in each set is k s ,k q ,k t , the text segments in the support set, the query set and the target set after padding are k s ,k q ,k t .

[0043] As an improvement of the above method, the step 5) specifically comprises:

[0044] Step 5-1) Use the word association feature m s and the corresponding label information to train a ridge regression classification model, and the training target is:

[0045]

[0046] Where y s is the label of the support set, w train is the weight vector of the ridge regression model, λ is the penalty coefficient of L2 regularization, and min represents the minimum value;

[0047] Step 5-2) When the ridge regression classification model is trained, the closed-form solution w train is obtained by using the following matrix operation:

[0048] w train = (m s Tm s + λI) -1 m s T y s

[0049] Where I is the unit vector matrix;

[0050] Step 5-3) Use the trained ridge regression classification model to predict the word association feature m q extracted from the query set, and use cross-entropy to calculate the classification loss

[0051]

[0052] ​

[0053] wherein, a and b are learnable parameters in meta-learning process, softmax is an activation function, y q is the text label of the query set, is the prediction result of the query set text by the ridge regression classification model;

[0054] Step 5-4) calculate the inter-word correlation feature m s and the domain adaptation loss is obtained by the distribution difference between m q and m

[0055]

[0056] wherein, f MMD (m q , m t ) is the maximum mean difference between m q and m t , and f CMD (m q , m t ) is the conditional mean difference between m q and m t .

[0057] As an improvement of the above method, the step 6) specifically comprises:

[0058] Step 6-1) add the classification loss and the domain adaptation loss to obtain the total loss

[0059]

[0060] Step 6-2) use the parameters obtained in step 6-1) as the weight value of this iteration; obtain a new parameter combination through steps 2), 3), 4) and 5-1); repeat the iteration until a cycle of iteration is completed;

[0061] Step 6-3) reset the order of the training text, go to step 1); repeat the execution until the optimal parameter combination is trained.

[0062] As an improvement of the above method, the step 7) specifically comprises:

[0063] Step 7-1) extract K samples from the target domain data set S target to form a target domain support set v s , and the corresponding label is extract Q samples to form a target domain query set v q ;

[0064] Step 7-2) Extract text features by using the trained word importance semantic encoding module and the inter-word association multi-scale perception module With

[0065] Step 7-3) Use the text feature n s Train the ridge regression classification model, and predict the text feature n q .

[0066] Step 7-4) Repeat steps 7-1) to 7-3) until a ridge regression classification model that meets the training requirements is obtained, thereby obtaining a trained text steganographic detection model.

[0067] Compared with the prior art, the advantages of the present application are:

[0068] The generative text steganographic analysis technical solution provided by the present application mines rich semantic information of text through a word importance semantic encoding module. First, a language representation model RoBERTa pre-trained in a large-scale corpus is used to extract word embedding representations containing rich semantic information. Then, a word importance mining structure is used to extract text features with higher relevance to steganographic analysis tasks. The inter-word association multi-scale perception module focuses on the changes in the inter-word relationship between adjacent words and non-adjacent words caused by steganography. First, a feature extraction module with different receptive fields is used to extract word association features of different scales. Then, an attention mechanism with a compression-excitation structure is used to obtain a global description of the word features. The distribution distance of the source domain and target domain text features extracted by the above modules is calculated and used as a domain adaptation loss to help the network extract domain-invariant features. The network is trained using a meta-learning strategy to maintain accurate discrimination between natural text and steganographic text in a small sample scenario. When testing the benchmark data set, the present application achieves better performance than existing methods. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 The generative text steganographic analysis method provided by the present application has a cross-corpus and cross-algorithm overall structure and training process schematic diagram.

[0070] Figure 2 The word importance semantic encoding module in the generative text steganographic analysis method provided by the present application has a schematic diagram.

[0071] Figure 3 The inter-word association multi-scale perception module in the generative text steganographic analysis method provided by the present application has a schematic diagram. DETAILED DESCRIPTION

[0072] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings and examples.

[0073] Example

[0074] Figure 1 The figure shows the overall structure and training process of the generative text steganalysis method that is cross-corpus and cross-algorithm, as provided by this invention. The proposed generative text steganalysis method consists of a word importance semantic encoding module, a multi-scale word association perception module, a distribution distance metric, and a meta-learning training strategy. In the figure, "→" indicates parameter passing, "⊕" indicates addition of two numbers, and "⊙" indicates pointwise matrix multiplication. The data processing of this method specifically includes:

[0075] Step 1) Divide the dataset. The steganographic texts generated using different steganography algorithms and different embedding rates on language models trained on different training corpora will be divided into different categories. The source domain metadata S consists of labeled data. source The target domain dataset S consists of unlabeled data. target .

[0076] Step 2) Apply the source domain metadata S obtained in Step 1) source With the target domain dataset S target The N-way, K-shot, and Q-query meta-learning strategies are employed to learn from the source domain meta-dataset S. source Extracting support sets query set Their corresponding labels are y s y q From the target domain dataset S target Extracting target set

[0077] Step 2) specifically includes:

[0078] Step 2-1) Extract N categories from the source domain metadata training set (N is less than the total number of categories in the source domain metadata set). When extracting, it is necessary to ensure that both natural text and steganographic text are included. Extract K samples from each category without repetition to form a support set, and extract Q samples to form a query set.

[0079] Step 2-2) Extract N categories from the target domain dataset (N is less than the total number of categories in the target domain dataset). When extracting, it is necessary to ensure that both natural text and steganographic text are included. Extract Q samples from each category to form the target set.

[0080] Steps 2-3) For the support set, query set, and target set, let the maximum text length in each set be k respectively. s ,k q ,k tText fragments within the set whose length is less than the maximum text length are padded. After normalization, the lengths of text fragments in the support set, query set, and target set are k, respectively. s ,k q ,k t .

[0081] Step 3) Process the set of text fragments obtained in Step 2). Semantic features n are obtained by passing them through the word importance semantic encoding module. s n q With n t .

[0082] Step 4) Process the text features n obtained in Step 3) s n q With n t Inter-word association features m are obtained through the multi-scale perception module for inter-word association. s m q With m t .

[0083] Step 5) Process the text features m obtained in Step 4) s m q With m t Using inter-word association features m s Train a ridge regression model, and use the trained ridge regression model to analyze the word association features m. q Make a prediction and obtain the classification loss. Calculate m q With m t The domain adaptation loss is obtained by comparing the maximum mean difference with the conditional mean difference.

[0084] Step 5) specifically includes:

[0085] Step 5-1) Utilize the inter-word association feature m s The ridge regression classification model is trained using its corresponding annotation information, with the following training objective.

[0086]

[0087] Among them, y s To support set tags, w train Let λ be the weight vector of the ridge regression model, and λ be the penalty coefficient for L2 regularization.

[0088] After the ridge regression classification model described in step 5-2) is trained, its closed-form solution is obtained using the following matrix operations.

[0089] w tain =(m s T ms + λI) -1 m s T y s

[0090] where I is the identity matrix.

[0091] Step 5-3) Use the trained ridge regression classification model to predict the inter-word association feature m q from the query set, and use cross-entropy to calculate the classification loss

[0092]

[0093]

[0094] where α and β are learnable parameters in the meta-learning process.

[0095] Step 5-4) Use the maximum mean difference and conditional mean difference to calculate the distribution difference between the inter-word association feature m s and m q to obtain the domain adaptation loss

[0096]

[0097]

[0098]

[0099]

[0100] where H is a reproducing kernel Hilbert space, φ(·) is a mapping function that maps the original feature space to RKHS, is the Gram matrix of the reproducing space, is the regularization matrix, are the hidden feature matrices obtained by mapping m q , m t , are the hidden feature matrices obtained by mapping the labels of m q , m t

[0101] Step 6) Add the classification loss and the domain adaptation loss obtained in step 5) to obtain the total loss Update the model parameters according to the total loss , and repeat the iteration until the optimal parameter combination is trained.

[0102] ​The step 6) specifically comprises:

[0103] The step 6-1) adds the classification loss and the domain adaptation loss to obtain a total loss

[0104]

[0105] The step 6-2) uses the parameters obtained in the step 6-1) as the weight values of this iteration; obtains a new parameter combination through the steps 2), 3), 4) and 5-1); and iterates repeatedly until a period of iteration is completed.

[0106] The step 6-3) resets the order of the training text, and goes to the step 1); and iterates repeatedly until the optimal parameter combination is trained.

[0107] The step 7) implements the generative text information hiding detection based on the optimal parameter model.

[0108] The step 7) specifically comprises:

[0109] The step 7-1) extracts K samples from the target domain data set S target to form a target domain support set v s , and the corresponding label is extracts Q samples to form a target domain query set v q .

[0110] The step 7-2) extracts text features and

[0111]

[0112]

[0113] wherein, represents the trained word importance semantic encoding module, and represents the trained inter-word association multi-scale perception module.

[0114] The step 7-3) trains a ridge regression classification model using the text features n s , and uses the trained ridge regression model to predict the text features n q .

[0115]

[0116]

[0117]

[0118] wherein, alpha * and beta * are the trained learnable parameters, and w test is the ridge regression model parameter.

[0119] Step 7-4) repeatedly performing step 7-1), step 7-2), and step 7-3) to obtain the detection accuracy on the target domain data set;

[0120] As Figure 2 shown, the word importance semantic encoding module proposed by the application is composed of a pre-trained language model RoBERTa and a word importance mining structure. In the figure, "⊙" represents matrix point-by-point multiplication, and "sigma" represents an activation function.

[0121] The step 3) specifically comprises:

[0122] Step 3-1) the pre-trained language representation model RoBERTa is composed of a word embedding layer and a 12-layer Transformer stack, and the RoBERTa can be used to extract rich semantic information contained in the input text segment. Given an input text segment T={t1,t2,…t j ,…,t len}, where len represents the length of the sentence, and t j represents the jth word. The word encoding layer is used to convert the words in T into word vectors, and a segment vector and a position vector are introduced to obtain an input matrix E={e1,e2,…,e j ,…,e len}, where e j represents the embedding vector of the jth word. After E passes through RoBERTa, the text word embedding representation P={p1,p2,…,p j ,…,p len} is obtained, where p j ∈[1,L WE ] is the word embedding representation of the jth word, and L WE is the word embedding length.

[0123] Step 3-2) the word importance mining structure includes average pooling, maximum pooling, and a convolution layer with a convolution kernel size of 2*1. The average pooling and maximum pooling are used to obtain text information contained in the text word embedding representation P from different perspectives, and the two are spliced to obtain double-angle text information C. The convolution kernel is used to fuse the information to obtain word-level importance information G. The weight coefficient G representing the word importance is obtained through the sigmoid activation function, and the word embedding representation is updated using the weight coefficient G. The semantic features V of the stego-sensitive words are found out, and the calculation formula is as follows

[0124] C = [MaxPool(P), AvgPool(P)]

[0125]

[0126] V=P⊙G

[0127] Where MaxPool and MaxPool represent the max pooling and average pooling operations, respectively, Conv f For a convolutional layer with kernel size (2,1), V∈[len,L] SF ], L SF The length of the semantic features of the word.

[0128] like Figure 3 As shown, the inter-word association multi-scale perception module proposed in this invention consists of a feature extraction module with different receptive fields and an attention module with a compression-excitation structure; in the figure, "⊙" represents matrix multiplication point by point.

[0129] Step 4) specifically includes:

[0130] Step 4-1) Utilize the feature extraction modules with different receptive fields to selectively extract inter-word relationship features at different scales from the semantic features V of steganalytically sensitive words. Then, perform max pooling operations on each feature and concatenate them to obtain inter-word relationship features O. This enables the perception of inter-word relationship information at different scales as a whole and improves network computational efficiency. The specific calculation is as follows.

[0131] O=[MaxPool(Conv1(V)),MaxPool(Conv2(V)),MaxPool(Conv3(V))]

[0132] Among them, Conv1, Conv2, and Conv3 each contain d elements of size (3, L). SF ),(4,L SF ),(5,L SF ) convolution kernel.

[0133] Step 4-2) describes an attention module with a compression-excitation structure. This module consists of two linear layers of unequal size. Compression operations are used to obtain a global description of the features, and excitation operations are used to find the interdependencies between different features, obtaining weight values ​​Q that represent the importance of different features. Finally, a dot product operation is used to fuse the weight information with the inter-word relationship features O to obtain the multi-scale association perception feature M.

[0134]

[0135] M=Q⊙O

[0136] in, and are the weight vector and bias of the two linear layers in the autoencoder, respectively, Q e (3d), M e (3d).

[0137] Finally, it should be noted that the above examples are merely intended to illustrate the technical solutions of the present application but not to limit the present application. Although the present application is described in detail with reference to the examples, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of these modifications and equivalent replacements should be included in the scope of the claims of the present application.

Claims

1. A generative text steganalysis method that is cross-corpus and cross-algorithm, the method comprising: Text fragments are input into a pre-built and trained text steganalysis detection model to distinguish between natural text and steganalysis. The text steganography detection model comprises a word importance semantic encoding module, a multi-scale word association perception module, and a ridge regression classification model connected in sequence; wherein... The word importance semantic encoding module is used to extract semantic features from text fragments; The inter-word association multi-scale perception module is used to obtain inter-word association features from semantic features; The ridge regression classification model is used to distinguish between natural text and steganographic text based on inter-word association features; The inter-word association multi-scale perception module includes a feature extraction module with different receptive fields and an attention module with a compression-excitation structure; wherein... The feature extraction module is used to selectively extract semantic features from steganographic sensitive words. The text describes a process to extract inter-word relationship features at different scales, including three branches. Each branch consists of two-dimensional convolutions with several kernels and max pooling. The three branches are then concatenated to satisfy the following equation: ； in, Each contains The sizes are respectively , , convolution kernel, The length of the semantic features of the word; The attention module comprises two linear layers of unequal size. It uses a compression operation to obtain a global description of the features, and then uses an activation operation to find the interdependencies between different features, thereby obtaining weight values ​​that represent the importance of different features. Weight information and word relationship features are realized through dot product operation. The fusion of these features yields multi-scale correlated perception features. The following equation is satisfied: ； ； in, , and , These are the weight vectors and biases of two linear layers, respectively. , , ⊙ represents the pointwise multiplication of matrices.

2. The generative text steganography analysis method that is cross-corpus and cross-algorithm according to claim 1, characterized in that, The word importance semantic encoding module includes a pre-trained language representation model RoBERTa and a word importance mining structure; wherein... The pre-trained language representation model RoBERTa includes word embedding layers and stacked 12 Transformer layers; The word importance mining structure includes average pooling, max pooling, and convolutional layers with a kernel size of 2*1.

3. The generative text steganography analysis method that is cross-corpus and cross-algorithm according to claim 2, characterized in that, The processing of the pre-trained language representation model RoBERTa specifically includes: For input text fragments ,in Represents sentence length. Representing the Each word, the word embedding layer will Words are converted into word vectors, and segment vectors and position vectors are introduced to obtain the input matrix. ,in, Representing the Embedding vectors of words, The text word embedding representation is obtained after passing through 12 layers of Transformer. ,in, It is the first Word embedding representation of a word, The word embedding length.

4. The generative text steganography method that is cross-corpus and cross-algorithm according to claim 3, characterized in that, The processing steps of the word importance mining structure specifically include: Obtaining text word embedding representations from different perspectives using average pooling and max pooling The text information in the two parts is concatenated to obtain the dual-angle text information. The convolution kernel processes the text information from two angles. Information fusion is performed to obtain word-level importance information. The weight coefficients representing word importance are obtained through the sigmoid activation function. It then uses this information to update the word embedding representation and finds the semantic features of steganography-sensitive words. The calculation formula is as follows: ； ； ； in, and These are max pooling and average pooling operations, respectively. For a convolutional layer with a kernel size of (2,1), , is the length of the semantic features of the word, and ⊙ represents the pointwise multiplication of the matrix.

5. The generative text steganography method that is cross-corpus and cross-algorithm according to claim 4, characterized in that, The method also includes a training step for a text steganalysis detection model; specifically including: Step 1) Divide the dataset: Divide the steganographic texts generated using different training corpora or different steganography algorithms with different embedding rates into different categories. The labeled data constitutes the source domain meta-dataset. Unlabeled data constitutes the target domain dataset. ; Step 2) Process the source domain metadata obtained in Step 1). With the target domain dataset A meta-learning strategy is adopted to learn from the source domain meta-dataset Extracting support sets Query set Their corresponding tags are respectively , From the target domain dataset Extracting target set ; Step 3) The set of text fragments obtained in Step 2) , , Semantic features are obtained by passing them through the word importance semantic encoding module. , and ; Step 4) Analyze the text features obtained in Step 3). , and Inter-word association features were obtained through the multi-scale inter-word association perception module. , and ; Step 5) Analyze the text features obtained in Step 4). , and Utilizing inter-word association features Train a ridge regression model and use the trained ridge regression model to analyze word association features. Make a prediction and obtain the classification loss. ,calculate and The domain adaptation loss is obtained by comparing the maximum mean difference with the conditional mean difference. ; Step 6) Apply the classification loss obtained in Step 5) Domain Adaptation Loss Adding the two together gives the total loss. According to the total loss Update the model parameters and iterate repeatedly until the optimal parameter combination is trained; Step 7) Obtain the trained text steganalysis detection model based on the optimal parameter combination.

6. The generative text steganography method that is cross-corpus and cross-algorithm according to claim 5, characterized in that, Step 2) specifically includes: Step 2-1) Extract N categories from the source domain metadata training set. N is less than the total number of categories in the source domain metadata training set. When extracting, it is necessary to ensure that both natural text and steganographic text are included. Extract K samples from each category without repetition to form the support set, and extract Q samples to form the query set. Step 2-2) Extract N categories from the target domain dataset, where N is less than the total number of categories in the target domain dataset. During extraction, it is necessary to ensure that both natural text and steg text are included. Extract Q samples from each category to form the target set. Steps 2-3) For the support set, query set, and target set, let the maximum text length in each set be respectively... , , Text fragments within the set whose length is less than the maximum text length are padded. After normalization, the lengths of text fragments in the support set, query set, and target set are respectively... , , .

7. The generative text steganography method that is cross-corpus and cross-algorithm according to claim 6, characterized in that, Step 5) specifically includes: Step 5-1) Utilize inter-word association features The ridge regression classification model is trained using its corresponding labeled information, and the training objective is... for: ； in, To support set tags, For the ridge regression model weight vector, for The penalty coefficient for regularization, This indicates taking the minimum value; Step 5-2) After the ridge regression classification model is trained, use the following matrix operations to obtain its closed-form solution. : ； in, It is a unit vector matrix; Step 5-3) Use the trained ridge regression classification model to analyze the word association features extracted from the query set. Make predictions and use cross-entropy to calculate the classification loss. : ； ； in, and These are the learnable parameters during the meta-learning process. For activation function, To query the set of text tags, The ridge regression classification model predicts the text of the query set. Step 5-4) Calculate inter-word association features and Domain adaptation loss is obtained by analyzing the differences in distribution between regions. : ； in, for The maximum mean difference between them. for The difference in conditional mean between them.

8. The generative text steganography method that is cross-corpus and cross-algorithm according to claim 7, characterized in that, Step 6) specifically includes: Step 6-1) The classification loss Domain Adaptation Loss Add them together to get the total loss : ； Step 6-2) Use the parameters obtained in Step 6-1) as the weight values ​​for this iteration; through Step 2), Step 3), Step 4) and Step 5-1), obtain a new combination of parameters; repeat the iteration until one iteration cycle is completed; Step 6-3) Reset the order of the training texts and go back to step 1); repeat this process until the optimal parameter combination is trained.

9. The generative text steganography method that is cross-corpus and cross-algorithm according to claim 8, characterized in that, Step 7) specifically includes: Step 7-1) From the target domain dataset K samples are extracted from the target domain to form the support set. The corresponding tag is Extract Q samples to form the target domain query set. ; Step 7-2) Extract text features using the trained word importance semantic encoding module and inter-word association multi-scale perception module. and ; Step 7-3) Utilizing text features Training a ridge regression classification model for text features Make predictions; Step 7-4) Repeat steps 7-1) to 7-3) until a ridge regression classification model that meets the training requirements is obtained, thus obtaining a well-trained text steganalysis detection model.

Images