A multi-class natural language steganalysis method
By building private and shared convolutional neural networks, the deep multi-task steganography analysis feature representation is solved, and the problem that targeted methods in the existing technology cannot detect multi-class steganography text is improved, and the generalization ability and detection accuracy of the model are improved. .
Patent Information
- Application Number
- CN202210264113.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-17
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-03-17
AI Technical Summary
The existing natural language steganography analysis methods are mainly targeted, and cannot effectively detect multiple steganography texts, and have poor generalization capabilities and cannot be generalized to other steganography text categories.
Using a multi-class natural language steganography analysis method, the steganography analysis task data set is represented as a word vector matrix, and a private convolutional neural network and a shared convolutional neural network are built to obtain the feature representation of deep multi-task steganography analysis, and use multiple Task learning trains the model to improve the generalization ability of the model.
It realizes simultaneous detection of multiple steganographic texts, improves model performance and generalization capabilities, enhances the detection accuracy of different types of steganographic texts, and alleviates the problem of data sparseness.
Smart Images

Figure CN114462382B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of encryption technology, and in particular to a multi-class natural language steganalysis method. Background Art
[0002] Natural language information hiding technology is an information security technology that embeds secret information into a natural language carrier (text) so that it cannot be detected by third parties other than the sender and the receiver, in order to achieve the purpose of hiding secret information. However, everything has two sides. While providing information security, natural language information hiding technology has inevitably become a tool for criminals. Because of this, natural language steganalysis technology, a countermeasure technology for natural language information hiding technology, has received attention. Natural language steganalysis technology analyzes natural language texts to determine whether there is hidden information in natural language texts.
[0003] At present, there are two mainstream directions of natural language information hiding technology, one is embedded steganography and the other is generative steganography. Among them, embedded steganography is to embed hidden information by modifying the syntax, vocabulary, semantics and other aspects of the existing normal text, while ensuring that the local and global semantics and statistics of the text remain unchanged, the semantics are correct, and the syntactic structure is reasonable. The generative steganography method, with the rapid development of deep learning in the field of natural language processing, can generate high-quality readable text with the help of the powerful feature learning ability of neural networks, and generative steganography is to embed hidden information through encoding during the text generation process. However, the detection research of natural language steganographic text, that is, the research on natural language steganalysis, is mainly targeted at type methods (that is, steganalysis is performed on normal text and a corresponding type of steganographic text alone), which are all classifications of normal text and a corresponding type of steganographic text.
[0004] In actual scenarios, there should be a large number of natural texts, normal texts (texts generated by generative steganography without embedded secret information), generative stegotexts, and embedded stegotexts coexisting together. Targeted methods have poor generalization ability and cannot be extended to the classification of other types of stegotexts.
[0005] To this end, the present invention provides a multi-class natural language steganalysis method, which can simultaneously perform steganalysis on multiple types of steganographic texts, thereby improving model performance and also improving the generalization ability of the steganalysis model. Summary of the invention
[0006] In order to achieve the purpose of the present invention, the following technical solutions are adopted:
[0007] A multi-class natural language steganalysis method comprises the following steps: step 1. representing a steganalysis task dataset as a word vector matrix; step 2. obtaining a deep multi-task steganalysis feature representation of a text; step 3. training a multi-task steganalysis model; step 4. performing steganalysis on a test set text using the optimal multi-task steganalysis model; wherein step 1 comprises: 1.1 defining a plurality of steganalysis tasks, namely: (1) a generative steganalysis task of steganalysis of a normal text; (2) a generative steganalysis task of steganalysis of a natural text; (3) a steganalysis task of steganalysis of an embedded steganalysis text and a normal text; (4) a steganalysis task of steganalysis of an embedded steganalysis text and a natural text; (5) a normal text and a natural text classification task; 1.2 representing the text in the dataset of each steganalysis task by a word vector matrix.
[0008] The multi-class natural language steganalysis method, wherein step 1.2 includes: obtaining each steganalysis task data set, dividing the data set into a training set and a test set, the data set includes a large number of classified texts, the texts are sentences; performing word segmentation and preprocessing on the texts in the data set to obtain a word vector matrix of the texts in the data set, as follows:
[0009]
[0010]
[0011] Where D k Indicates that task k contains N k A dataset of texts, and Represents a text representation vector matrix and the corresponding classification label in task k, V represents the text The word vector of a word in , l represents the length of the text, d represents the dimension of the vector; k is a natural number between 1 and 5.
[0012] The multi-class natural language steganalysis method, wherein step 2 comprises:
[0013] 2.1 Build a separate convolutional neural network for each steganalysis task, called a private convolutional neural network, to obtain the unique feature representation of each steganalysis task, as follows:
[0014] (2.1.1) Randomly extract a text from a steganalysis task training set, input the word vector matrix of this text into the private convolutional neural network of the steganalysis task to which the text belongs, use multiple convolution kernel matrices of different widths to extract feature information simultaneously, and generate candidate feature representations:
[0015] c t =f(W·x t:t+h-1 +b)
[0016] Where W is a convolution kernel matrix; h is the width of the convolution kernel matrix, b is the bias term; f is a nonlinear activation function; x represents the word vector matrix of the selected text;
[0017] (2.1.2) For the features obtained by different convolution kernel matrices, a maximum pooling operation is performed, that is, the maximum value of each candidate feature representation is taken to obtain the maximum pooling feature:
[0018]
[0019] (2.1.3) Connect the maximum pooling features obtained by different convolution kernel matrices together to obtain the private feature representation of the text, which is specifically expressed as:
[0020]
[0021] Where m is the number of convolution kernel matrices;
[0022] 2.2 Build the same convolutional neural network for all tasks, called a shared convolutional neural network, to capture shared feature representations that are less relevant to a specific steganalysis task, as follows:
[0023] (2.2.1) In step 2.1.1, while the word vector matrix of the randomly extracted text is input into the private convolutional neural network of the steganalysis task to which the text belongs, the word vector matrix of the text is input into the pre-built shared convolutional neural network, and multiple convolution kernel matrices of different widths are used to simultaneously extract feature information to generate candidate feature representations:
[0024] s t =f(W·x t:t+h-1 +b)
[0025] Where W is a convolution kernel matrix; h is the width of the convolution kernel matrix, b is the bias term; f is a nonlinear activation function; x represents the word vector matrix of the extracted text;
[0026] (2.2.2) For the features obtained by different convolution kernel matrices, a maximum pooling operation is performed, that is, each candidate feature representation takes the maximum value to obtain the maximum pooling feature:
[0027]
[0028] (2.2.3) Connect the maximum pooling features obtained by different convolution kernel matrices together to obtain the shared feature representation of the text, which is specifically expressed as:
[0029]
[0030] Where m is the number of convolution kernel matrices;
[0031] 2.3 The feature representation of the private convolutional neural network output of the text and the feature representation of the shared convolutional neural network output are spliced together to obtain the deep multi-task steganalysis text representation vector of the text:
[0032] H=(z·o).
[0033] The multi-class natural language steganalysis method, wherein step 3 comprises:
[0034] The deep multi-task steganalysis text representation vector of the randomly extracted text obtained in step 2 is input into the softmax layer of the task to which the text belongs. The formula is expressed as follows:
[0035]
[0036] The multi-task steganalysis model is trained using the training set texts of all steganalysis tasks, the network parameters are updated using back propagation, and the loss function is minimized through iterative optimization of the model to obtain the optimal multi-task steganalysis model;
[0037] The loss function is defined as follows:
[0038]
[0039] Where K represents the number of steganalysis tasks, N k represents the number of samples in the k-th steganalysis task dataset, β k represents the weight of the k-th steganalysis task, represents the true label of the i-th sample in the k-th steganalysis task, It represents the probability that the i-th sample in the k-th steganalysis task contains stegographic information.
[0040] The multi-class natural language steganalysis method, wherein step 4 comprises:
[0041] Input the test set text into the optimal multi-task steganalysis model and output the predicted label Set a detection threshold α, and the final analysis results are as follows:
[0042]
[0043] At this time, task k belongs to the task of generating steganalysis of steganalysis text and normal text;
[0044]
[0045] At this time, task k belongs to the task of generating steganalysis of steganalysis text and natural text;
[0046]
[0047] At this time, task k belongs to the embedded steganalysis task of steganalysis of steganalysis text and normal text;
[0048]
[0049] At this time, task k belongs to the embedded steganalysis task of steganalysis of steganalysis text and natural text;
[0050]
[0051] At this time, task k belongs to the natural text and normal text classification task. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a flowchart of a multi-class natural language steganalysis method;
[0053] Figure 2 Schematic diagram of the model framework of multiple natural language steganalysis methods. DETAILED DESCRIPTION
[0054] The following is combined with Figure 1-2 , the specific implementation methods of the present invention are described in detail.
[0055] like Figure 1 As shown, the multi-class natural language steganalysis method includes the following steps:
[0056] Step 1. Represent the steganalysis task dataset as a word embedding matrix
[0057] 1.1 Five steganalysis tasks are defined, namely: (1) generative steganalysis task of steganalysis between steganalysis text and normal text; (2) generative steganalysis task of steganalysis between steganalysis text and natural text; (3) embedded steganalysis task of steganalysis text and normal text; (4) embedded steganalysis task of steganalysis text and natural text; (5) normal text and natural text classification task.
[0058] 1.2 Represent the text in the dataset of each steganalysis task through a word vector matrix, including:
[0059] A data set for each steganalysis task is obtained, and the data set is divided into a training set and a test set. The data set includes a large amount (eg, more than 10,000) of classified texts, and the texts are sentences.
[0060] The classification is exemplified as follows: for steganalysis task (1), i.e., generative steganalysis and normal text steganalysis task, each text in the task dataset has been classified and assigned a classification label, which is whether the text is generative steganalysis or normal text. The classification for other steganalysis tasks is similar and will not be described one by one.
[0061] The text in the dataset is segmented and preprocessed to obtain the text representation vector of the dataset: for example, the Google News 300-dimensional word vector file obtained by Word2vec training is used as the vocabulary, and the corresponding word vector of each word in each text in the dataset is obtained through the vocabulary. The word vector matrix is generated according to the word sequence of each steganalysis task text, which is expressed as follows:
[0062]
[0063]
[0064] Where D k Indicates that task k contains N k A dataset of texts, and Represents the word vector matrix and corresponding classification label of a text in task k, V represents the text The word vector of a word in , l represents the length of the text, d represents the dimension of the vector; k is a natural number between 1 and 5.
[0065] Step 2. Deep multi-task steganalysis feature representation acquisition of text
[0066] 2.1 Construct a separate convolutional neural network (called a private convolutional neural network) for each steganalysis task to obtain the unique feature representation of each steganalysis task. The process of obtaining the feature representation is as follows:
[0067] (2.1.1) Randomly extract a text from a steganalysis task training set, input the word vector matrix of this text into the private convolutional neural network of the steganalysis task to which the text belongs, use multiple convolution kernel matrices of different widths to extract feature information simultaneously, and generate candidate feature representations:
[0068] c t =f(W·x t:t+h-1 +b)
[0069] Where W is a convolution kernel matrix; h is the width of the convolution kernel matrix, b is the bias term; f is a nonlinear activation function; x represents the word vector matrix of the selected text.
[0070] (2.1.2) For the features obtained by different convolution kernel matrices, a maximum pooling operation is performed, that is, the maximum value of each candidate feature representation is taken to obtain the maximum pooling feature:
[0071]
[0072] (2.1.3) Connect the maximum pooling features obtained by different convolution kernel matrices together to obtain the private feature representation of the text, which is specifically expressed as:
[0073]
[0074] Where m is the number of convolution kernel matrices. The total number of z is
[0075] 2.2 Build the same convolutional neural network for all tasks (called a shared convolutional neural network) to capture shared feature representations that are less relevant to the specific steganalysis task in step 2.1, thereby obtaining information that cannot be captured by private convolutional neural networks, as follows:
[0076] (2.2.1) In step 2.1.1, while the word vector matrix of the randomly extracted text is input into the private convolutional neural network of the steganalysis task to which the text belongs, the word vector matrix of the text is input into the pre-built shared convolutional neural network, and multiple convolution kernel matrices of different widths are used to simultaneously extract feature information to generate candidate feature representations:
[0077] s t =f(W·x t:t+h-1 +b)
[0078] Where W is a convolution kernel matrix; h is the width of the convolution kernel matrix, b is the bias term; f is a nonlinear activation function; x represents the word vector matrix of the extracted text.
[0079] (2.2.2) For the features obtained by different convolution kernel matrices, a maximum pooling operation is performed, that is, each candidate feature representation takes the maximum value to obtain the maximum pooling feature:
[0080]
[0081] (2.2.3) Connect the maximum pooling features obtained from different convolution kernel matrices together to obtain the shared feature representation of each steganalysis task, which is specifically expressed as:
[0082]
[0083] Where m is the number of convolution kernel matrices. The total number of o is
[0084] 2.3 The feature representation of the private convolutional neural network output of the text and the feature representation of the shared convolutional neural network output are spliced together to obtain the deep multi-task steganalysis text representation vector of the text. The formula is expressed as:
[0085] H=(z·o)
[0086] Step 3. Training of multi-task steganalysis model
[0087] The deep multi-task steganalysis text representation vector of the randomly extracted text obtained in step 2 is input into the softmax layer of the task to which the text belongs. The formula is expressed as follows:
[0088]
[0089] The final output value It reflects the probability that the model believes that the text represented by the text representation vector of the current text contains hidden information. The predicted label given to the current text by the multi-task steganalysis model.
[0090] The multi-task steganalysis model is trained using the training set text of all steganalysis tasks. This patent follows the supervised learning framework, uses back propagation to update network parameters, and minimizes the loss function through iterative optimization of the model to obtain the optimal multi-task steganalysis model. The loss function is defined as follows:
[0091]
[0092] Where K represents the number of steganalysis tasks, N k represents the number of samples in the k-th steganalysis task dataset, β k represents the weight of the k-th steganalysis task, represents the true label of the i-th sample in the k-th steganalysis task, It represents the probability that the i-th sample in the k-th steganalysis task contains stegographic information.
[0093] The loss function calculates the average cross entropy between the predicted label and the true label as the prediction error. Through model training, the prediction error can become smaller and smaller, that is, the prediction result is getting closer and closer to the true label.
[0094] Step 4. Use the optimal multi-task steganalysis model to perform steganalysis on the test set text. Input the test set text into the optimal multi-task steganalysis model and output the predicted label. By setting a detection threshold α, the final detection result is as follows:
[0095]
[0096] At this time, task k belongs to the task of steganalysis of generative steganalysis text and normal text.
[0097]
[0098] At this time, task k belongs to the task of generative steganalysis of steganalysis text and natural text.
[0099]
[0100] At this time, task k belongs to the task of steganalysis of embedded steganalysis text and normal text.
[0101]
[0102] At this time, task k belongs to the embedded steganalysis task of steganalysis of steganalysis text and natural text.
[0103]
[0104] At this time, task k belongs to the natural text and normal text classification task.
[0105] Through the above steps, the present invention divides the steganalysis task into five steganalysis tasks for the first time, and uses a private neural network constructed for each steganalysis task to generate a feature representation containing the specific domain information of the steganalysis task and unique steganalysis information; uses a shared convolutional neural network constructed for all tasks to generate a feature representation with low correlation with the specific task, low task sensitivity, and implicit connections between multiple steganalysis tasks, which is beneficial to improving the detection accuracy of each steganalysis task.
[0106] Through the present invention, it is possible to obtain for each steganalysis task the information specific to other steganalysis tasks, thereby improving the steganalysis performance of each steganalysis task; at the same time, the multi-task learning mechanism adopted by the present invention can help alleviate the data sparsity problem by using effective information from other related tasks, thereby effectively solving the problem of the scarcity of data sets.
Claims
1. A multi-class natural language steganalysis method, characterized in that The following steps are involved: Step 1. Represent the steganalysis task dataset as a word vector matrix; Step 2. Obtain the deep multi-task steganalysis feature representation of the text; Step 3. Training of the multi-task steganalysis model; Step 4. Using the multi-task steganalysis model to perform steganalysis on the test set text; wherein step 1 includes: 1.1 defining multiple steganalysis tasks, namely: (1) generative steganalysis task of steganalysis of steganalysis text and normal text; (2) generative steganalysis task of steganalysis of steganalysis text and natural text; (3) embedded steganalysis task of steganalysis of steganalysis text and normal text; (4) embedded steganalysis task of steganalysis of steganalysis text and natural text; (5) normal text and natural text classification task; 1.2 representing the text in the data set of each steganalysis task by a word vector matrix; Step 1.2 includes: obtaining each steganalysis task data set, dividing the data set into a training set and a test set, the data set includes a large number of classified texts, the texts are sentences; performing word segmentation and preprocessing on the texts in the data set to obtain the word vector matrix of the texts in the data set, as follows: Where D k Indicates that task k contains N k A dataset of texts, and Represents a text representation vector matrix and the corresponding classification label in task k, V represents the text The word vector of a word in , l represents the length of the text, d represents the dimension of the vector; k is a natural number between 1 and 5; Step 2 includes: 2.1 Build a separate convolutional neural network for each steganalysis task, called a private convolutional neural network, to obtain the unique feature representation of each steganalysis task, as follows: (2.1.1) Randomly extract a text from a steganalysis task training set, input the word vector matrix of this text into the private convolutional neural network of the steganalysis task to which the text belongs, use multiple convolution kernel matrices of different widths to extract feature information simultaneously, and generate candidate feature representations: c t =f c (W·x t:t+h-1 +b) Where W is a convolution kernel matrix; h is the width of the convolution kernel matrix, b is the bias term; f c is a nonlinear activation function; x represents the word vector matrix of the selected text; (2.1.2) For the features obtained by different convolution kernel matrices, a maximum pooling operation is performed, that is, the maximum value of each candidate feature representation is taken to obtain the maximum pooling feature: (2.1.3) Connect the maximum pooling features obtained by different convolution kernel matrices together to obtain the private feature representation of the text, which is specifically expressed as: Where m is the number of convolution kernel matrices; 2.2 Build the same convolutional neural network for all tasks, called a shared convolutional neural network, to capture shared feature representations that are less relevant to a specific steganalysis task, as follows: (2.2.1) In step 2.1.1, while the word vector matrix of the randomly extracted text is input into the private convolutional neural network of the steganalysis task to which the text belongs, the word vector matrix of the text is input into the pre-built shared convolutional neural network, and multiple convolution kernel matrices of different widths are used to simultaneously extract feature information to generate candidate feature representations: s t =f s (W·x t:t+h-1 +b) Where W is a convolution kernel matrix; h is the width of the convolution kernel matrix, b is the bias term; f s is a nonlinear activation function; x represents the word vector matrix of the extracted text; (2.2.2) For the features obtained by different convolution kernel matrices, a maximum pooling operation is performed, that is, each candidate feature representation takes the maximum value to obtain the maximum pooling feature: (2.2.3) Connect the maximum pooling features obtained by different convolution kernel matrices together to obtain the shared feature representation of the text, which is specifically expressed as: Where m is the number of convolution kernel matrices; 2.3 The feature representation of the private convolutional neural network output of the text and the feature representation of the shared convolutional neural network output are spliced together to obtain the deep multi-task steganalysis text representation vector of the text: H = (z·o); Step 3 includes: The deep multi-task steganalysis text representation vector of the randomly extracted text obtained in step 2 is input into the softmax layer of the task to which the text belongs. The formula is expressed as follows: The multi-task steganalysis model is trained using the training set texts of all steganalysis tasks, the network parameters are updated using back propagation, and the loss function is minimized through iterative optimization of the model to obtain the optimal multi-task steganalysis model; The loss function is defined as follows: Where K represents the number of steganalysis tasks, N k represents the number of samples in the k-th steganalysis task dataset, β k represents the weight of the k-th steganalysis task, represents the true label of the i-th sample in the k-th steganalysis task, It represents the probability that the i-th sample in the k-th steganalysis task contains stegographic information.
