A generative text information hiding detection method and system based on inter-word dependency

Abstract

The present invention discloses a generative text information hiding detection method and system based on word-to-word dependency, the method comprising: preprocessing the text information to be detected; mapping the preprocessed text into a distributed embedding vector; inputting the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text to be detected contains secret information; the text information hiding detection model is based on the word-to-word dependency attention mechanism and the aggregated dependency feature, and is implemented through global weighted reasoning. The method of the present invention can perform high-precision generative text information hiding detection, improves the detection accuracy, and is more suitable for generative text information hiding detection in complex situations. Through testing and verification, the present invention has achieved detection performance that exceeds that of existing methods.

Description

Technical Field

[0001] The present invention relates to the field of information security and deep learning technology, and in particular to a generative text information hiding detection method and system based on word-to-word dependency. Background Art

[0002] Information hiding, also known as steganography, is a technique that embeds secret information into a carrier so that it is difficult for the regulator to detect it. Existing text information hiding techniques can be divided into four categories. The first category is a method based on text image feature modification, which embeds secret information by modifying the text character spacing, line spacing, font color or specific pixel values ​​that are imperceptible to the human eye. The second category is a method based on text content replacement, which embeds secret information by adding, deleting or replacing some words and sentences in the original text. The third category is a generative text information hiding method, which directly generates a natural text based on the secret information to achieve information hiding. The fourth category is a method based on invisible character embedding, which embeds secret information by inserting invisible characters in the character encoding table into the text. Among these four methods, generative text information hiding technology has become a recent research hotspot because of its high embedding efficiency and the fact that it does not require modification of the carrier text. It is one of the methods that pose the greatest threat to network security. In order to eliminate these threats, it is necessary to carry out research on corresponding generative text information hiding detection methods.

[0003] Existing generative text information hiding detection methods can be divided into two categories, methods based on manually designed features and methods based on deep learning. The main idea of ​​the methods based on manually designed features is to extract various statistical features from the original document according to the characteristics of text data and steganography algorithms, and then input these features into machine learning models (such as support vector machines) for classification. The main disadvantages of this type of method are the cumbersome process of manually designing features, high error detection rate and lack of robustness. In recent years, with the introduction of deep learning technology into generative text information hiding detection, it has made great progress. However, these methods also have some problems.

[0004] Among the existing generative text steganalysis methods based on deep learning, some methods do not extract context-dependent features from the text, and directly input the embedding vector obtained by word embedding of the text into the classification network. Some methods use the recurrent neural network (RNN) structure to extract context-dependent information. However, there are many problems with the RNN structure, such as insufficient utilization of long-distance context, inability to simultaneously model forward and backward dependencies, and gradient vanishing or gradient exploding during training. In addition, in the design of the classification network, the existing network does not introduce a selection mechanism to filter out noise information when processing intermediate features, resulting in low feature processing efficiency. Summary of the invention

[0005] The purpose of the present invention is to overcome the technical defects of the existing generative text information hiding detection method, and propose a generative text information hiding detection method and system based on word dependency.

[0006] In order to achieve the above object, the present invention proposes a generative text information hiding detection method based on word dependency, the method comprising:

[0007] Preprocess the text information to be detected;

[0008] Map the preprocessed text into distributed embedding vectors;

[0009] Input the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text to be detected contains secret information;

[0010] The text information hiding detection model is based on the inter-word dependency attention mechanism and aggregated dependency features and is implemented through global weighted reasoning.

[0011] As an improvement of the above method, the text information to be detected is preprocessed; specifically, the text information to be detected is padded or discarded to regularize the number of words contained in the text segment to a preset length.

[0012] As an improvement of the above method, the mapping of the preprocessed text into a distributed embedding vector specifically includes:

[0013] Read the preprocessed text, perform static distributed embedding mapping and dynamic distributed embedding mapping on the input text containing k words, and obtain a static distributed embedding vector E of length l corresponding to each piece of text s and the dynamic distributed embedding vector E d , the two distributed embedding vectors are added point by point to obtain the embedding vector E.

[0014] As an improvement of the above method, the text information hiding detection model includes a serially connected word dependency extraction module and a global weighted reasoning module; wherein,

[0015] The inter-word dependency extraction module is used to extract explicit inter-word dependency features and potential inter-word dependency features between text words and perform dependency fusion;

[0016] The global weighted reasoning module is used to comprehensively utilize the inter-word dependency features in the extracted text to make a decision on whether the current text segment contains secret information.

[0017] As an improvement of the above method, the inter-word dependency extraction module is an inter-word dependency feature fusion module connected in parallel with an explicit dependency attention module and a potential dependency attention module. The processing process of the inter-word dependency extraction module specifically includes:

[0018] The embedding vector E is used to extract explicit inter-word dependencies through the explicit dependency attention module to obtain the explicit dependency feature f e At the same time, the embedding vector E is used to extract the potential inter-word dependency through the potential dependency attention module to obtain the potential dependency feature f l , f e and f l Through the inter-word dependency feature fusion module, cross-fusion processing is performed to obtain the fused dependency feature f c .

[0019] As an improvement of the above method, the global weighted reasoning module includes a feature position attention module, a regional convolution module, an inter-word convolution module and a linear decision module connected in series in sequence; the processing process of the global weighted reasoning module specifically includes:

[0020] The fused dependency feature f c The feature position is optimized and selected through the feature position attention mechanism. After information aggregation, splicing and increasing the attention weight coefficient, the weighted feature map F is obtained. W ; F W The locally enhanced feature F is obtained by sequentially passing through the regional convolution module and the two weight-sharing inter-word convolution modules. L , and then the linear decision module is used to perform feature mapping and normalization of the high-level abstract features to the output layer output results to obtain the classification result of whether it contains secret information.

[0021] As an improvement of the above method, the feature position attention module is an average pooling and a maximum pooling connected in parallel, followed by connecting convolution layers with different kernel sizes and a softmax activation function layer, and then multiplying the output of the softmax activation function layer by the input of the feature position attention module point by point.

[0022] As an improvement of the above method, the linear decision module includes a fully connected layer and a Sigmoid function connected in sequence; wherein,

[0023] The fully connected layer includes two neurons for realizing feature mapping of high-level abstract features to output results of the output layer;

[0024] The Sigmoid function is used to perform normalization processing to obtain a classification result of whether the input text contains secret information.

[0025] As an improvement of the above method, the method further includes a training step of a text information hiding detection model; specifically including:

[0026] Step 1) Establish a sample set, and randomly divide the text in the sample set into m groups according to each group of a segments; the sample set includes a number of text segments and a label corresponding to each text segment whether it contains secret information;

[0027] Step 2) Randomly select a group of texts, input the preprocessed texts into the text information hiding detection model, obtain the prediction results, and use cross entropy to calculate the error between the prediction results and the true labels to obtain the true labels The error between the predicted result y Repeat step 2) until the sample training is completed, and go to step 3);

[0028] Step 3) According to the error Select the optimal parameter combination to obtain a trained text information hiding detection model.

[0029] A generative text information hiding detection system based on word dependency, the system comprises: a text information hiding detection model, a preprocessing module, a distributed reading module and an output module; wherein,

[0030] The preprocessing module is used to preprocess the text information to be detected;

[0031] The distributed reading module is used to map the preprocessed text into a distributed embedding vector;

[0032] The output module is used to input the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text information to be detected contains secret information;

[0033] The text information hiding detection model is based on the inter-word dependency attention mechanism and aggregated dependency features, and is implemented through global weighted reasoning.

[0034] Compared with the prior art, the advantages of the present invention are:

[0035] 1. The method of the present invention can perform high-precision generative text information hiding detection, improves the detection accuracy, and is more suitable for generative text information hiding detection in complex situations;

[0036] 2. The generative text information hiding detection technical solution provided by the present invention performs multi-dimensional mapping on the input original text through a distributed reading module to obtain a corresponding distributed embedding vector;

[0037] 3. The inter-word dependency extraction module is used to extract the explicit inter-word dependencies and potential inter-word dependencies between distributed embedding vectors. The extraction of these two dependency features is mainly completed through the explicit dependency attention and potential dependency attention mechanisms, and the dependency feature aggregation module is used to integrate the explicit and potential dependency features extracted respectively;

[0038] 4. Based on the obtained aggregated dependency features, the global weighted reasoning module is used to first select the feature position, and then the weighted feature map is input into the convolution module composed of convolution windows of different kernel sizes and activation functions. Finally, the fully connected layer makes an inference decision on whether the text segment is embedded with secret information;

[0039] 5. When testing benchmark datasets, the present invention achieves performance that exceeds existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the generative text information hiding detection method based on word-to-word dependency of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0042] Example 1

[0043] like Figure 1 As shown, embodiment 1 of the present invention proposes a generative text information hiding detection method based on word dependency, the method comprising:

[0044] Preprocess the text information to be detected;

[0045] Map the preprocessed text into distributed embedding vectors;

[0046] Input the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text to be detected contains secret information;

[0047] The text information hiding detection model is based on the inter-word dependency attention mechanism and aggregated dependency features and is implemented through global weighted reasoning.

[0048] The generative text information hiding detection method proposed in the present invention completes data processing by a distributed reading module, an inter-word dependency extraction module, and a global weighted reasoning module. The specific process includes:

[0049] Step 1) Data preprocessing: randomly divide the texts required for training into m groups with a segments in each group, and regularize the length of each segment by padding and discarding. After regularization, the length of each segment is 20 words;

[0050] Step 2) Randomly read a group of texts, and use the static mapping and dynamic mapping in the distributed reading module to perform 300-dimensional distributed embedding mapping on the input text containing 20 words, and obtain a distributed embedding vector E with a length of 300 corresponding to each text segment. s ∈R 20×300 and E d ∈R 20×300 , add the two together to get the final embedding vector E∈R 20×300 , the static mapping and dynamic mapping are completed by a static dictionary and a dynamic dictionary respectively, wherein the static dictionary keeps parameters unchanged after loading the corpus pre-trained dictionary, and the parameters in the dynamic dictionary are randomly initialized and updated according to the gradient back propagation algorithm in the subsequent network training process;

[0051] Step 3) The embedding vector E∈R obtained in step 2) 20×300 , through different attention mechanisms, explicit inter-word dependency extraction and potential inter-word dependency extraction are performed to capture the rich inter-word dependency information in the text segment from two different perspectives, and obtain the explicit dependency feature f e and the potential dependency feature f l .

[0052] Step 4) For the dependency feature f obtained in step 3) e and f l , perform cross-fusion processing, integrate the dependency features of the same text segment from different perspectives, and obtain the fused dependency feature f c .

[0053] Step 5) The fusion feature f obtained in step 4) c , optimize the feature position selection, increase the importance of the local features with discriminability in the global context, filter out unnecessary feature noise, and obtain the weighted fusion feature f c ', for f c 'Perform regional fusion processing, integrate the distributed inter-word dependency information, and obtain the feature map f after dimensionality reduction d , extract the local correlation information between different words, and gradually increase the receptive field, and finally obtain the feature map f n .

[0054] Step 6) The feature map f obtained in step 5) is n , transform it into a one-dimensional feature, and pass it through the detection module to obtain the prediction result of whether the current text segment contains secret information.

[0055] Step 7) Using the gradient descent method to update the parameters in the above model; iterating repeatedly until the optimal parameter combination is trained;

[0056] Step 8) Implement generative text information hiding detection based on the optimal parameter model.

[0057] The method comprises: performing multi-dimensional mapping on the input original text through a distributed reading module to obtain a corresponding distributed embedding vector; using an inter-word dependency extraction module to extract explicit inter-word dependencies and potential inter-word dependencies between distributed embedding vectors, wherein the extraction of these two dependency features is mainly completed through explicit dependency attention and potential dependency attention mechanisms, and using a dependency feature aggregation module to integrate the respectively extracted explicit and potential dependency features; based on the obtained aggregated dependency features, firstly selecting feature positions thereof by using a global weighted inference module, and then inputting the weighted feature map into a convolution module formed by stacking convolution windows of different kernel sizes and activation functions, and finally making an inference decision on whether secret information is embedded in the text segment by a fully connected layer.

[0058] like Figure 1 As shown in FIG. 1 , the inter-word dependency extraction module proposed in the present invention is composed of an explicit dependency attention module, a potential dependency attention module, and a dependency fusion module. “⊙” represents the point-by-point addition of the matrix, “σ” represents the point-by-point multiplication of the matrix, and “σ” represents the activation function.

[0059] The step 3) specifically includes:

[0060] Step 3-1) Use the position encoding module to model the relationship between the text sequence and the previous and next text sequences. The position encoding vector can be obtained by the following formula:

[0061]

[0062]

[0063] Where pos represents the position and i represents the dimension. As can be seen from the formula, the position encoding vector values ​​of even and odd dimensions are calculated by different functions. Each dimension of the position encoding will correspond to a sinusoidal signal. Next, the sequence directional features modeled by the position encoding vector PE are superimposed on the original distributed vector E to obtain a distributed vector E with forward and backward directional information. O :

[0064] E O =E+PE

[0065] Step 3-2) The explicit dependency attention includes global average pooling and a convolution layer with a convolution kernel size of 3, a step size of 1, and a padding of 1. The global average pooling integrates the feature information of different dimensions of the word distributed vector, and uses a one-dimensional convolution kernel to slide in the neighborhood of the word. The number of input channels and output channels is 1, and a sigmoid activation function is used to obtain the weight coefficient representing the explicit dependency feature between the current word and the adjacent words before and after. The calculation formula is as follows:

[0066] P=GAP(E O )

[0067] P C =1d-conv(P)

[0068]

[0069] L=P Z ⊙E O

[0070] Among them, "GAP" represents global average pooling, "1d-conv" represents one-dimensional convolution operation, "σ" represents sigmoid activation function, and "⊙" represents point-by-point multiplication;

[0071] Step 3-3) The potential dependency attention branch is composed of a linear layer. The distributed vector obtained in step 3-1) is mapped to Q, K and V through three different linear layers. The number of neurons in the linear layer is the same as the dimension of the original distributed vector (300). The transpose of Q and K is subjected to matrix multiplication. After scaling, it is normalized by the softmax activation function and then multiplied by V. The calculation results of the above operation under different heads are concatenated and mapped by the linear layer to obtain the final potential inter-word dependency feature G. The calculation formula is as follows:

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] in,"(·) (i) " represents the intermediate operation result of the i-th head, "W Q ”, “W K ”, “W V ” and “W ML” represents the weight matrix of the linear layer, Represents the scaling factor, “Concat” represents the feature concatenation operation, and “N h " indicates the number of headers, which is set to 8 in the implementation.

[0078] The step 4) specifically includes:

[0079] Step 4-1) The explicit dependency feature L and the potential dependency feature G are respectively subjected to layer normalization and then concatenated. The calculation formula is as follows:

[0080] B=Concat(LayerNorm(L),LayerNorm(G))

[0081]

[0082] Among them, "LayerNorm" represents the layer normalization operation, "E(x)" and "Var(x)" represent the mean and standard deviation functions respectively, "η" and "β" represent the learnable affine transformation coefficients, which are updated along with the network training, and "ε" is a manually set hyperparameter, which is set to 1e-5 in the implementation;

[0083] Step 4-2) Input B into the dependency fusion module, which consists of convolution kernels of different sizes and activation functions. The calculation formula is as follows:

[0084] B c1 =conv 1×1 (σ(conv 3×7 (B)))

[0085] B c2 =conv 1×1 (B)

[0086] B c =B c1 +B c2

[0087] Among them, "conv k " represents a two-dimensional convolution operation with a convolution kernel size of k. In the figure, the parameters of the convolution kernel on the left side of the dependency fusion module from bottom to top are: input channel 2, output channel 16, padding 1*3; input channel 16, output channel 1, padding zero, and the parameters of the convolution kernel on the right side are: input channel 2, output channel 1, padding zero.

[0088] Step 4-3) For B c Perform linear transformation and normalization operations, and the calculation formula is as follows:

[0089] B L =BC W L +b L +B C

[0090] B N =LayerNorm(B L )

[0091] Among them, "W L " and "b L ” represents the weight matrix and bias coefficient of the linear layer.

[0092] The global weighted reasoning module structure of the present invention is as follows: Figure 1 shown.

[0093] The step 5) specifically includes:

[0094] Step 5-1) Extract the inter-word dependency and obtain the feature B that combines the latent dependency and explicit dependency N The feature position optimization selection is completed by the feature position attention mechanism. First, the original features are aggregated by average pooling and maximum pooling. The two results are concatenated and passed through convolution layers with different kernel sizes and softmax activation function layers to obtain the attention weight coefficient. The coefficient is multiplied point by point with the original feature to obtain the weighted feature map. The calculation formula is as follows:

[0095] F P =Concat(AvgPool(B N ),MaxPool(B N ))

[0096] F S =Conv 1×1 (F P )

[0097] M A (F S )=σ(F S +Conv 3×7 (F S ))

[0098] F W =M A (F S )⊙F S

[0099] Among them, “AvgPool” and “MaxPool” represent average pooling and maximum pooling operations, and “MaxPool” represents the average pooling and maximum pooling operations. A (F S )” represents the obtained attention weight coefficient, “F W” represents the features weighted by attention weights.

[0100] Step 5-2) The weighted feature F W After a two-dimensional convolution with a convolution kernel size of 3×300, the reduced-dimensional feature map F is obtained. R , for F R Perform ReLU nonlinear feature activation operation to obtain the output feature F A , as input to the subsequent parts of the model;

[0101] Step 5-3) Set feature F A After two weight-shared convolution kernels with a size of 3×1 and ReLU nonlinear feature activation operations, the locally enhanced feature F is obtained. L .

[0102] The detection module includes a fully connected layer and a Sigmoid function; the fully connected layer includes two neurons to realize feature mapping of high-level abstract features to output results of the output layer; the Sigmoid normalization function is used for normalization processing to obtain a classification result of whether the input text contains secret information.

[0103] Step 7) specifically includes:

[0104] Step 7-1) The error between the model output result and the true label is calculated; the error calculation adopts cross entropy, which can be expressed as:

[0105]

[0106] in, Represents the true label The error between the predicted result y, p(x i ) represents the forward output of the model, q(x i ) represents the true label, Σ represents the sum;

[0107] Step 7-2) uses the parameters obtained in step 7-1) as the weight value of this iteration; randomly select a group of texts from the remaining texts, and obtain a new parameter combination through steps 2), 3), 4), 5), 6) and 7-1); iterate repeatedly until one iteration cycle is completed;

[0108] Step 7-3) reshuffles the training text and goes to step 1); repeats until the optimal parameter combination is trained.

[0109] Step 8) specifically includes:

[0110] Step 8-1) For the text to be detected, the number of words contained in the text segment is regularized to 20 by using operations such as filling and discarding;

[0111] Step 8-2) Input the regularized text into the optimal parameter model and obtain the prediction result through forward transmission of the model;

[0112] Step 8-3) Compare the category corresponding to the maximum probability value of the prediction result of each text segment with the true label. If they are consistent, the prediction is correct, otherwise, the prediction is wrong;

[0113] Step 8-4) executes step 8-1), step 8-2) and step 8-3) until all the texts to be tested are tested.

[0114] Example 2

[0115] Embodiment 2 of the present invention provides a high-precision generative text information hiding detection system based on word-to-word dependency, which is implemented by a generative text information hiding detection network model constructed according to the method of embodiment 1. The system includes:

[0116] A preprocessing module, used for preprocessing the text information to be detected;

[0117] Distributed reading module, used to map preprocessed text into distributed embedding vectors;

[0118] An output module is used to input the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text information to be detected contains secret information;

[0119] The text information hiding detection model is based on the inter-word dependency attention mechanism and aggregated dependency features, and is implemented through global weighted reasoning.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A generative text information hiding detection method based on word dependency, the method comprising: Preprocess the text information to be detected; Map the preprocessed text into distributed embedding vectors; Input the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text to be detected contains secret information; The text information hiding detection model is based on the inter-word dependency attention mechanism and the aggregated dependency features, and is implemented through global weighted reasoning; The text information hiding detection model includes a serially connected word dependency extraction module and a global weighted reasoning module; wherein, The inter-word dependency extraction module is used to extract explicit inter-word dependency features and potential inter-word dependency features between text words and perform dependency fusion; The global weighted reasoning module is used to comprehensively utilize the inter-word dependency features in the extracted text to make a decision on whether the current text segment contains secret information; The inter-word dependency extraction module is an explicit dependency attention module and a potential dependency attention module connected in parallel to an inter-word dependency feature fusion module. The processing process of the inter-word dependency extraction module specifically includes: The embedding vector E is used to extract explicit inter-word dependencies through the explicit dependency attention module to obtain the explicit dependency feature f e At the same time, the embedding vector E is used to extract the potential inter-word dependency through the potential dependency attention module to obtain the potential dependency feature f l , f e and f l Through the inter-word dependency feature fusion module, cross-fusion processing is performed to obtain the fused dependency feature f c .

2. The generative text information hiding detection method based on word-to-word dependency according to claim 1 is characterized in that: The preprocessing of the text information to be detected specifically includes: using padding or discarding operations on the text information to be detected to regularize the number of words contained in the text segment to a preset length.

3. The generative text information hiding detection method based on word-to-word dependency according to claim 2 is characterized in that: The method maps the preprocessed text into a distributed embedding vector; specifically includes: Read the preprocessed text, perform static distributed embedding mapping and dynamic distributed embedding mapping on the input text containing k words, and obtain a static distributed embedding vector E of length l corresponding to each piece of text s and the dynamic distributed embedding vector E d , the two distributed embedding vectors are added point by point to obtain the embedding vector E.

4. The generative text information hiding detection method based on word-to-word dependency according to claim 1 is characterized in that: The global weighted reasoning module includes a feature position attention module, a regional convolution module, an inter-word convolution module and a linear decision module connected in series in sequence; the processing process of the global weighted reasoning module specifically includes: The fused dependency feature f c The feature position is optimized and selected through the feature position attention mechanism. After information aggregation, splicing and increasing the attention weight coefficient, the weighted feature map F is obtained. W ; F W The locally enhanced feature F is obtained by sequentially passing through the regional convolution module and the two weight-sharing inter-word convolution modules. L , and then the linear decision module is used to perform feature mapping and normalization of the high-level abstract features to the output layer output results to obtain the classification result of whether it contains secret information.

5. The generative text information hiding detection method based on word-to-word dependency according to claim 4 is characterized in that: The feature position attention module is an average pooling and a maximum pooling connected in parallel, which are then connected in sequence to convolution layers with different kernel sizes and a softmax activation function layer, and then the output of the softmax activation function layer is multiplied point by point with the input end of the feature position attention module.

6. The generative text information hiding detection method based on word-to-word dependency according to claim 4 is characterized in that: The linear decision module includes a fully connected layer and a Sigmoid function connected in sequence; wherein, The fully connected layer includes two neurons for realizing feature mapping of high-level abstract features to output results of the output layer; The Sigmoid function is used to perform normalization processing to obtain a classification result of whether the input text contains secret information.

7. The generative text information hiding detection method based on word-to-word dependency according to claim 1 is characterized in that: The method also includes a training step of a text information hiding detection model; specifically including: Step 1) Establish a sample set, and randomly divide the text in the sample set into m groups according to each group of a segments; the sample set includes a number of text segments and a label corresponding to each text segment whether it contains secret information; Step 2) Randomly select a group of texts, input the preprocessed texts into the text information hiding detection model, obtain the prediction results, and use cross entropy to calculate the error between the prediction results and the true labels to obtain the true labels The error between the predicted result y Repeat step 2) until the sample training is completed, and go to step 3); Step 3) According to the error Select the optimal parameter combination to obtain a trained text information hiding detection model.

8. A generative text information hiding detection system based on word dependency, the system comprising: Text information hiding detection model, preprocessing module, distributed reading module and output module; among them, The preprocessing module is used to preprocess the text information to be detected; The distributed reading module is used to map the preprocessed text into a distributed embedding vector; The output module is used to input the distributed embedding vector into a pre-established and trained text information hiding detection model to obtain a decision on whether the text information to be detected contains secret information; The text information hiding detection model is based on the inter-word dependency attention mechanism and the aggregated dependency features, and is implemented through global weighted reasoning; The text information hiding detection model includes a serially connected word dependency extraction module and a global weighted reasoning module; wherein, The inter-word dependency extraction module is used to extract explicit inter-word dependency features and potential inter-word dependency features between text words and perform dependency fusion; The global weighted reasoning module is used to comprehensively utilize the inter-word dependency features in the extracted text to make a decision on whether the current text segment contains secret information; The inter-word dependency extraction module is an explicit dependency attention module and a potential dependency attention module connected in parallel to an inter-word dependency feature fusion module. The processing process of the inter-word dependency extraction module specifically includes: The embedding vector E is used to extract explicit inter-word dependencies through the explicit dependency attention module to obtain the explicit dependency feature f e At the same time, the embedding vector E is used to extract the potential inter-word dependency through the potential dependency attention module to obtain the potential dependency feature f l , f e and f l Through the inter-word dependency feature fusion module, cross-fusion processing is performed to obtain the fused dependency feature f c .

Images