Steganography and extraction method for hiding secret information in GPT-2 model

Abstract

The invention relates to a steganography and extraction method for hiding secret information in a GPT-2 model, which comprises the following steps: a sender obtains a secret key for embedding and extracting hidden information and to-be-hidden information, inputs a data set to train the GPT-2 model, constructs a model loss function, adjusts model parameters according to the loss function, and obtains the hidden information; and when information obtained after secret key processing is carried out on coverage parameters of a full connection layer of the GPT-2 model is the same as information to be hidden, training is completed, the steganographic model is transmitted through a public channel, a receiver obtains a secret key for embedding and extracting hidden information and obtains the steganographic model in the public channel, and the steganographic model is obtained. The hidden information is extracted through the secret key and the steganographic model, and compared with the prior art, the method has the advantages that the undetectability of the model is improved, so that the security of information transmission is improved, the capacity of embedded information is improved, and the like.

Description

Technical Field

[0001] The present invention belongs to the field of information hiding, and in particular, relates to a steganography and extraction method for hiding secret information in a GPT-2 model. Background Art

[0002] In recent years, with the rapid development of the Internet and mobile communication technology, individuals and enterprises have become increasingly dependent on the Internet for information transmission and communication. However, information security and privacy protection also face severe challenges. Whether it is the government, enterprises or individual users, they all need to protect sensitive information from theft, tampering or illegal monitoring. Especially in the fields of finance, military, and medicine, the leakage of information may bring huge economic losses and even endanger national security. Therefore, how to secretly transmit information in public communication channels has become an important research topic.

[0003] Steganography is a technique that hides secret information in a covering medium, making it difficult to detect. Unlike traditional encryption technology, steganography not only focuses on the content protection of secret information, but also emphasizes the concealment of secret information transmission. Although the encrypted information cannot be deciphered, it is still easy to attract attention and interception; through steganography, secret information can be embedded in multimedia files such as pictures, audio, and video, making it look the same as ordinary data, thus avoiding suspicion. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a steganography and extraction method for hiding secret information in the GPT-2 model.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides a steganography and extraction method for hiding secret information in a GPT-2 model, comprising the following steps: a sender obtains a secret key for embedding and extracting hidden information and information to be hidden, inputs a data set to train the GPT-2 model, constructs a model loss function, and adjusts model parameters according to the loss function until information obtained after the coverage parameters of a fully connected layer of the GPT-2 model are processed by the secret key is the same as the information to be hidden, the training is completed, the steganographic model is transmitted through a public channel, a receiver obtains a secret key for embedding and extracting hidden information, obtains the steganographic model on a public channel, and extracts hidden information through the secret key and the steganographic model.

[0007] Furthermore, the GPT-2 model includes multiple Transformer modules, each Transformer module includes layer normalization, multi-head attention layer and fully connected layer.

[0008] Furthermore, the secret key and the information to be hidden are binary sequences.

[0009] Furthermore, the coverage parameter has the same size and dimension as the secret key.

[0010] Furthermore, the data set includes: AG News data set and IMDB data set.

[0011] Furthermore, the coverage parameters of the fully connected layer of the GPT-2 model are the coverage parameters of the fully connected layer of the last Transformer module of the GPT-2 model.

[0012] Furthermore, the information obtained after the cover parameters of the fully connected layer of the GPT-2 model are processed by the secret key is the same as the information to be hidden. The specific process includes: converting the cover parameters into a one-dimensional sequence w = [w1, w2, ..., w m×n ], according to the one-dimensional sequence w and the secret key K = {k1, k2, ..., k i ,...,k m×n}∈{0,1}, perform one-dimensional average pooling operation to obtain the intermediate sequence Y, the formula is: Y=Avgpool(w⊙K), and then process the intermediate sequence Y with Sigmoid function to obtain information M′, the formula is:

[0013] M′=σ(100×(σ(Y)-0.5))

[0014]

[0015] Round the information M′ to get the hidden information The formula is:

[0016]

[0017] According to the hidden information extracted and the information to be hidden M = {m1,m2,...,m i ,...,m t}∈{0,1} calculate the extraction error e, the formula is:

[0018]

[0019] Where t is the number of sequences of the information to be hidden M. If e = 0, it means that the information is extracted The same as the information to be hidden M. Furthermore, the loss function of the GPT-2 model is:

[0020] Loss=Loss0+β·Loss e

[0021] Among them, Loss0 is the cross entropy loss function of the GPT-2 model for text classification, Loss e To ensure the loss function of embedding secret information, the mean square error loss function is used. e for:

[0022]

[0023] β is a hyperparameter used to balance the text classification task and the steganography task.

[0024] 9. According to claim 1, a steganography and extraction method for hiding secret information in a GPT-2 model is characterized in that the hidden information is extracted through a secret key and a model that completes the steganography, and the extraction formula is:

[0025]

[0026]

[0027] in, is the one-dimensional sequence of coverage parameters of the fully connected layer of the last Transformer module of the model, To extract hidden information.

[0028] Furthermore, the value of β is related to the size of the model training data set and the hidden information. When the hidden information is 147,456 bits and the training data set is the AG News data set, the value of β is set to 900. When the hidden information is 147,456 bits and the training data set is the IMDB data set, the value of β is set to 1100.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] (1) The present invention ensures the security requirements of information steganography and extraction during the communication transmission process by hiding information in the model and transmitting it through public channels and secret keys.

[0031] (2) The present invention embeds hidden information during the training process of the GPT-2 model instead of embedding it by modifying network parameters, so it has less impact on the performance of the original task, improves the undetectability of the model, and thus improves the security of information transmission.

[0032] (3) The present invention avoids the impact of information hiding on the original task performance by hiding information in the last fully connected layer of the Transformer module of the model, thereby improving the undetectability of the model, thereby improving the security of information transmission and increasing the capacity of embedded information.

[0033] (4) The present invention improves the robustness of the method by embedding hidden information during the training process of the GPT-2 model.

[0034] (5) The present invention implements GPT-2 model steganography based on Transformer architecture. No additional decoding network is required, and the receiver only needs to use a unique key to extract secret information, which reduces the risk of information exposure and improves the security of information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is the steganographic framework diagram of GPT-2 in the present invention;

[0036] Figure 2 Schematic diagram of the average pooling operation used in the present invention;

[0037] Figure 3 It is a schematic diagram of the result of formula 3 in the present invention;

[0038] Figure 4 Schematic diagram of the loss function of GPT-2 in the present invention;

[0039] Figure 5 The error graphs of the hyperparameter β and the classification task and the steganography task are shown in the present invention;

[0040] Figure 6 It is the information embedding capacity and classification task and steganography task error graph of the present invention;

[0041] Figure 7 Parameter distribution diagram of the steganographic GPT-2 model of the present invention and the original GPT-2;

[0042] Figure 8 It is the information extraction error diagram under different noises of the present invention;

[0043] Fig. 9 This is the information extraction error graph under random pruning of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0045] Embodiment 1:

[0046] With the development of deep learning, researchers have begun to explore information hiding methods in neural networks - model steganography. Model steganography is to hide secret information into the network model without affecting the performance of the model. Compared with traditional steganography methods, model steganography not only improves the capacity and robustness of steganography, but also greatly increases the undetectability of secret information. In addition, the steganographic network model can still complete various deep learning tasks. With the rapid improvement of computing power and the large-scale growth of data, researchers are deeply exploring larger and more complex generative network models to achieve a higher level of artificial intelligence. As an important representative of this trend, large language models have developed rapidly since 2018, and hundreds of different variants have emerged. With its large parameters and extensive text data, large language models have demonstrated excellent natural language processing and generation capabilities, and are capable of a variety of tasks such as intelligent question and answer, text generation, and text translation. Large language models have become a frontier field of artificial intelligence research. Since large language models have a large number of parameters, this lays the foundation for the realization of high-capacity information hiding methods.

[0047] The present invention proposes a model steganography method for hiding secret information in a large language model. Specifically, the sender disguises the process of hiding secret information as the training process of the GPT-2 model. During its training process, the secret information is embedded in the Transformer module of the GPT-2 model. In order to minimize the impact of information hiding on the performance of the original task, the secret information is hidden in the fully connected layer of the last Transformer module of the GPT-2 model. After the receiver obtains the steganographic GPT-2 model from the public channel, it can extract the secret information using a unique secret key. In addition, ordinary users can use the steganographic GPT-2 model to complete various downstream tasks without being aware of the existence of secret information, thereby realizing the covert transmission of secret information.

[0048] The present invention adopts the following technical solution:

[0049] In order to save computing costs, the present invention selects the smallest version of the GPT-2 model to achieve information hiding. Figure 1 As shown, the GPT-2 model has a total of 12 Transformer modules, each of which includes layer normalization, multi-head attention layers, and fully connected layers. In the process of training the GPT-2 model for text classification tasks, the sender uses a secret key to encode the fully connected layer in the 12th Transformer module of the GPT-2 model to embed secret information. The specific steps of the present invention are as follows:

[0050] (1) By Figure 1It can be seen that the embedding target is the fully connected layer of GPT-2, and the parameters of this layer are called coverage parameters. First, the coverage parameters are converted into a one-dimensional sequence w = [w1, w2, ..., w m×n ]. K={k1,k2,...,k i ,...,k m×n}∈{0,1} is the key used to embed and extract secret information, the secret information M={m1,m2,...,m i ,...,m t}∈{0,1} and K are both binary sequences, and w and K have the same size and dimension. Secondly, the secret information M can be added to the coverage parameter w during the model training process. The multiplication formula is as follows:

[0051] w⊙K=w i ×k i (1)

[0052] (2) In order to find a suitable embedding capacity and ensure the successful embedding of secret information, it is necessary to sample the result of formula (1). To this end, the present invention adopts a one-dimensional average pooling operation, such as Figure 2 In addition, average pooling can maintain the overall data characteristics of the coverage parameters and reduce the impact of embedded secret information on the original task performance.

[0053] Y=Avgpool(w☉K) (1)

[0054] (3) After average pooling by formula (2), the size of Y is the same as the secret information M. Next, in order to make the sequence Y closer to M, Y is processed by formula (3):

[0055] M′=σ(100×(σ(Y)-0.5)) (3)

[0056]

[0057] Where σ(·) represents the Sigmoid function, which can map the input to the range [0,1]. First, the sequence Y is mapped to between 0 and 1 through the Sigmoid function, then it is translated and expanded 100 times, and then mapped again through the Sigmoid function, which will make more values ​​in the sequence Y close to 0 and 1. Therefore, this operation can effectively accelerate the convergence of the steganographic task. Figure 3 As shown, the effect of formula (3) is more intuitively explained.

[0058] (4) Since the loss function is used to adjust the model parameters in the process of training GPT-2 for text classification tasks to hide secret information, it is necessary to reconsider the loss function of the GPT-2 model. Figure 4As shown in the figure, Loss0 is the cross entropy loss function of the GPT-2 model for text classification, which mainly ensures the performance of the model in text classification tasks; Loss e It is the loss function to ensure the embedding of secret information, and adopts the mean square error loss function. Therefore, the loss of GPT-2 is defined as follows:

[0059] Loss=Loss0+β·Loss e 2

[0060]

[0061] Where β is a hyperparameter used to balance the text classification task and the steganography task. When β is small, it may cause the secret information to fail to embed. When β is large, the efficiency of information embedding will become higher, but at the same time it will affect the accuracy of GPT-2 in the classification task.

[0062] (5) After the GPT-2 model training is completed, the GPT-2 steganographic model containing secret information can be obtained. At this time, the coverage parameter embedded with secret information is The steganalysis model can be transmitted through public channels, such as Git Hub, Google Drive, etc. After the receiver obtains the steganalysis model, it can first Convert to Sequence Then use the unique secret key K to extract the secret information. Specifically, first Multiply it by K and then perform average pooling to get Then use formula (3) to process To obtain a sequence close to the secret information M. Finally, the complete secret information can be extracted by rounding. The specific secret information extraction formula is as follows:

[0063]

[0064]

[0065] At this point, the GPT-2 model with the secret information embedded is called a steganalysis model. The coverage parameters of the steganalysis model are The steganalysis model can be transmitted through public channels, such as Git Hub, Google Drive, etc.

[0066] (6) In addition, the present invention uses the extraction error e to evaluate the steganographic ability of GPT-2, which is defined as follows:

[0067]

[0068] Where t is the number of sequences of information M to be hidden. When e=0, it means It is completely consistent with M, so all secret information can be correctly extracted.

[0069] Embodiment 2:

[0070] The parts not mentioned in this embodiment are the same as those in Embodiment 1.

[0071] like Figure 1 As shown, the present invention provides a steganographic method for hiding secret information in GPT-2. The text classification datasets used include AG News (four-category dataset) and IMDB (two-category dataset). First, the present invention determines the optimal value of the hyperparameter β through experiments. Then, the feasibility of the method is verified from three aspects: embedding capacity, undetectability, and robustness.

[0072] according to Figure 1 The steganographic architecture built on GPT-2 to hide secret information is shown. All experiments of the present invention are implemented on the Pytorch platform and trained using a single NVIDIA GeForce RTX 3080Ti (12GB) under Windows 10 system. The experiment uses the Adam optimizer for parameter optimization with a batch size of 8. The learning rate on the AG News dataset is set to 5E-5, and the learning rate on the IMDB dataset is set to 5E-4.

[0073] Determine the hyperparameter β. As can be seen from formula (5), β can adjust the Loss0 of the classification task and the Loss of the steganography task e . Thus, the classification accuracy of the classification task and the extraction error of the steganography task are balanced. When the embedding capacity is 147,456 bits, the present invention explores the influence of different β on the classification accuracy and extraction error. Taking the AG News dataset as an example, Figure 5 As shown, from Figure 5 As can be seen from Figure a, when β = 0, the extraction error of the secret information is 0.499; as β increases, the extraction error gradually decreases to 0. Figure 5 As can be seen in Figure b, as β increases, the classification accuracy of the classification task decreases slowly, but generally remains between 0.925 and 0.935. The experimental results show that increasing β can effectively reduce the extraction error while having little effect on the classification accuracy. Figure 5 Middle a picture and Figure 5 In the middle b figure, when β = 900, the extraction error of secret information is 0, and the classification accuracy remains at 0.935. Therefore, when using the AG News dataset, the value of β can be set to 900. Similarly, when using the IMDB dataset, the value of β can be set to 1100.

[0074] Embedding capacity is one of the most important indicators of model steganography. Therefore, based on the parameter β selected in the above experiment, we further determined the maximum embedding capacity in GPT-2 model steganography. The maximum embedding capacity is determined by the extraction error and classification accuracy. When the classification accuracy is high and the extraction error is 0, the corresponding embedding capacity is the maximum embedding capacity. According to the coverage parameter and the size of the pooling kernel, we set the length of the secret information to N = {0, 58982, 78643, ..., 147467, ..., 235929}. Taking the AG New dataset as an example, Figure 6 As shown, from Figure 6 As can be seen in Figure a, when the embedding capacity is less than approximately 150,000 bits, the extraction error is 0; when the embedding capacity is greater than 150,000 bits, the extraction error begins to slowly increase. Figure 6 As can be seen in Figure b, as the embedding capacity increases, the classification accuracy decreases slightly, but it still remains in the range of [0.92, 0.935] overall, which shows that there is a trade-off between the steganographic task and the classification task: successfully embedding more secret information comes at the expense of the accuracy of the classification task. Figure 6 Middle a picture and Figure 6 In Figure b, the maximum embedding capacity can be set to 131,072 bits. Figure 6 The triangle in Figure b shows that when using the IMDB dataset, the maximum embedding capacity is 147,456 bits.

[0075] The easiest way for an attacker to distinguish between a stego model and the original model is to compare the difference in their parameter distributions. Therefore, under the premise that the secret data can be correctly extracted, we compare the parameter distributions of the stego GPT-2 model and the original GPT-2 to prove that this method has good undetectability. If the distribution of the coverage parameters of GPT-2 before and after steganography is more similar, the stego GPT-2 and the original GPT-2 models will be more difficult to distinguish, indicating that the undetectability will be higher. Figure 7 As shown in the figure, the parameter distribution of the original GPT-2 and the parameter distribution of the steganographic GPT-2. Figure 7 Middle a picture and Figure 7 Figure b in the middle shows the parameter distribution results after embedding 47,185 bits and 131,072 bits in the AG News dataset respectively; Figure 7 Figure C and Figure 7 Figure d in the middle shows the parameter distribution results of embedding 58,982 bits and 147,456 bits in the IMDB dataset. Figure 7 Middle a picture and Figure 7 Figure b, Figure 7 Figure C and Figure 7It can be seen from the figure d that when different embedding capacities can be correctly extracted, as the embedding capacity increases significantly, the similarity of the parameter distribution of the steganographic GPT-2 model and the original model gradually decreases. Figure 7 Middle a picture and Figure 7 Figure b, Figure 7 Figure C and Figure 7 As can be seen from the figure d in the figure, the difference in parameter distribution on the AG News dataset is greater than that on the IMDB dataset. In summary, the more complex the task and the larger the embedding capacity, the greater the difference in parameter distribution between the steganographic GPT-2 model and the original model, and the undetectability of the steganographic model will decrease accordingly. In order to more objectively illustrate the undetectability of this method. We use KL divergence to measure the undetectability of the model before and after steganography. The KL divergence formula is as follows:

[0076]

[0077] Where P(w) and Q(w) represent the probability of the original model and the stego model covering parameter w respectively. The lower the divergence value, the higher the undetectability of the stego GPT-2 model. Through calculation, it is found that the divergence value on the IMDB dataset is lower than that on the AG News dataset. At the same time, when 58,982 bits of secret information are embedded, its divergence value is 0.31; when 147,456 bits are embedded, its divergence value is 0.53. This is consistent with Figure 7 The conclusion is consistent with that of , that is, the larger the embedding capacity, the more complex the task, and the lower the undetectability. Therefore, high undetectability can be obtained by appropriately reducing the embedding capacity. In summary, this information hiding method has good undetectability.

[0078] The steganographic GPT-2 model can be transmitted through a public channel. After obtaining the steganographic model, the receiver can use the secret key to extract the secret data. However, since the public channel is lossy, the received steganographic GPT-2 model may contain a certain degree of noise. But within a certain noise range, the receiver can still extract the secret data. In order to verify the robustness of this method, we simulated a lossy channel by adding Gaussian noise and uniform noise to the Y vector at the decoding end. The Gaussian noise obeys Uniform noise obeys U(-Φ2, Φ2). We experimentally analyze the impact of different Φ1 and Φ2 on the extraction error. Figure 8 The extraction error on the AG News dataset is shown. When Φ1<0.06 or Φ2<0.17, 131,072 bits of secret information can be correctly extracted. This shows that when the embedding capacity is 131,072 bits, the ability of this method to resist Gaussian noise is weaker than N(0, 0.06 2), the ability to resist uniform noise is weaker than U(-0.17, 0.17). In addition, from Figure 8 As can be seen from , the robustness decreases as the embedding capacity increases. For example, Figure 8 Figure b shows that when Φ2>0.17, the extraction error gradually increases with the increase of embedding capacity. This means that there is a trade-off between robustness and embedding capacity, and increasing the embedding capacity will inevitably sacrifice the robustness of the steganographic task. Since the embedding capacity of the proposed scheme is very large (e.g., 131,072 bits), we can appropriately reduce the embedding capacity in exchange for higher robustness.

[0079] After the large language model is trained, it is usually necessary to deploy it to mobile devices or embedded systems, such as Raspberry Pi. However, due to the large number of parameters and computational complexity of the large model, it is not realistic to deploy it directly on these devices. In order to ensure the performance of the model while reducing the number of parameters and computational complexity so that it can be effectively deployed on mobile devices, model compression methods are usually used. For GPT-2 after steganography, model compression may affect the extraction of secret data. For this reason, we chose a random pruning method to evaluate the impact of pruning on the robustness of the steganography task. Specifically, since we cannot predict which pruning strategy the user will adopt, we randomly prune a certain proportion of the coverage parameters. Fig. 9 The experimental results on the AG News dataset are shown. When 10% of the coverage parameters are randomly pruned, 131,072 bits of data can still be correctly extracted. As the embedding capacity decreases, more parameters can be pruned. Only 64% of the parameters are needed to correctly extract 78,643 bits of secret information. This once again illustrates the trade-off between embedding capacity and parameters. In summary, the information hiding method has good robustness.

[0080] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A steganography and extraction method for hiding secret information in a GPT-2 model, characterized in that: The following steps are involved: The sender obtains the secret key for embedding and extracting hidden information and the information to be hidden, inputs the data set to train the GPT-2 model, constructs the model loss function, and adjusts the model parameters according to the loss function until the information obtained after the coverage parameters of the fully connected layer of the GPT-2 model are processed by the secret key is the same as the information to be hidden. After the training is completed, the completed steganographic model is transmitted through a public channel. The receiver obtains the secret key for embedding and extracting hidden information, and obtains the completed steganographic model on the public channel, and extracts the hidden information through the secret key and the completed steganographic model.

2. A steganography and extraction method for hiding secret information in a GPT-2 model according to claim 1, characterized in that: The GPT-2 model includes multiple Transformer modules, each of which includes layer normalization, multi-head attention layers, and fully connected layers.

3. A steganography and extraction method for hiding secret information in a GPT-2 model according to claim 1, characterized in that: The secret key and the information to be hidden are binary sequences.

4. A steganography and extraction method for hiding secret information in a GPT-2 model according to claim 1, characterized in that: The coverage parameter has the same size and dimensions as the key.

5. A steganography and extraction method for hiding secret information in a GPT-2 model according to claim 1, characterized in that: The data sets include: AG News data set and IMDB data set.

6. A method for steganography and extraction of secret information in a GPT-2 model according to claim 1, characterized in that: The coverage parameters of the fully connected layer of the GPT-2 model are the coverage parameters of the fully connected layer of the last Transformer module of the GPT-2 model.

7. A steganography and extraction method for hiding secret information in a GPT-2 model according to claim 1, characterized in that: The information obtained after the cover parameters of the fully connected layer of the GPT-2 model are processed by the secret key is the same as the information to be hidden. The specific process includes: converting the cover parameters into a one-dimensional sequence w = [w1, w2, ..., w m×n ], according to the one-dimensional sequence w and the secret key K = {k1, k2, ..., k i ,...,k m×n }∈{0,1}, perform one-dimensional average pooling operation to obtain the intermediate sequence Y, the formula is: Y=Avgpool(w⊙K), and then process the intermediate sequence Y with Sigmoid function to obtain the information M ′ , the formula is: M ′ =σ(100×(σ(Y)-0.5)) The information M ′ Rounding to get hidden information The formula is: According to the hidden information extracted and the information to be hidden M = {m1,m2,...,m i ,...,m t }∈{0,1} calculate the extraction error e, the formula is: Where t is the number of sequences of the information to be hidden M. If e = 0, it means that the information is extracted The same as the information M to be hidden.

8. A method for steganography and extraction of secret information in a GPT-2 model according to claim 1, characterized in that: The loss function of the GPT-2 model is: Loss=Loss0+β·Loss e Among them, Loss0 is the cross entropy loss function of the GPT-2 model for text classification, Loss e To ensure the loss function of embedding secret information, the mean square error loss function is used. e for: β is a hyperparameter used to balance the text classification task and the steganography task.

9. A method for steganography and extraction of secret information in a GPT-2 model according to claim 1, characterized in that: The hidden information is extracted by using the secret key and the steganographic model, and the extraction formula is: in, is the one-dimensional sequence of coverage parameters of the fully connected layer of the last Transformer module of the model, To extract hidden information.

10. A steganography and extraction method for hiding secret information in a GPT-2 model according to claim 1, characterized in that: The value of β is related to the size of the model training data set and the hidden information. When the hidden information is 147,456 bits and the training data set is the AG News data set, the value of β is set to 900. When the hidden information is 147,456 bits and the training data set is the IMDB data set, the value of β is set to 1100.

Images