VR virtual scene information hiding method based on deep learning

Abstract

The invention relates to a VR virtual scene information hiding method based on deep learning, and the method is characterized in that a virtual scene picture provided by VR equipment is firstly obtained, and then a segment of watermark or a picture is used as confidential information input for embedding processing. According to the method, a reversible neural network is adopted as a basic network, an Eca attention mechanism is introduced into a hidden network layer, and image hiding and confidential information restoration are achieved. In the image hiding and restoring process, network parameters are shared, the number of parameters is reduced, and therefore the resource cost of model training is saved. In the model training process, the hidden loss and the restoration loss are set to account for 30% and 70% of the total loss respectively, and network training is carried out in an end-to-end mode. And finally, storing the optimal model by comparing the total loss value, and constructing model.pt as a pre-training model for information hiding. According to the method, the defects of the current image hiding technology in the aspects of concealment, reducibility and safety are overcome.

Description

Technical Field

[0001] The invention belongs to the field of modern communication and digital security, and in particular relates to an improved information hiding method based on a reversible network. Background Art

[0002] As an advanced information security technology, information hiding aims to cleverly embed confidential image information into signals to protect the security and privacy of transmission. This technology makes the embedded information almost imperceptible to unauthorized observers, while also focusing on the reversibility of information. Only under certain conditions can an authorized recipient accurately restore the original confidential information. Therefore, image hiding technology demonstrates privacy protection capabilities in the field of network security and has become an important research and application direction in the field of modern communications and digital security.

[0003] In order to improve the concealment of confidential information, various image hiding methods based on deep learning have emerged in recent years. Baluja proposed a method based on deep learning to hide the full-size secret image into the cover image. This scheme initially explored the application of deep neural networks in the field of steganography. [1] On this basis, Rehman et al. added regular loss to ensure end-to-end joint training. However, both of them have the problem of color distortion. [2] Zhu et al proposed an image hiding technology based on deep learning. [3] , using a learnable, end-to-end image hiding technology model to solve the problem of color distortion. However, the hiding capacity of this work needs to be further improved. Baluja tried to connect the secret images together and generate the cover image through a hidden autoencoder [4] However, this work easily causes texture duplication artifacts in images. Guan et al. proposed a hiding algorithm based on low-frequency wavelet loss, which effectively solved the problem of texture duplication artifacts by not hiding any secret information in the low-frequency sub-band. [5] Therefore, the present invention proposes a hiding method based on deep learning to further improve the concealment of confidential data hiding.

[0004] [1] S. Baluja, "Hiding images in plain sight: deep steganography," presented at the Proceedings of the 31st International Conference on NeuralInformation Processing Systems, Long Beach, California, USA, 2017.

[0005] [2] A.ur Rehman, R.Rahim, S.Nadeem, and S.ul Hussain, "End-to-End TrainedCNN Encoder-Decoder Networks for Image Steganography," in Computer Vision-ECCV2018 Workshops, Cham, 2019: Springer International Publishing, pp.723-729.

[0006] [3] J. Zhu, R. Kaplan, J. Johnson, and L. Fei-Fei, "HiDDeN: Hiding Data WithDeep Networks," in Computer Vision-ECCV 2018, Cham, 2018: Springer International Publishing, pp. 682-697.

[0007] [4] S. Baluja, "Hiding Images within Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.42, no.7, pp.1685-1697, 2020.

[0008] [5] Z.Guan et al., "DeepMIH: Deep Invertible Network for Multiple ImageHiding," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.45, no.1, pp.372-390, 2023. Summary of the invention

[0009] The purpose of the present invention is to provide a VR virtual scene information hiding method based on deep learning to make up for the deficiencies of current image hiding technology in terms of concealment, restoration and security.

[0010] To achieve the above object, the technical solution of the present invention mainly includes the following steps:

[0011] The first step is experimental data and settings. For network training, we use a self-made dataset to train the model, and the dataset is about 6K. The test dataset includes DIV 2K, 100 images with a resolution of 1024*1024. COCO dataset, 5,000 images with a resolution of 256*256 and ImageNet dataset, 50,00 images with a resolution of 256*256. When testing, make sure that the carrier and secret images have the same resolution. The training block size is 1024*1024, and the total number of iterations is about 500. The training batch size is set to 8, of which 1 / 2 is selected as the carrier Cover image and 1 / 2 as the secret image secret. The Adam optimizer uses standard parameters and the initial learning rate is 1×10 -6 , halved every 50 iterations.

[0012] Step 2: Network selection and improvement. The INN (Invertible Neural Network) reversible network is selected as the baseline network, and the network parameters are convolutionally activated. The results of the previous layer are concatenated with the original input to perform linear scaling and exponential operations on the results. The self-improved Eca attention mechanism is introduced in the Invblock network layer and integrated into the hidden module, which is added to the last layer of the network to adaptively adjust the weights of the features. Residual connections are made by stacking multiple convolutional layers, and the channel attention mechanism is utilized. This allows the network to pay more attention to important areas containing confidential information during the hiding process to improve the concealment and security of the hiding effect. The network shares network parameters during image hiding and restoration, reducing the number of parameters and saving training resource costs.

[0013] The third step is model training. The preprocessed dataset is trained for 500 epochs to obtain the inference model of image information hiding. The training batch size is set to 8, half of which are carrier images and the other half are secret images. The Adam optimizer is used with an initial learning rate of 1×10 -6 , halved every 50 iterations. The model is saved through the loss function. The total loss function consists of three different losses: hidden loss, restoration loss and low-frequency wavelet loss. Among them, the hidden loss is the loss between the hidden image Steg and the original image Cover. The restoration loss is the loss between the confidential image and the restored confidential image. The low-frequency wavelet loss is the loss between the hidden image and the carrier image in the low-frequency wavelet. For fairness, the three loss methods use the same calculation formula.

[0014] The fourth step is reasoning and evaluation. The VR virtual scene image and the COCO dataset image are input into the trained reasoning model to obtain the VR image containing the secret image and the restored secret image. Four indicators are used to evaluate the performance of the method: peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), root mean square error (RMSE) and mean absolute error (MAE). The larger the PSNR and SSIM values, the smaller the RMSE and MAE values, and the better the image quality.

[0015] The present invention adopts deep learning combined with attention mechanism, which can hide and restore confidential information in real time and accurately. A set of efficient data preprocessing and enhancement processes are designed to significantly improve the model's ability to recognize hidden information, ensuring the accuracy and reliability of information hiding and restoration. The present invention has the advantages of simple operation, low cost, strong adaptability, etc. It can be widely used in various VR virtual scenes, providing an effective technical means for information hiding and data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flowchart of an embodiment of the present invention;

[0017] Figure 2 This is a diagram of the self-improved secret image feature extraction structure of the present invention.

[0018] Figure 3 , Figure 4 This is the hidden and restored network structure of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0020] Reference Figure 1 , Figure 2 , Figure 3 and Figure 4 , a VR virtual scene information hiding method based on deep learning, mainly includes the following steps:

[0021] The first step is to preprocess the training data set, collect and annotate the VR scene images, and perform data enhancement on some VR scene images. The collected VR scene images are adjusted to the same size to ensure normal input for model training.

[0022] In the second step, the INN reversible network is selected as the baseline network. The self-improved Eca attention mechanism is introduced in the Invblock network layer and integrated into the hidden module. It is added to the last layer of the network to adaptively adjust the weight of the features and focus on important feature areas more accurately. This greatly improves the accuracy and robustness of target detection while effectively reducing the amount of calculation.

[0023] Thus, an improved INN network is constructed as the main means of image hiding, and model.pt is used as the pre-trained model for image hiding;

[0024] The third step is to input the data set preprocessed in the first step into the pre-trained model for classification and training for 500 epochs to obtain the inference model for image information hiding;

[0025] In the fourth step, the actual VR scene image data is input into the inference model obtained in the third step, so as to output the VR image containing hidden information and the restored secret image, and compare their PSNR and SSIM.

[0026] Reference Figure 1 , Figure 2 , Figure 3 and Figure 4 ,The specific operation scheme of a VR virtual scene information hiding method based on deep learning is as follows.

[0027] The first step is to prepare the dataset.

[0028] (1) Prepare image data

[0029] In order to make the model more generalizable and robust, and to achieve satisfactory results in hiding and restoring images. A total of 6,000 images were collected and screened by consulting public databases on the Internet, crawling with crawler scripts, and actually shooting, collecting, integrating and screening. In order to adapt to the structure of the neural network, all images were normalized to obtain a set of images with a size of 1024*1024. The total number of iterations was about 500. The training batch size was set to 8, of which 1 / 2 was selected as the original cover image and 1 / 2 as the secret image secret. The Adam optimizer used standard parameters and the initial learning rate was 1×10 -6 , halved every 50 iterations.

[0030] (2) Preprocess the image.

[0031] Perform a one-dimensional discrete wavelet transform on the original image and the confidential image, and decompose the input data x according to the spatial dimension (height and width). Divide x into even rows and odd rows, and then divide these rows by columns. As shown in Formula 1:

[0032]

[0033] Here, x_even_rows means selecting the even-numbered rows of data x, and x_odd_rows means selecting the odd-numbered rows of data x.

[0034] For each pair (x_even_rows, x_odd_rows), split the columns into even columns and odd columns. 1 , x 2 x 3 and x 4 Corresponding to four different sub-bands, the pixel values ​​of even columns and odd columns are separated in the horizontal direction. As shown in Formula 2:

[0035]

[0036] Through the four arithmetic operations, four wavelet coefficients are calculated: low low frequency (LL), low high frequency (LH), high low frequency (HL), and high frequency (HH). These calculations follow the standard combination of Haar wavelet bases to generate representations of different frequency components. As shown in Formula 3:

[0037]

[0038] Finally, torch.cat((LL, HL, LH, HH), 1) is used to concatenate the four wavelet coefficients along the channel dimension (the first dimension, index 1) to form a new tensor. Each input channel is converted into four output channels, representing information of different frequencies.

[0039] The second step is to train the deep convolutional neural network.

[0040] (1) Taking the INN reversible network as the benchmark network, in this invention, we first integrate the high-frequency information of the original image and the confidential image obtained in the first step into the hidden network. Compared with the previous INN reversible network, the INN_Pro reversible network structure proposed in this invention is more concise and efficient. The hiding and restoration processes of this structure share network parameters, and the two processes are completely reversible. However, in the hiding and restoration processes, the order of processing image information is opposite.

[0041] (2) This paper proposes a network that combines convolution with attention mechanism. The network optimizes the extraction of secret information features, thereby enhancing the image hiding effect. Specifically, the input is first convolved 5 times, and the output of each convolution is used as the input of the next convolution. Then it is spliced ​​on the two-dimensional channel, as shown in Formula 4. Subsequently, the spliced ​​values ​​are globally averaged pooled, as shown in Formula 5. Next, by calculating the size of the adaptive convolution kernel, the sigmoid activation function is used to map the weights between (0-1) to obtain the eigenvalues ​​under different weights, as shown in Formula 6. Then, the obtained eigenvalues ​​are multiplied with the original feature map to adjust the feature attention of different regions. Finally, the value obtained in the previous step is convolved and the activation function is applied, as shown in Formula 7. Introducing the proposed feature focus network in the image hiding framework helps to improve the perception of important information, thereby improving the quality and security of the hidden image. The feature extraction formula is defined as follows:

[0042]

[0043] In Formula 4, H and W represent the height and width of the input tensor, respectively, and X cij represents the value after convolution, δ represents the ReLU activation function, W i Each convolution uses a different convolution kernel.

[0044]

[0045] In formula 5, Indicates that X cij Add the values ​​of each element of Z c Represents the global average pooling amount.

[0046] Y c =σ(W 2 *δ(W 1 *Z c )) (6)

[0047] In Formula 6, Y c represents the attention weight, * represents the convolution operation, δ represents the ReLU activation function, σ represents the Sigmoid activation function, W 2 and W 2 are the convolution kernel parameters respectively.

[0048] Y cij =Conv*(X cij ·Y c ) (7)

[0049] In Formula 7, Y cij The obtained attention weight Y c With the input feature map Xcij Multiply and finally perform a convolution.

[0050] By learning the weight distribution of these different scale features, we can adaptively focus on more important features. This method effectively combines global and local information and enhances the model's ability to understand image details and overall layout.

[0051] (3) In order to obtain a model with high hiding and high restoration effects. In this invention, in order to better improve the hiding ability of the image, an MSE-Loss is proposed. Loss function In this article, the total loss function consists of three different losses: hiding loss, restoration loss and low-frequency wavelet loss. Among them, the hiding loss is the loss between the hidden image Steg and the original image Cover. The restoration loss is the loss between the confidential image and the restored confidential image. The low-frequency wavelet loss is the loss of the hidden image and the carrier image in the low-frequency wavelet. For fairness, the three loss methods use the same calculation formula.

[0052] Hiding loss: The forward hiding process is to hide the confidential image in the carrier image to generate the Steg image. It is required that the cover image Steg is highly indistinguishable from the original image cover. In order to achieve this goal, the hiding loss Lguid is defined as follows:

[0053]

[0054] In Formula 8, is the network parameter, N is the number of training samples, x (n) cover is the actual value, x (n) steg is the predicted value of the model, and Lguid(θ) represents the average evaluation of the loss of the sample.

[0055] Restoration loss: During the restoration process, the auxiliary variable m is sampled from a Gaussian distribution to help recover the secret image Steg generated by the forward hiding process. To achieve this goal, we define the restoration loss as follows:

[0056]

[0057] In Formula 9, is the network parameter, N is the number of training samples, represents a secret image, represents the restored secret image, Δ x Represents the difference between the secret image Secret and the restored image Rev-secret.

[0058] Low-frequency wavelet loss: This paper proposes a low-frequency band wavelet loss L lpf , by calculating the low-frequency loss of image information, the image hiding ability of the network is increased, L lpfThe lower it is, the closer the low-frequency subband of the original image Cover is to the low-frequency subband of the hidden image Steg after wavelet decomposition. lpf The definition is as follows:

[0059]

[0060] In formula 10, is the network parameter, N is the number of training samples, Represents the low-frequency subband part of the original image Cover, Represents the low-frequency subband part of the hidden image Steg, Δ x It represents the difference between the original image Cover and the hidden image Steg in the low-frequency part.

[0061] The third step is to test the detection effect of this system

[0062] This study compares the proposed method with some existing image hiding methods. These include a traditional image steganography method, called LSB, and 5 deep learning-based methods: HiDDeN, Weng et al. and Baluja, HiNet, Deep-MIH. In the experiments, we use the COCO dataset to train HiDDeN, Weng et al. and Baluja, HiNet, Deep-MIH models under the same conditions to ensure the objectivity of the experimental comparison.

[0063] We compare the experimental results of the proposed method with other methods, including PSNR, SSIM, MAE and RMSE obtained on DIV-2K, COCO and ImageNet datasets. From the table, we can see that the proposed method has improved the four indicators in cover / steg and secret / re-cover images.

[0064] The embodiment of the present invention adopts a lightweight model.pt model combined with the Eca attention mechanism, and its generalization ability is strong enough. It can not only adapt to pictures in VR scenes, but also achieve satisfactory results in concealment, restoration, and security. At the same time, we use statistical steganalysis tools named SRNet, Zhu-Net, and SiaStegNet to evaluate the performance of the method in this paper, ensuring the accuracy and reliability of the hidden results.

[0065] VR scenes combined with image information hiding technology can not only protect and transmit sensitive information by providing safe, hidden and interactive means, but also enhance user experience and the diversity of application scenarios. This technology has broad application potential in many fields, from military to education, to entertainment and medical treatment, and can use this technical means to enhance the value and effect of applications.

[0066] The examples described above clearly illustrate the embodiments of the present invention in order to better illustrate the technical ideas of the present invention. For technicians in this field, various forms of changes and modifications can be made on the basis of this specification, and the obvious and reasonable extensions of these changes and modifications still fall within the scope of protection of the present invention.

Claims

1. A VR virtual scene information hiding method based on deep learning, characterized in that The main steps include: The first step is to obtain images related to VR virtual scenes and collect public datasets on the Internet, such as COCO data. Use images related to VR virtual scenes as carrier images and COCO datasets as secret images. When testing, ensure that the carrier and secret images have the same resolution. The training block size is 1024×1024 to ensure normal input for model training; In the second step, the INN reversible network is selected as the baseline network, and the self-improved Eca attention mechanism is introduced into the custom Invblock network layer and integrated into the hidden module, and added to the last layer of the network to adaptively adjust the weight of the features and more accurately focus on important feature areas to improve the concealment and security of the hidden effect. The network shares network parameters in image hiding and restoration, which reduces the number of parameters to a certain extent and saves resource costs for model training; In the third step, the dataset preprocessed in the first step is input into the reversible network for 500 epoch training to obtain the inference model of image information hiding. The training batch size is set to 8, of which 1 / 2 is selected as the carrier image and 1 / 2 as the secret image. The Adam optimizer uses standard parameters and the initial learning rate is 1×10 -6 , halved every 50 iterations; In the fourth step, the VR virtual scene and the COCO dataset images are input into the inference model obtained in the third step, so as to obtain the VR image containing the secret image and restore the secret image. In order to evaluate the performance of the proposed method, four indicators are used to measure the quality of the hidden image and the restored secret image, including the peak signal-to-noise ratio (PSNR) similarity index (SSIM), root mean square error (RMSE) and mean absolute error (MAE). The larger the PSNR and SSIM values, the smaller the RMSE and MAE values, and the better the image quality.

2. The method for hiding VR virtual scene information based on deep learning according to claim 1, characterized in that: ① Perform convolution activation on the network parameters. By concatenating the result of the previous layer with the original input, the result is linearly scaled and exponentially operated, so that the network can effectively reduce the number of parameters and computational complexity, greatly improving the convergence speed, generalization and robustness; ② By stacking multiple convolutional layers for residual connection and using the channel attention mechanism, the network pays more attention to the important areas containing confidential information during the hiding process, so as to improve the concealment and security of the hiding effect.

Images