A method of image information hiding based on semantic weighting

Abstract

The present invention discloses an image information hiding method based on semantic weighting, which belongs to the field of information hiding technology. The main contribution is to propose an image information hiding scheme that processes secret images from a high-order feature level, in order to meet the needs of secure transmission of massive secret images and powerful third-party steganalysis technology, while ensuring the quality of reconstructed secret images, to achieve a large load and minimize the impact of embedded secret images on the perceptual characteristics of the carrier image. First, an image information hiding system model is constructed, and then a network model of an image information hiding method based on semantic weighting is constructed, including a semantic preprocessing network, a hiding network and a decryption network. According to the needs of training the network model, a reverse gradient propagation model and a loss function model are constructed, and finally network training, testing and system performance evaluation are carried out. The evaluation results show that compared with the existing image information hiding method that does not consider semantic weighting, the scheme proposed by the present invention has better performance.

Description

Technical Field

[0001] The invention belongs to the technical field of information hiding, and in particular is an image information hiding method based on semantic weighting. Background Art

[0002] With the development of computer network technology and the popularization of smart devices, a large amount of secret information involving personal privacy or organizational secrets is transmitted in public channels, which is very easy to be stolen and attacked by third parties, bringing serious security problems. Information hiding technology embeds secret information into natural carriers without changing the perceptual characteristics of the carriers. It completes the transmission of secret information through the transmission of the secret carriers on public channels, which is considered to be an effective means to ensure information security. Among them, information hiding with images as carriers is called image information hiding, which is currently the most widely used information hiding technology.

[0003] Early image information hiding methods directly modified the lowest bit of the carrier image pixel value into secret information. This method had little impact on the visual effect of the carrier image and low complexity, but the generated secret image had obvious histogram statistical characteristics and was easily cracked by a third party using steganalysis technology. The more mature adaptive image information hiding method first analyzed the embedding cost of each pixel in the carrier image, and then selected the area with complex texture and content, and used STC (Syndrome-Trellis Codes) to embed secret information. This method has good security, but it relies on manual processing of the carrier image and has high complexity.

[0004] The development of deep neural networks and the intelligence of edge devices have led to the rise of methods for hiding image information using deep neural networks, mainly including image information hiding methods based on autoencoders (AE) and image information hiding methods based on generative adversarial networks (GAN). The image information hiding method based on AE achieves the embedding and reconstruction of secret images of the same size as the carrier image through end-to-end training of encoders and decoders. This method has a large load, but the visual effect of the generated secret images needs to be further improved. The image information hiding method based on GAN introduces third-party steganalysis technology for adversarial training, which has good security, but the load of the carrier image is small.

[0005] Faced with the massive demand for secure transmission of secret information and powerful third-party steganalysis technology, a method of image information hiding with large load, better hiding effect and greater security is needed. None of the existing methods consider the changes in the local semantic importance of secret information. Taking secret images as an example, they are only processed from the low-order pixel level and all the extracted features are completely embedded in the carrier image. However, different areas of the secret image have different semantic importance: areas with complex textures and content have higher semantic importance and need to retain more features for reconstruction; areas with simple textures and content have lower semantic importance and only need some features to achieve better reconstruction effects. Since the embedding of redundant secret image features will affect the perceptual characteristics of the carrier image, it is necessary to process the secret image from the high-order feature level and consider image information hiding methods based on semantic weighting. Summary of the invention

[0006] In order to solve the above problems, the present invention provides an image information hiding method based on semantic weighting in combination with an automatic encoder, which semantically crops the extracted secret image features according to the semantic importance of the secret image, and then embeds the cropped secret image features into a carrier image of the same size as the secret image to generate a secret image. Under the premise of ensuring the quality of the reconstructed secret image, the influence of the embedded secret image on the perceptual characteristics of the carrier image is reduced, a larger load capacity and better hiding effect are achieved, and the security of information hiding is improved.

[0007] The specific steps include:

[0008] Step 1: Construct an image information hiding system model;

[0009] The system model includes a transmitter and a receiver.

[0010] The processing process of the entire system is as follows: 1) The sending end randomly selects the carrier image and the secret image, extracts features and performs semantic importance analysis on the secret image, generates a semantic importance map representing the semantic importance of different regions of the secret image, and then crops the secret image features according to the semantic importance map, retaining only the key features that affect the quality of the secret image reconstructed by the receiving end; 2) The sending end performs semantic weighting, that is, connects the secret image features after semantic cropping with the carrier image on the channel, uses a deep neural network to achieve deep fusion and embedding, and generates a secret image with similar perceptual characteristics to the carrier image, and transmits the secret image to the receiving end through an open channel; 3) The receiving end receives the secret image, extracts the secret image features from it using a deep neural network, and restores the reconstructed secret image.

[0011] The main evaluation indicators of the system include the similarity of the perceptual characteristics of the carrier image and the secret image, and the similarity of the perceptual characteristics of the secret image and the reconstructed secret image.

[0012] Step 2: Construct a network model of image information hiding method based on semantic weighting;

[0013] The network model includes a semantic preprocessing network, a hidden network, and a decryption network. The sending end uses the semantic preprocessing network and the hidden network to embed the secret image into the carrier image and generate the secret image. The receiving end uses the decryption network to recover the reconstructed secret image from the secret image. The detailed network modeling steps are as follows:

[0014] Step 201, constructing a semantic preprocessing network model;

[0015] The purposes of the semantic preprocessing network are as follows: 1) to enlarge or reduce the secret image to ensure that it has the same size as the carrier image. In the present invention, the sizes of the secret image and the carrier image are both 64×64×3, where 64 represents the number of pixels in the width and height of the image, and 3 represents the number of RGB channels of the image; 2) to extract secret image features; 3) to generate a semantic importance map representing the semantic importance of different regions of the secret image, and to crop the secret image features according to the semantic importance map for channel connection and embedding with the carrier image.

[0016] Its network model includes feature extraction module, semantic importance analysis module and feature clipping module.

[0017] The feature extraction module takes a secret image of size 64×64×3 as input and outputs a secret image feature of size 64×64×65. The network structure consists of two convolutional modules (Conv Block). Conv Block is a multi-scale feature fusion network that extracts features of different scales of the image through multiple different convolution kernels and then performs feature fusion. The specific convolution kernel sizes are 3×3, 4×4, and 5×5, the number of convolution kernel channels are 50, 10, and 5, the convolution step size is 1, and the activation function is the ReLU function. After completing the feature extraction of these three scales, the extracted features are connected in the channel dimension.

[0018] The semantic importance analysis module takes the secret image features of size 64×64×65 output by the feature extraction module as input and outputs the secret image semantic importance map of size 64×64×1. The network structure consists of two Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels and the convolution step are both 1, and the activation function is the Sigmoid function. The semantic information of each pixel of the secret image is first analyzed by the Conv Block, and then the semantic importance of each pixel is calculated by the convolution layer, and the Sigmoid function is used to map it to a value between 0 and 1 to obtain the semantic importance map of the secret image. The larger the value of the semantic importance map at a certain pixel, the more important the pixel is, and the more feature channels need to be retained and embedded in the carrier image for reconstruction.

[0019] The feature cropping module takes the secret image features of size 64×64×65 output by the feature extraction module and the secret image semantic importance map of size 64×64×1 output by the semantic importance analysis module as input, and outputs the secret image features of size 64×64×65 after semantic weighted cropping. This module consists of mask generation and dot multiplication cropping. First, a semantic importance mask of the same size as the secret image feature is generated from the secret image semantic importance map, and then the mask is dot-multiplied with the secret image feature to retain the key features required for reconstructing the secret image and crop redundant features, that is, to set the feature value to 0, so as to ensure that while achieving a good effect of reconstructing the secret image, the impact of the embedded secret image features on the perceptual characteristics of the carrier image is reduced.

[0020] Assuming that the secret image feature output by the feature extraction module is F(W,H,C), and the secret image semantic importance map output by the semantic importance analysis module is M(W,H,1), then the eigenvalues ​​of the secret image feature output by the feature cropping module should be expressed as:

[0021]

[0022] Where W and H represent the width and height of the image or feature respectively, C represents the number of channels of the feature, F′ represents the secret image feature after semantic weighted cropping by the feature cropping module, (w,h,c) represents a position in the feature map, w=1,...,W, h=1,...H, c=1,...,C.

[0023] Specifically, mask generation first requires quantizing and approximating the semantic importance map, which is divided into L quantization levels:

[0024]

[0025] Wherein, Q represents a quantization operation, l represents a certain quantization level, and l=1,...,L.

[0026] Mask generation then expands the semantic importance map M(W,H,1) into a semantic importance mask M′(W,H,C) of the same size as the secret image features. Each position of the semantic importance mask takes a value of 0 or 1, where 0 means that the secret image feature point corresponding to the position needs to be cropped, and 1 means that the secret image feature point corresponding to the position needs to be retained. The greater the weight of the semantic importance map of a pixel, the more channels with the semantic importance mask value of 1 for the pixel, and the more secret image features are retained and embedded in the carrier image. The values ​​of each position of the semantic importance mask can be expressed as:

[0027]

[0028] Point product cropping multiplies the semantic importance mask with the secret image feature to achieve semantic weighting, retain the key features that affect the effect of reconstructing the secret image, and discard redundant features. The feature values ​​of the secret image feature after cropping can be expressed as:

[0029] F′(w,h,c)=F(w,h,c)*M(w,h,c)

[0030] Step 202: construct a hidden network model;

[0031] The purpose of the hidden network is to embed and deeply fuse the secret image features with the carrier image, generate a secret image with similar perceptual characteristics to the carrier image, and transmit it in an open channel to avoid being detected, stolen, and attacked by a third party. Since the human sensory system is insensitive to subtle changes in high-frequency images or areas with complex textures and content, the hidden network is responsible for analyzing and determining these areas and embedding the secret image features.

[0032] The hidden network first needs to connect the semantically weighted cropped secret image features of size 64×64×65 output by the semantic preprocessing network and the carrier image of size 64×64×3 on the channel to obtain a preliminary combination of the secret image and the carrier image of size 64×64×68, and then use this as input to output a secret image of size 64×64×3. The structure of its network model consists of five Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels is 3, the convolution step is 1, and the activation function is the Tanh function. First, the Conv Block is used to determine the area in the carrier image that is suitable for embedding the secret image features and realize the deep fusion of the two. Then, the Tanh function is used to map the fusion result to a value between -1 and 1 to obtain the secret image.

[0033] Step 203: construct a decryption network model;

[0034] The purpose of the decryption network is to process the received secret image, extract the embedded secret image features, and restore the reconstructed secret image. The restored reconstructed secret image should have similar perceptual characteristics to the sender's original secret image, so that the receiver can use the secret image for subsequent tasks and realize the communication function of information hiding.

[0035] The decryption network receives a 64×64×3 secret image as input and outputs a 64×64×3 reconstructed secret image. Its network structure consists of five Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels is 3, the convolution step is 1, and the activation function is the Tanh function. First, the Conv Block analyzes the secret image features, extracts the embedded semantically weighted cropped secret image features, fills and restores them, and then maps the extraction results to values ​​between -1 and 1 through the Tanh function to obtain the secret image.

[0036] Determine the area in the carrier image that is suitable for embedding the secret image features and achieve deep fusion of the two, and then map the fusion result to a value between -1 and 1 through the Tanh function to obtain the reconstructed secret image.

[0037] Step 3: Construct a reverse gradient propagation model based on semantic weighted image information hiding method;

[0038] To achieve image information hiding, an end-to-end training network model is required. The training of the network model requires continuous iteration of reverse gradient propagation. Both the hidden network and the decryption network in the network model can implement reverse gradient propagation. However, because the quantization and semantic importance graph channel expansion in the mask generation module of the semantic preprocessing network are not differentiable, the reverse gradient propagation is interrupted. The gradient before this module is zero everywhere and cannot be iterated. To solve this problem, the mask generation is re-expressed as:

[0039]

[0040] in, Represents the ceiling function.

[0041] The reverse gradient generated by the mask can be expressed as:

[0042]

[0043] Note that the unrepresented method is still used in the forward propagation, and this re-representation is only used in the reverse gradient propagation. As a result, the network model is differentiable everywhere, the reverse gradient can be transmitted, and end-to-end training can be performed.

[0044] Step 4: Construct a loss function model;

[0045] The network model training relies on the loss function model, which needs to consider the similarity of the perceptual characteristics of the carrier image and the secret image, as well as the perceptual characteristics of the secret image and the reconstructed secret image. The similarity of the perceptual characteristics of the carrier image and the secret image determines the information hiding effect and security. The higher the similarity, the better the security and the better the hiding effect. The perceptual characteristics of the secret image and the reconstructed secret image determine the degree of realization of the communication function. The higher the similarity, the better the secret image transmission effect and the higher the accuracy of subsequent task processing at the receiving end. If only one side is considered, the network will tend to ensure the similarity of this perceptual characteristic and ignore the other side. In extreme cases, the generated secret image may not contain the secret image, resulting in communication failure, or it may only contain the secret image, resulting in third-party detection, theft and attack.

[0046] Therefore, the loss function model consists of two parts. The mean square error (MSE) of two images is used to measure the similarity of their perceptual characteristics. The smaller the MSE, the higher the similarity. The MSE calculation formula is as follows:

[0047]

[0048] Among them, x and y represent the two images whose perceptual characteristics are to be compared.

[0049] Assuming that C and C′ represent the carrier image and the secret image respectively, S and S′ represent the secret image and the reconstructed secret image respectively, the loss function can be expressed as:

[0050] Loss(C,C′,S,S′)=Loss_C+β*Loss_S=MSE(C,C′)+β*MSE(S,S′)

[0051] Among them, Loss_C and Loss_S represent the loss function of the similarity between the carrier image and the secret image, and the loss function of the similarity between the secret image and the reconstructed secret image, respectively. β represents the weight of the two, which is used to weigh the quality of the secret image and the reconstructed secret image. The larger β is, the more critical the quality of the reconstructed secret image is.

[0052] Step 5: Based on the network model and loss function model of the semantic weighted image information hiding method, a deep neural network training model is constructed;

[0053] Step 501: construct a data set;

[0054] The Tiny-ImageNet dataset is used. This dataset has 200 image classes, each of which has 500 training images, 50 validation images, and 50 test images. It is one of the most commonly used datasets in the field of computer vision.

[0055] To ensure that the network model has reasonable and uniform input, all images are first randomly cropped into a size of 64×64×3, and then the value range of RGB channels from 0 to 255 is mapped to the value range of 0 to 1, and the images are standardized to conform to the standard normal distribution with a mean of 0 and a standard deviation of 1, which speeds up the convergence of the network model. Standardization can be expressed as:

[0056]

[0057] Among them, x represents the image data, μ x represents the mean of the image, σ x Represents the standard deviation of the image.

[0058] 10,000 training images are randomly selected from the processed Tiny-ImageNet dataset as the training set, and the training set is divided into two groups to form secret image-carrier image pairs for training the network model.

[0059] Step 502: setting hyper parameters;

[0060] The training environment was built based on the UBUNTU operating system and the NVIDIA GTX1080 graphics card, and the network model was implemented using the PyTorch framework. The main hyperparameters were set as follows: 1) The batch size was set to 64, and the number of training epochs was set to 300; 2) The semantic preprocessing network, hidden network, and decryption network were all optimized using the Adam optimizer, and the learning rate was set to 0.001; 3) The quantization level L of the semantic importance map was set to 13; 4) The loss function weight β was set to 0.75, 1, and 1.25.

[0061] Step 6: Based on the network model and training model of the semantic weighted image information hiding method, a deep neural network test and system performance evaluation model is constructed;

[0062] Step 601: construct a network test model;

[0063] The 10,000 test images from the processed Tiny-ImageNet dataset are used as the test set, which is divided into two groups to form secret image-carrier image pairs for testing the network model. The test model hyperparameters are the same as those during training.

[0064] Step 602: construct a system performance evaluation model;

[0065] The performance of image information hiding system includes the load of carrier image, the similarity of perceptual characteristics between carrier image and secret image, and the similarity of perceptual characteristics between secret image and reconstructed secret image.

[0066] The carrier image load has objective performance indicators: absolute embedding capacity and relative embedding capacity. Absolute embedding capacity refers to the number of bytes of the secret image contained in the carrier image, and relative embedding capacity refers to the number of bytes of the secret image contained per pixel in the carrier image.

[0067] The similarity of the perceptual characteristics of two images includes objective performance indicators such as Average Pixel Deviation (APD), Average Pixel Square Deviation (APSD), Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

[0068] APD refers to the average deviation of the values ​​of each pixel and each RGB channel of the two images, and APSD refers to the square of the average deviation of the values ​​of each pixel and each RGB channel of the two images. The smaller the APD and APSD values, the more similar the perceptual characteristics of the two images are.

[0069] PSNR refers to the ratio of the maximum signal power to the signal noise power. The secret image embedded in the carrier image can be regarded as noise that affects the quality of the carrier image, and the carrier image embedded in the secret image can be regarded as noise that affects the quality of the secret image. The larger the PSNR value, the more similar the perceptual characteristics of the two images are. The calculation formula is as follows:

[0070]

[0071] Here, n represents the number of bits per pixel.

[0072] SSIM measures the correlation between different regions of an image from three aspects: brightness, contrast, and structure. The larger the SSIM value, the more similar the perceptual characteristics of the two images are. The calculation formula is as follows:

[0073]

[0074]

[0075]

[0076] SSIM(x,y)=[l(x,y) α c(x,y) β s(x,y) γ ]

[0077] Among them, x and y represent two images, l(x,y) represents brightness comparison, c(x,y) represents contrast comparison, s(x,y) represents structure comparison, and μ x and μy represents the mean of x and y, σ x and σ y represents the standard deviation of x and y, σ xy represents the covariance, c 1 、c 2 and c 3 It is a constant to avoid the denominator being 0. α, β and γ are generally set to 1, and the calculation formula is as follows:

[0078]

[0079] The degree of similarity of the perceptual characteristics of two images is measured by subjective performance indicators, such as the residual difference between the two images and the statistical characteristics of the histogram. The residual difference between the two images evaluates whether there is color difference and artifacts, etc. The smaller the residual difference, the better the visual effect. The statistical characteristics of the histogram evaluate the ability to resist third-party steganalysis. The closer the statistical characteristics of the histogram, the better the ability to resist steganalysis.

[0080] The advantages of the present invention are:

[0081] (1) A semantic weighted image information hiding method. The network model includes a semantic preprocessing network, a hiding network, and a decryption network. Under the premise of ensuring the quality of reconstructed secret images, it achieves a larger load and better hiding effect, thereby improving the security of information hiding.

[0082] (2) A semantic preprocessing network that processes the secret image from a high-level feature level. Different from existing methods that only process from a low-level pixel level, this network considers the changes in the local semantic importance of the secret image, retains more features for regions with complex textures and content for reconstruction, and crops redundant features for regions with simple textures and content to avoid the impact of embedding on the quality of the carrier image;

[0083] (3) A semantically weighted image information hiding method is proposed. The loss function is constructed by considering the similarity between the perceptual characteristics of the carrier image and the secret image, as well as the perceptual characteristics of the secret image and the reconstructed secret image. The method is trained and tested on the Tiny-ImageNet dataset, providing a variety of objective and subjective indicators to evaluate the system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 A schematic diagram of the image information hiding system model constructed by the present invention;

[0085] Figure 2 A schematic diagram of a network model of an image information hiding method based on semantic weighting constructed by the present invention;

[0086] Figure 3 A schematic diagram of the network structure of the convolution module in the network model constructed in the present invention;

[0087] Figure 4 This is a graph showing the convergence performance and convergence results of the number of training rounds when the weight β of the loss function of the present invention is 1;

[0088] Figure 5 The performance and result diagram of the hiding and reconstruction effects and the number of training rounds when the weight β of the loss function of the present invention is 1;

[0089] Figure 6 The residuals between the carrier image and the secret image, and between the secret image and the reconstructed secret image when the weight β of the loss function of the present invention is 1;

[0090] Figure 7 The histogram statistical feature comparison between the carrier image and the secret image, and between the secret image and the reconstructed secret image when the weight β of the loss function of the present invention is 1. DETAILED DESCRIPTION

[0091] In order to more clearly understand the technical principle of the present invention, the embodiments of the present invention are specifically and in detail described below with reference to the accompanying drawings.

[0092] A Semantic-Based Image Steganography Method is applied to an image information hiding system, including a transmitter and a receiver. The transmitter embeds a secret image into a carrier image, and transmits the secret image to the receiver by transmitting the carrier image to avoid being noticed by a third party. The receiver recovers and reconstructs the secret image from the received carrier image to perform subsequent tasks. Existing methods are difficult to simultaneously meet the requirements of efficient transmission of massive secret images and secure transmission in the face of powerful third-party steganalysis technology. Secret images are only processed from a low-order pixel level, without considering the changes in the local semantic importance of the secret image, and all secret image features are embedded in the carrier image. The present invention processes secret images from a high-order feature level, considering the local semantic importance of the secret image, retaining more features in areas with complex textures and content for reconstruction, and cutting redundant features in areas with simple textures and content to avoid affecting the carrier image.

[0093] According to the above method ideas, the challenges faced by the present invention include: 1) The semantic importance of different regions in different images is different, and it is necessary to consider a reasonable and universal method for extracting the semantic importance of images, and design a specific implementation of retaining and cropping features according to changes in local semantic importance; 2) Since the end-to-end training of the network model relies on the condition that it is differentiable everywhere, in the case of non-differentiable conditions, the gradient cannot be back-propagated. In order to prevent training failure, a reverse gradient propagation model needs to be designed for non-differentiable operations.

[0094] The overall process includes six steps: building an image information hiding system model, building a network model based on the semantic weighted image information hiding method, building a reverse gradient propagation model, building a loss function model, training based on the network model and loss function, and testing based on the network model and training results. Among them, building a network model based on the semantic weighted image information hiding method includes building a semantic preprocessing network model, building a hidden network model, and building a decryption network model. Training based on the network model and loss function includes building a data set and setting hyperparameters. Testing based on the network model and training results includes building a test machine and building a system performance evaluation model.

[0095] The specific steps include:

[0096] Step 1: Construct an image information hiding system model;

[0097] like Figure 1 As shown, the system model includes a transmitting end and a receiving end.

[0098] The processing process of the entire system is as follows: 1) The sending end randomly selects the carrier image and the secret image, extracts features and performs semantic importance analysis on the secret image, generates a semantic importance map representing the semantic importance of different regions of the secret image, and then crops the secret image features according to the semantic importance map, retaining only the key features that affect the quality of the secret image reconstructed by the receiving end; 2) The sending end performs semantic weighting, that is, connects the secret image features after semantic cropping with the carrier image on the channel, uses a deep neural network to achieve deep fusion and embedding, and generates a secret image with similar perceptual characteristics to the carrier image, and transmits the secret image to the receiving end through an open channel; 3) The receiving end receives the secret image, extracts the secret image features from it using a deep neural network, and restores the reconstructed secret image.

[0099] The main evaluation indicators of the system include the similarity of the perceptual characteristics of the carrier image and the secret image, and the similarity of the perceptual characteristics of the secret image and the reconstructed secret image.

[0100] Step 2: Construct a network model of image information hiding method based on semantic weighting;

[0101] like Figure 2 As shown in the figure, the network model includes a semantic preprocessing network, a hidden network, and a decryption network. The sending end uses the semantic preprocessing network and the hidden network to embed the secret image into the carrier image and generate the secret image. The receiving end uses the decryption network to recover the secret image from the secret image and reconstruct the secret image. The detailed network modeling steps are as follows:

[0102] Step 201, constructing a semantic preprocessing network model;

[0103] The purposes of the semantic preprocessing network are as follows: 1) to enlarge or reduce the secret image to ensure that it has the same size as the carrier image. In the present invention, the sizes of the secret image and the carrier image are both 64×64×3, where 64 represents the number of pixels in the width and height of the image, and 3 represents the number of RGB channels of the image; 2) to extract secret image features; 3) to generate a semantic importance map representing the semantic importance of different regions of the secret image, and to crop the secret image features according to the semantic importance map for channel connection and embedding with the carrier image.

[0104] Its network model includes feature extraction module, semantic importance analysis module and feature clipping module.

[0105] The feature extraction module takes a secret image of size 64×64×3 as input and outputs a secret image feature of size 64×64×65. The network structure consists of two convolutional modules (Conv Block). Figure 3 As shown in the figure, Conv Block is a multi-scale feature fusion network, which extracts features of different scales of the image through multiple different convolution kernels, and then performs feature fusion. Specifically, the convolution kernel sizes are 3×3, 4×4, and 5×5, the number of convolution kernel channels are 50, 10, and 5, the convolution step size is 1, and the activation function is the ReLU function. After completing the feature extraction of these three scales, the extracted features are connected in the channel dimension.

[0106] The semantic importance analysis module takes the secret image features of size 64×64×65 output by the feature extraction module as input and outputs the secret image semantic importance map of size 64×64×1. The network structure consists of two Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels and the convolution step are both 1, and the activation function is the Sigmoid function. The semantic information of each pixel of the secret image is first analyzed by the Conv Block, and then the semantic importance of each pixel is calculated by the convolution layer, and the Sigmoid function is used to map it to a value between 0 and 1 to obtain the semantic importance map of the secret image. The larger the value of the semantic importance map at a certain pixel, the more important the pixel is, and the more feature channels need to be retained and embedded in the carrier image for reconstruction.

[0107] The feature cropping module takes the secret image features of size 64×64×65 output by the feature extraction module and the secret image semantic importance map of size 64×64×1 output by the semantic importance analysis module as input, and outputs the secret image features of size 64×64×65 after semantic weighted cropping. This module consists of mask generation and dot multiplication cropping. First, a semantic importance mask of the same size as the secret image feature is generated from the secret image semantic importance map, and then the mask is dot-multiplied with the secret image feature to retain the key features required for reconstructing the secret image and crop redundant features, that is, to set the feature value to 0, so as to ensure that while achieving a good effect of reconstructing the secret image, the impact of the embedded secret image features on the perceptual characteristics of the carrier image is reduced.

[0108] Assuming that the secret image feature output by the feature extraction module is F(W,H,C), and the secret image semantic importance map output by the semantic importance analysis module is M(W,H,1), then the eigenvalues ​​of the secret image feature output by the feature cropping module should be expressed as:

[0109]

[0110] Where W and H represent the width and height of the image or feature respectively, C represents the number of channels of the feature, F′ represents the secret image feature after semantic weighted cropping by the feature cropping module, (w,h,c) represents a position in the feature map, w=1,...,W, h=1,...H, c=1,...,C.

[0111] Specifically, mask generation first requires quantizing and approximating the semantic importance map, which is divided into L quantization levels:

[0112]

[0113] Wherein, Q represents a quantization operation, l represents a certain quantization level, and l=1,...,L.

[0114] Mask generation then expands the semantic importance map M(W,H,1) into a semantic importance mask M′(W,H,C) of the same size as the secret image features. Each position of the semantic importance mask takes a value of 0 or 1, where 0 means that the secret image feature point corresponding to the position needs to be cropped, and 1 means that the secret image feature point corresponding to the position needs to be retained. The greater the weight of the semantic importance map of a pixel, the more channels with the semantic importance mask value of 1 for the pixel, and the more secret image features are retained and embedded in the carrier image. The values ​​of each position of the semantic importance mask can be expressed as:

[0115]

[0116] Point product cropping multiplies the semantic importance mask with the secret image feature to achieve semantic weighting, retain the key features that affect the effect of reconstructing the secret image, and discard redundant features. The feature values ​​of the secret image feature after cropping can be expressed as:

[0117] F′(w,h,c)=F(w,h,c)*M(w,h,c)

[0118] Step 202: construct a hidden network model;

[0119] The purpose of the hidden network is to embed and deeply fuse the secret image features with the carrier image, generate a secret image with similar perceptual characteristics to the carrier image, and transmit it in an open channel to avoid being detected, stolen, and attacked by a third party. Since the human sensory system is insensitive to subtle changes in high-frequency images or areas with complex textures and content, the hidden network is responsible for analyzing and determining these areas and embedding the secret image features.

[0120] The hidden network first needs to connect the semantically weighted cropped secret image features of size 64×64×65 output by the semantic preprocessing network and the carrier image of size 64×64×3 on the channel to obtain a preliminary combination of the secret image and the carrier image of size 64×64×68, and then use this as input to output a secret image of size 64×64×3. The structure of its network model consists of five Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels is 3, the convolution step is 1, and the activation function is the Tanh function. First, the Conv Block is used to determine the area in the carrier image that is suitable for embedding the secret image features and realize the deep fusion of the two. Then, the Tanh function is used to map the fusion result to a value between -1 and 1 to obtain the secret image.

[0121] Step 203: construct a decryption network model;

[0122] The purpose of the decryption network is to process the received secret image, extract the embedded secret image features, and restore the reconstructed secret image. The restored reconstructed secret image should have similar perceptual characteristics to the sender's original secret image, so that the receiver can use the secret image for subsequent tasks and realize the communication function of information hiding.

[0123] The decryption network receives a 64×64×3 secret image as input and outputs a 64×64×3 reconstructed secret image. Its network structure consists of five Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels is 3, the convolution step is 1, and the activation function is the Tanh function. First, the Conv Block analyzes the secret image features, extracts the embedded semantically weighted cropped secret image features, fills and restores them, and then maps the extraction results to values ​​between -1 and 1 through the Tanh function to obtain the reconstructed secret image.

[0124] Step 3: Construct a reverse gradient propagation model based on semantic weighted image information hiding method;

[0125] To achieve image information hiding, an end-to-end training network model is required. The training of the network model requires continuous iteration of reverse gradient propagation. Both the hidden network and the decryption network in the network model can implement reverse gradient propagation. However, because the quantization and semantic importance graph channel expansion in the mask generation module of the semantic preprocessing network are not differentiable, the reverse gradient propagation is interrupted. The gradient before this module is zero everywhere and cannot be iterated. To solve this problem, the mask generation is re-expressed as:

[0126]

[0127] in, Represents the ceiling function.

[0128] The reverse gradient generated by the mask can be expressed as:

[0129]

[0130] Note that the unrepresented method is still used in the forward propagation, and this re-representation is only used in the reverse gradient propagation. As a result, the network model is differentiable everywhere, the reverse gradient can be transmitted, and end-to-end training can be performed.

[0131] Step 4: Construct a loss function model;

[0132] The network model training relies on the loss function model, which needs to consider the similarity of the perceptual characteristics of the carrier image and the secret image, as well as the perceptual characteristics of the secret image and the reconstructed secret image. The similarity of the perceptual characteristics of the carrier image and the secret image determines the information hiding effect and security. The higher the similarity, the better the security and the better the hiding effect. The perceptual characteristics of the secret image and the reconstructed secret image determine the degree of realization of the communication function. The higher the similarity, the better the secret image transmission effect and the higher the accuracy of subsequent task processing at the receiving end. If only one side is considered, the network will tend to ensure the similarity of this perceptual characteristic and ignore the other side. In extreme cases, the generated secret image may not contain the secret image, resulting in communication failure, or it may only contain the secret image, resulting in third-party detection, theft and attack.

[0133] Therefore, the loss function model consists of two parts. The mean square error (MSE) of two images is used to measure the similarity of their perceptual characteristics. The smaller the MSE, the higher the similarity. The MSE calculation formula is as follows:

[0134]

[0135] Among them, x and y represent the two images whose perceptual characteristics are to be compared.

[0136] Assuming that C and C′ represent the carrier image and the secret image respectively, S and S′ represent the secret image and the reconstructed secret image respectively, the loss function can be expressed as:

[0137] Loss(C,C′,S,S′)=Loss_C+β*Loss_S=MSE(C,C′)+β*MSE(S,S′)

[0138] Among them, Loss_C and Loss_S represent the loss function of the similarity between the carrier image and the secret image, and the loss function of the similarity between the secret image and the reconstructed secret image, respectively. β represents the weight of the two, which is used to weigh the quality of the secret image and the reconstructed secret image. The larger β is, the more critical the quality of the reconstructed secret image is.

[0139] Step 5: Based on the network model and loss function model of the semantic weighted image information hiding method, a deep neural network training model is constructed;

[0140] Step 501: construct a data set;

[0141] The Tiny-ImageNet dataset is used. This dataset has 200 image classes, each of which has 500 training images, 50 validation images, and 50 test images. It is one of the most commonly used datasets in the field of computer vision.

[0142] To ensure that the network model has reasonable and uniform input, all images are first randomly cropped into a size of 64×64×3, and then the value range of RGB channels from 0 to 255 is mapped to the value range of 0 to 1, and the images are standardized to conform to the standard normal distribution with a mean of 0 and a standard deviation of 1, which speeds up the convergence of the network model. Standardization can be expressed as:

[0143]

[0144] Among them, x represents the image data, μ x represents the mean of the image, σ x Represents the standard deviation of the image.

[0145] 10,000 training images are randomly selected from the processed Tiny-ImageNet dataset as the training set, and the training set is divided into two groups to form secret image-carrier image pairs for training the network model.

[0146] Step 502: setting hyper parameters;

[0147] The training environment was built based on the UBUNTU operating system and the NVIDIA GTX1080 graphics card, and the network model was implemented using the PyTorch framework. The main hyperparameters were set as follows: 1) The batch size was set to 64, and the number of training epochs was set to 300; 2) The semantic preprocessing network, hidden network, and decryption network were all optimized using the Adam optimizer, and the learning rate was set to 0.001; 3) The quantization level L of the semantic importance map was set to 13; 4) The loss function weight β was set to 0.75, 1, and 1.25.

[0148] Step 6: Based on the network model and training model of the semantic weighted image information hiding method, a deep neural network test and system performance evaluation model is constructed;

[0149] Step 601: construct a network test model;

[0150] The 10,000 test images from the processed Tiny-ImageNet dataset are used as the test set, which is divided into two groups to form secret image-carrier image pairs for testing the network model. The test model hyperparameters are the same as those during training.

[0151] Step 602: construct a system performance evaluation model;

[0152] The performance of image information hiding system includes the load of carrier image, the similarity of perceptual characteristics between carrier image and secret image, and the similarity of perceptual characteristics between secret image and reconstructed secret image.

[0153] The carrier image load has objective performance indicators: absolute embedding capacity and relative embedding capacity. Absolute embedding capacity refers to the number of bytes of the secret image contained in the carrier image, and relative embedding capacity refers to the number of bytes of the secret image contained per pixel in the carrier image.

[0154] The similarity of the perceptual characteristics of two images includes objective performance indicators such as Average Pixel Deviation (APD), Average Pixel Square Deviation (APSD), Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

[0155] APD refers to the average deviation of the values ​​of each pixel and each RGB channel of the two images, and APSD refers to the square of the average deviation of the values ​​of each pixel and each RGB channel of the two images. The smaller the APD and APSD values, the more similar the perceptual characteristics of the two images are.

[0156] PSNR refers to the ratio of the maximum signal power to the signal noise power. The secret image embedded in the carrier image can be regarded as noise that affects the quality of the carrier image, and the carrier image embedded in the secret image can be regarded as noise that affects the quality of the secret image. The larger the PSNR value, the more similar the perceptual characteristics of the two images are. The calculation formula is as follows:

[0157]

[0158] Here, n represents the number of bits per pixel.

[0159] SSIM measures the correlation between different regions of an image from three aspects: brightness, contrast, and structure. The larger the SSIM value, the more similar the perceptual characteristics of the two images are. The calculation formula is as follows:

[0160]

[0161]

[0162]

[0163] SSIM(x,y)=[l(x,y) α c(x,y) β s(x,y) γ ]

[0164] Among them, x and y represent two images, l(x,y) represents brightness comparison, c(x,y) represents contrast comparison, s(x,y) represents structure comparison, and μ x and μy represents the mean of x and y, σ x and σ y represents the standard deviation of x and y, σ xy represents the covariance, c 1 、c 2 and c 3 It is a constant to avoid the denominator being 0. α, β and γ are generally set to 1, and the calculation formula is as follows:

[0165]

[0166] The degree of similarity of the perceptual characteristics of two images is measured by subjective performance indicators, such as the residual difference between the two images and the statistical characteristics of the histogram. The residual difference between the two images evaluates whether there is color difference and artifacts, etc. The smaller the residual difference, the better the visual effect. The statistical characteristics of the histogram evaluate the ability to resist third-party steganalysis. The closer the statistical characteristics of the histogram, the better the ability to resist steganalysis.

[0167] Taking the loss function weight β as 1 as an example, the convergence performance and convergence results of the training rounds (Epoch) are as follows: Figure 4 As shown in the figure, the loss function value of the network model decreases with the increase of the number of training rounds, and convergence is achieved after about 150 Epochs.

[0168] Taking the loss function weight β as 1 as an example, the performance and results of hiding and reconstruction effects and the number of training rounds are as follows Figure 5 As shown in the figure. The loss function value of the network model decreases with the increase of the number of training rounds, and convergence is achieved after about 150 Epochs. Among them, the network model of (a) is only initialized and not trained, and the generated secret image and the reconstructed secret image are both empty; (b) After the network model is trained for 1 Epoch, the generated secret image and the reconstructed secret image have taken shape; (c) After the network model is trained for 20 Epochs, the generated secret image and the reconstructed secret image have achieved good results, but the secret image is vaguely visible in the secret image, and there is a significant color difference between the secret image and the reconstructed secret image, and the details are lost; (d) After the network model training is completed, the carrier image and the secret image, the secret image and the reconstructed secret image are highly similar, and the difference can no longer be detected from the visual effect, realizing the image information hiding function.

[0169] The purpose of the present invention is to reduce the influence of the embedded secret image on the perceptual characteristics of the carrier image while ensuring the quality of the reconstructed secret image, thereby achieving a larger load and better hiding effect and improving the security of information hiding. In order to reflect the advantages of the present invention, objective performance indicators are compared with existing image information hiding methods that do not consider semantic weighting in terms of carrier image load, similarity between carrier image and perceptual characteristics of the carrier image and the secret image, and similarity between perceptual characteristics of the secret image and the reconstructed secret image.

[0170] The present invention embeds a secret image of size 64×64×3 into a carrier image of size 64×64×3. The absolute embedding capacity is 12288 bytes and the relative embedding capacity is 1 byte per pixel per channel, which is the same as the existing image information hiding method based on autoencoder without considering semantic weighting, but is superior to the image information hiding method based on generative adversarial network and has a larger carrier image load capacity.

[0171] The objective comparison results of the similarity of the image perceptual characteristics between the present invention and the existing image information hiding method without considering semantic weighting are shown in Table 1, where β represents the weight of the loss function. As the value of β decreases, the proportion of the similarity between the perceptual characteristics of the carrier image and the secret image in the loss function increases, the network model tends to generate secret images with better hiding effects, and the information hiding is more secure. Compared with the existing image information hiding method without considering semantic weighting, the secret images generated by the present invention have better quality in all β values ​​and all objective performance indicators, and are more similar to the perceptual characteristics of the carrier image. When the β values ​​are 0.75 and 1.25, the quality of the reconstructed secret images generated by the present invention is better than that of the existing image information hiding method without considering semantic weighting in all objective performance coordinates.

[0172] In order to intuitively reflect the effect of the present invention, subjective performance index evaluation is carried out from two aspects: the residual between the carrier image and the secret image, the residual between the secret image and the reconstructed secret image, and the histogram statistical characteristics between the carrier image and the secret image, the residual between the secret image and the reconstructed secret image.

[0173] Figure 6 The residuals between the carrier image and the secret image, and between the secret image and the reconstructed secret image of the present invention are shown, from left to right, respectively, the carrier image, the secret image, the residual between the carrier image and the secret image, the residual between the carrier image and the secret image magnified five times, the secret image, the reconstructed secret image, the residual between the secret image and the reconstructed secret image, and the residual between the secret image and the reconstructed secret image magnified five times. The visual effects of the carrier image and the secret image, and the secret image and the reconstructed secret image are similar, with small color difference. The residual between the carrier image and the secret image has nothing to do with the secret image, and the secret image and the reconstructed secret image are randomly distributed at all pixels.

[0174] Figure 7 The figure shows the comparison of the histogram statistical characteristics of the carrier image and the secret image, and the secret image and the reconstructed secret image of the present invention, from left to right, respectively, the carrier image, the secret image, the histogram statistical characteristics of the carrier image, the histogram statistical characteristics of the secret image, the secret image, the reconstructed secret image, the histogram statistical characteristics of the secret image, and the histogram statistical characteristics of the reconstructed secret image. The histogram statistical characteristics of the carrier image and the secret image, and the histogram statistical characteristics of the secret image and the reconstructed secret image are similar, and the differences are evenly distributed in all RGB channels, so the present invention has the ability to resist third-party steganalysis.

[0175] Table 1

[0176]

[0177] In summary, by implementing the image information hiding method based on semantic weighting of the present invention, the influence of the embedded secret image on the perceptual characteristics of the carrier image can be reduced while ensuring the quality of the reconstructed secret image, a larger load and better hiding effect can be achieved, the security of information hiding can be improved, the needs of secure transmission of massive secret information can be met, and powerful third-party steganalysis technology can be responded to. Compared with the existing image information hiding method that only processes from the low-order pixel level and does not consider semantic weighting, the present invention processes the secret image features from the high-order feature level according to the changes in the local semantic importance of the secret image, retains more features for areas with complex textures and contents for reconstruction, and cuts redundant features for areas with simple textures and contents to avoid affecting the perceptual characteristics of the carrier image.

[0178] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, they can also make several improvements and modifications, such as changing the basic structure of the network model, replacing the convolution module with other network structures such as U-Net and dense network, or applying the semantic preprocessing network of the present invention to video or text information hiding. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for hiding image information based on semantic weighting, the specific steps comprising: Step 1: Construct an image information hiding system model; The system model includes a transmitter and a receiver; The processing process of the whole system is as follows: 1) The sending end randomly selects the carrier image and the secret image, extracts features and performs semantic importance analysis on the secret image, generates a semantic importance map representing the semantic importance of different regions of the secret image, and then crops the secret image features according to the semantic importance map, retaining only the key features that affect the quality of the secret image reconstructed by the receiving end; 2) The sending end connects the semantically weighted, i.e., semantically cropped, secret image features with the carrier image on the channel, uses a deep neural network to achieve deep fusion and embedding, and generates a secret image with similar perceptual characteristics to the carrier image, and transmits the secret image to the receiving end through an open channel; 3) The receiving end receives the secret image, extracts the secret image features from it using a deep neural network, and recovers the reconstructed secret image; The main evaluation indicators of the system include the similarity of the perceptual characteristics of the carrier image and the secret image, and the similarity of the perceptual characteristics of the secret image and the reconstructed secret image; Step 2: Construct a network model of image information hiding method based on semantic weighting; The network model includes a semantic preprocessing network, a hidden network, and a decryption network. The sending end uses the semantic preprocessing network and the hidden network to embed the secret image into the carrier image to generate the secret image. The receiving end uses the decryption network to recover the reconstructed secret image from the secret image. The detailed network modeling steps are as follows: Step 201, constructing a semantic preprocessing network model; The purpose of the semantic preprocessing network is as follows: 1) to enlarge or reduce the secret image to ensure that it has the same size as the carrier image. In the present invention, the size of the secret image and the carrier image is 64×64×3, where 64 represents the number of pixels of the image width and height, and 3 represents the number of RGB channels of the image; 2) to extract the features of the secret image; 3) to generate a semantic importance map representing the semantic importance of different regions of the secret image, and to crop the secret image features according to the semantic importance map for channel connection and embedding with the carrier image; Its network model includes feature extraction module, semantic importance analysis module and feature clipping module; The feature extraction module takes a secret image of size 64×64×3 as input and outputs a secret image feature of size 64×64×65. The network structure consists of two convolutional modules (Conv Block). Conv Block is a multi-scale feature fusion network that extracts features of different scales of the image through multiple different convolution kernels and then performs feature fusion. The specific convolution kernel sizes are 3×3, 4×4, and 5×5, respectively. The number of convolution kernel channels is 50, 10, and 5, respectively. The convolution step size is 1, and the activation function is the ReLU function. After completing the feature extraction of these three scales, the extracted features are connected in the channel dimension. The semantic importance analysis module takes the secret image features of size 64×64×65 output by the feature extraction module as input and outputs a secret image semantic importance map of size 64×64×1. The network structure consists of two Conv Blocks and a convolution layer. The convolution kernel size of the convolution layer is 3×3, the number of convolution kernel channels and the convolution step are both 1, and the activation function is the Sigmoid function. The semantic information of each pixel of the secret image is first analyzed by the Conv Block, and then the semantic importance of each pixel is calculated by the convolution layer and mapped to a value between 0 and 1 by the Sigmoid function to obtain the semantic importance map of the secret image. The larger the value of the semantic importance map at a certain pixel, the more important the pixel is, and the more feature channels need to be retained and embedded in the carrier image for reconstruction. The feature cropping module takes the secret image features of size 64×64×65 output by the feature extraction module and the secret image semantic importance map of size 64×64×1 output by the semantic importance analysis module as input, and outputs the secret image features of size 64×64×65 after semantic weighted cropping. This module consists of mask generation and dot multiplication cropping. First, a semantic importance mask of the same size as the secret image feature is generated from the secret image semantic importance map, and then the mask is dot-multiplied with the secret image feature to retain the key features required for reconstructing the secret image and crop redundant features, that is, setting the feature value to 0, so as to ensure that while achieving a good secret image reconstruction effect, the impact of the embedded secret image features on the perceptual characteristics of the carrier image is reduced; Assuming that the secret image feature output by the feature extraction module is F(W,H,C), and the secret image semantic importance map output by the semantic importance analysis module is M(W,H,1), then the eigenvalues ​​of the secret image feature output by the feature cropping module should be expressed as: Where W and H represent the width and height of the image or feature, respectively, C represents the number of channels of the feature, F′ represents the secret image feature after semantic weighted cropping by the feature cropping module, (w,h,c) represents a position in the feature map, w=1,...,W, h=1,...H, c=1,...,C; Specifically, mask generation first requires quantizing and approximating the semantic importance map, which is divided into L quantization levels: Wherein, Q represents a quantization operation, l represents a certain quantization level, l=1,...,L; Mask generation then expands the semantic importance map M(W,H,1) into a semantic importance mask M′(W,H,C) of the same size as the secret image features. The value of each position of the semantic importance mask is 0 or 1. 0 means that the secret image feature point corresponding to the position needs to be cropped, and 1 means that the secret image feature point corresponding to the position needs to be retained. The larger the weight of the semantic importance map of a pixel, the more channels with the semantic importance mask value of 1 for the pixel, and the more secret image features are retained and embedded in the carrier image. The values ​​of each position of the semantic importance mask can be expressed as: Point product cropping multiplies the semantic importance mask with the secret image feature to achieve semantic weighting, retain the key features that affect the effect of reconstructing the secret image, and discard redundant features. The feature values ​​of the secret image feature after cropping can be expressed as: F′(w,h,c)=F(w,h,c)*M(w,h,c) Step 202: construct a hidden network model; The purpose of the hidden network is to embed and deeply fuse the secret image features with the carrier image, generate a secret image with similar perceptual characteristics to the carrier image, and transmit it in an open channel to avoid being detected, stolen, and attacked by a third party. Since the human sensory system is insensitive to subtle changes in high-frequency images or areas with complex textures and content, the hidden network is responsible for analyzing and determining these areas and embedding the secret image features. The hidden network first needs to connect the semantically weighted cropped secret image features of size 64×64×65 output by the semantic preprocessing network and the carrier image of size 64×64×3 on the channel to obtain a preliminary combination of the secret image and the carrier image of size 64×64×68, and then use this as input to output a secret image of size 64×64×3. The structure of its network model consists of five Conv Blocks and a convolutional layer. The convolution kernel size of the convolutional layer is 3×3, the number of convolution kernel channels is 3, the convolution step is 1, and the activation function is the Tanh function. First, the Conv Block is used to determine the area in the carrier image that is suitable for embedding the secret image features and realize the deep fusion of the two. Then, the Tanh function is used to map the fusion result to a value between -1 and 1 to obtain the secret image. Step 203: construct a decryption network model; The purpose of the decryption network is to process the received secret image, extract the embedded secret image features, and restore the reconstructed secret image. The restored reconstructed secret image should have similar perceptual characteristics to the sender's original secret image, so that the receiver can use the secret image for subsequent tasks and realize the communication function of information hiding. The decryption network receives a secret image of size 64×64×3 as input and outputs a reconstructed secret image of size 64×64×3. Its network structure consists of five Conv Blocks and one convolutional layer. The convolution kernel size of the convolutional layer is 3×3, the number of convolution kernel channels is 3, the convolution step is 1, and the activation function is the Tanh function. The Conv Block first analyzes the secret image features, extracts the embedded secret image features after semantic weighted cropping, fills and restores them, and then maps the extraction results to values ​​between -1 and 1 through the Tanh function to obtain the secret image. Determine the area in the carrier image that is suitable for embedding the secret image features and achieve deep fusion of the two, and then map the fusion result to a value between -1 and 1 through the Tanh function to obtain the reconstructed secret image; Step 3: Construct a reverse gradient propagation model based on semantic weighted image information hiding method; To achieve image information hiding, an end-to-end training network model is required. The training of the network model requires continuous iteration of reverse gradient propagation. Both the hidden network and the decryption network in the network model can achieve reverse gradient propagation. However, because the quantization and semantic importance graph channel expansion in the mask generation module of the semantic preprocessing network are not differentiable, the reverse gradient propagation is interrupted. The gradient before this module is zero everywhere and cannot be iterated. To solve this problem, the mask generation is re-expressed as: in, represents the ceiling function; The reverse gradient generated by the mask can be expressed as: Note that the unrepresented method is still used in the forward propagation. This re-representation is only used in the reverse gradient propagation. Therefore, the network model can be guided everywhere, the reverse gradient can be transmitted, and end-to-end training can be performed. Step 4: Construct a loss function model; The network model training relies on the loss function model, and needs to consider the similarity of the perceptual characteristics of the carrier image and the secret image, as well as the perceptual characteristics of the secret image and the reconstructed secret image. The similarity of the perceptual characteristics of the carrier image and the secret image determines the information hiding effect and security. The higher the similarity, the better the security and the better the hiding effect. The perceptual characteristics of the secret image and the reconstructed secret image determine the degree of realization of the communication function. The higher the similarity, the better the secret image transmission effect and the higher the accuracy of subsequent task processing at the receiving end. If only one party is considered, the network will tend to ensure the similarity of this perceptual characteristic and ignore the other party. In extreme cases, the generated secret image may not contain the secret image, resulting in communication failure, or it may only contain the secret image, resulting in third-party detection, theft and attack. Therefore, the loss function model consists of two parts. The mean square error (MSE) of the two images is used to measure the similarity of their perceptual characteristics. The smaller the MSE, the higher the similarity. The MSE calculation formula is as follows: Among them, x and y represent the two images whose perceptual characteristics are to be compared; Assuming that C and C′ represent the carrier image and the secret image respectively, S and S′ represent the secret image and the reconstructed secret image respectively, the loss function can be expressed as: Loss(C,C′,S,S′)=Loss_C+β*Loss_S=MSE(C,C′)+β*MSE(S,S′) Among them, Loss_C and Loss_S represent the loss function of the similarity between the carrier image and the secret image, and the loss function of the similarity between the secret image and the reconstructed secret image, respectively. β represents the weight of the two, which is used to weigh the quality of the secret image and the reconstructed secret image. The larger β is, the more critical the quality of the reconstructed secret image is. Step 5: Based on the network model and loss function model of the semantic weighted image information hiding method, a deep neural network training model is constructed; Step 501: construct a data set; The Tiny-ImageNet dataset is used. This dataset has 200 image classes, each of which has 500 training images, 50 validation images, and 50 test images. It is one of the most commonly used datasets in the field of computer vision. To ensure that the network model has reasonable and unified input, all images are first randomly cropped into a size of 64×64×3, and then the value range of RGB channels from 0 to 255 is mapped to the value range of 0 to 1. The images are standardized to conform to the standard normal distribution with a mean of 0 and a standard deviation of 1, which speeds up the convergence of the network model. The standardization can be expressed as: Where x represents the image data, μ x represents the mean of the image, σ x Indicates the standard deviation of the image; Randomly select 10,000 training images from the processed Tiny-ImageNet dataset as the training set, and divide the training set into two groups to form secret image-carrier image pairs, so as to train the network model; Step 502: setting hyper parameters; The training environment was built based on the UBUNTU operating system and the NVIDIA GTX1080 graphics card, and the network model was implemented using the PyTorch framework. The main hyperparameter settings are as follows: 1) The batch size (Batch Size) is set to 64, and the number of training rounds (Epoch) is set to 300; 2) The semantic preprocessing network, hidden network, and decryption network are all optimized using the Adam optimizer, and the learning rate is set to 0.001; 3) The semantic importance map quantization level L is set to 13; 4) The loss function weight β is set to 0.75, 1, and 1.25; Step 6: Based on the network model and training model of the semantic weighted image information hiding method, a deep neural network test and system performance evaluation model is constructed; Step 601: construct a network test model; The processed 10,000 test images of the Tiny-ImageNet dataset are used as the test set. The test set is divided into two groups to form secret image-carrier image pairs, so as to test the network model. The test model hyperparameters are the same as those during training. Step 602: construct a system performance evaluation model; The performance of image information hiding system includes the load of carrier image, the similarity of perceptual characteristics between carrier image and secret image, and the similarity of perceptual characteristics between secret image and reconstructed secret image. The carrier image load has objective performance indicators, namely, absolute embedding capacity and relative embedding capacity. The absolute embedding capacity refers to the number of bytes of the secret image contained in the carrier image, while the relative embedding capacity refers to the number of bytes of the secret image contained per pixel in the carrier image. The similarity of the perceptual characteristics of two images includes objective performance indicators such as Average Pixel Deviation (APD), Average Pixel Square Deviation (APSD), Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM). APD refers to the average deviation of the values ​​of each pixel and each RGB channel of the two images, and APSD refers to the square of the average deviation of the values ​​of each pixel and each RGB channel of the two images. The smaller the APD and APSD values ​​are, the more similar the perceptual characteristics of the two images are. PSNR refers to the ratio of the maximum signal power to the signal noise power. The secret image embedded in the carrier image can be regarded as noise that affects the quality of the carrier image, and the carrier image embedded in the secret image can be regarded as noise that affects the quality of the secret image. The larger the PSNR value, the more similar the perceptual characteristics of the two images are. The calculation formula is as follows: Where n represents the number of bits per pixel; SSIM measures the correlation between different regions of an image from three aspects: brightness, contrast, and structure. The larger the SSIM value, the more similar the perceptual characteristics of the two images are. The calculation formula is as follows: SSIM(x,y)=[l(x,y) α ·c(x,y) β ·s(x,y) γ ] Among them, x and y represent two images, l(x,y) represents brightness comparison, c(x,y) represents contrast comparison, s(x,y) represents structure comparison, and μ x and μ y represents the mean of x and y, σ x and σ y represents the standard deviation of x and y, σ xy represents the covariance, c1, c2 and c3 are constants, avoid the denominator to be 0, α, β and γ are set to 1, and the calculation formula is as follows: The similarity of the perceptual characteristics of the two images has subjective performance indicators: the residual and histogram statistical features of the two images are compared. The residual of the two images evaluates whether there is color difference and artifacts. The smaller the residual, the better the visual effect. The histogram statistical features evaluate the ability to resist third-party steganalysis. The closer the histogram statistical features, the better the ability to resist steganalysis.

2. The image information hiding method based on semantic weighting according to claim 1, characterized in that: The semantic preprocessing network model described in step 2; Since different regions of the secret image have different semantic importance: regions with complex textures and content have higher semantic importance and need to retain more features for reconstruction; The semantic importance of areas with simple textures and contents is low, and only some features are needed to achieve good reconstruction effects. Redundant secret image feature embedding will affect the perceptual characteristics of the carrier image. Step 2 processes the secret image from a high-order feature level, generates a semantic importance map representing the semantic importance of different areas of the secret image, generates a semantic importance mask consisting of 0 and 1 based on the semantic importance map, and crops the secret image features for channel connection and embedding with the carrier image.

3. The image information hiding method based on semantic weighting according to claim 1, characterized in that: The reverse gradient propagation model described in step three; since the realization of image information hiding requires end-to-end training of the network model, the training of the network model requires continuous iteration of reverse gradient propagation, and the quantization of the mask generation module of the semantic preprocessing network in the network model and the channel expansion of the semantic importance graph are not differentiable, resulting in the interruption of reverse gradient propagation. The gradient before this module is zero everywhere and cannot be iterated. It is proposed to re-represent the mask generation so that reverse gradient propagation can be performed.

Images