Information hiding method based on deep learning

Abstract

The invention provides an information hiding method based on deep learning. The robustness and the concealment of digital image watermarks are effectively improved. The method mainly comprises the following steps of: firstly, extracting interest points of an image by utilizing an interest point extraction network; secondly, two-step screening is carried out through an interest point screening network, and sparsity and robustness of interest points are ensured; thirdly, embedding watermark information around the robust interest point through a watermark encoder, and introducing an adversarial training network in the training process to improve the robustness of the watermark; and finally, extracting hidden watermark information by using a watermark decoder, and screening potential sub-watermark blocks through the sub-watermark block screening model. According to the method, the data is embedded into the robust feature of the image, and the robustness of the model is enhanced by adopting the adversarial training network, so that various attacks are effectively resisted, and the accuracy and security of watermark information are ensured.

Description

Technical Field

[0001] The present invention relates to information hiding technology, and in particular to an information hiding method based on deep learning. Background Art

[0002] The popularization and application of multimedia technology has brought a lot of convenience to society, but it has also brought a lot of risks to information protection. Digital image information is widely used in communication, but this information is easily tampered with illegally. Digital image watermarking algorithm is a powerful tool for copyright protection. Existing watermarking methods can be mainly divided into spatial domain, frequency domain, hybrid algorithm and learning-based model. Spatial domain watermark hides information in the invisible area of ​​the cover image, but when attacked by color difference, it may cause decoding errors due to the loss of information.

[0003] Information hiding technology is a technology that embeds information data with confidentiality value into carriers such as ordinary images, audio or strings through technical means to achieve the hiding of secret data. Different from the true meaning hidden by traditional cryptography, information hiding technology aims to hide the existence of information and restore the hidden information through corresponding algorithms to achieve the purpose of secret information transmission. According to the different hidden objects, information hiding technology can be divided into string information, image information and audio information. The image used to hide secret data is called carrier image, and the information or image recovered from it is called secret information or image. Information hiding technology has been widely used in many fields such as copyright protection and information authentication.

[0004] When attacked by color difference changes and cropping, it will cause decoding errors. Frequency domain watermarking algorithms embed information by applying the watermark signal to the frequency. The discrete cosine transform (DCT) based method and the watermark are dispersed over the entire spatial image after the encoded image is inversely transformed. After encoding the image, the watermark is dispersed over the entire spatial image. They achieve excellent concealment, but are not strong enough in terms of anti-cropping and compression.

[0005] For the hybrid watermarking algorithm based on SIFT (Scale Invariant Feature Transform) and ORB (Oriented Fast and Rotated Brief), they use the adaptive region selection method based on texture information and feature points ORB to locate texture information and feature points to locate the region to embed the DCT watermark, which has better concealment and invisibility, but the watermark capacity is limited, and the feature points often move under attack. But the watermark capacity is limited, and the feature points often move under attack. The watermarking algorithm based on deep learning significantly improves the robustness under different attacks and the invisibility under different attacks.

[0006] Watermarks can be divided into spatial domain and frequency domain watermarks according to the embedding method.

[0007] Spatial domain digital watermark embeds watermark information by directly modifying the pixel values ​​of the image. The least significant bit modification (LSB) algorithm is simple and easy to operate, so it is widely used. However, this type of algorithm has poor robustness after being attacked by noise, and is easily removed by attacks such as filtering. The frequency domain method mainly transforms the image in the frequency domain and then changes it at a specific frequency, making the watermark more imperceptible and robust. Correspondingly, the algorithm is more complex and has stronger ability to resist different types of attacks. Commonly used methods include the spatial domain watermarking algorithm SVD (Singular Value Decomposition) based on singular value decomposition and the frequency domain watermarking algorithm based on DCT (Discrete Cosine Transform), which preprocesses the watermark information and embeds it into the singular values ​​of the image. Frequency domain-based algorithms generally have good hiding properties, but have low information capacity. With the emergence of deep learning technology, the learning ability of neural networks has made great progress in watermarking technology. Watermarking algorithms based on learning models have made great progress in information hiding capacity and resistance to different noises. These algorithms are mainly divided into generative, adversarial and staged algorithms. Generative algorithms directly use adversarial networks to generate images, while staged algorithms achieve better embedding effects and robustness through phased training of encoders and decoders.

[0008] An excellent information hiding algorithm needs to perform well in three aspects: embedding capacity, robustness, and imperceptibility. The embedding capacity is the ratio of the length of the embedded secret information to the carrier image. The larger the ratio, the larger the embedding capacity and the stronger the algorithm's ability to carry information. Robustness refers to the ability to extract secret information. The ultimate goal of information hiding is to transmit information secretly. If the extracted information has a large error, or even cannot recover the information to be conveyed, then the robustness of the algorithm is very poor. Imperceptibility represents the difference between the algorithm after embedding information and the original image without embedded information. The smaller the difference, the smaller the impact of information embedding on the original image, the better the imperceptibility, the more difficult it is to distinguish between the carrier image and the secret image, and the better the effect.

[0009] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0010] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide an information hiding method based on deep learning to solve the problems of poor resistance to motion distortion, difficulty in targeted local embedding and poor resistance to complex noise in deep learning hidden networks.

[0011] To achieve the above object, the present invention adopts the following technical solutions:

[0012] A deep learning-based information hiding method comprises the following steps:

[0013] S1. Extract interest points of the image using an interest point extraction network;

[0014] S2. The extracted interest points are screened through an interest point screening network to obtain sparse and robust interest points, wherein the interest point screening network includes two steps of screening: the first step of screening is to calculate the Euclidean distance between the interest points and compare it with a preset threshold. If the Euclidean distance is less than the preset threshold, one of the interest points is discarded; the second step of screening is to delete unstable interest points by comparing the coordinate changes of the same interest point before and after the attack on the images after different attack processing;

[0015] S3. embedding watermark information in the area around the selected interest point through a watermark encoder; in the training process of the watermark encoder, the watermark encoder and a discriminator for attempting to identify the watermark form an adversarial training network, and introducing simulated noise and different attacks as a distortion layer in the adversarial training network;

[0016] S4. The hidden watermark information is extracted from the encrypted image through the watermark decoder, and the potential sub-watermark blocks are further screened out through the sub-watermark block screening model.

[0017] Furthermore, in step S1, the interest point extraction network uses a Superpoint model, and the Superpoint model includes a shared encoder for processing an input image and reducing its dimension; the shared encoder includes a convolutional layer, spatial downsampling and a nonlinear activation function.

[0018] Furthermore, in step S1, the interest point extraction network includes two decoders, which are respectively used to generate interest points based on probability maps and descriptors for characterizing features of interest points. The pixel value of each point in the image represents the probability of the point becoming an interest point.

[0019] Furthermore, in step S3, the watermark encoder adopts a structure similar to U-net, and uses a ConvBNRelu layer including a convolution layer, a normalization layer and an activation layer to process the carrier image; wherein the preprocessed information, the upsampled carrier image and the original carrier image are stacked together in the channel dimension; the stacked image is sent to the ConvBNRelu layer for downsampling operation, the image size is reduced to the size of the original carrier image, and the number of channels is restored to 3 channels; the watermark information is embedded in the pixel image by the method of upsampling first and then downsampling.

[0020] Furthermore, in step S3, the discriminator includes three ConvBNRelu layers with a step size of 1, a global average pooling layer and a linear connection layer, and the ConvBNRelu layer includes a convolution layer, a normalization layer and an activation layer.

[0021] Furthermore, in step S3, during the training process of the watermark encoder, the image output by the encoder is compared with the original image, and different types of losses are calculated to perform gradient descent, thereby better guiding the generation of images with hidden information.

[0022] Furthermore, in step S4, the loss function is a weighted combination of L1 loss, mean square error (MSE) loss, adversarial network loss, and structural similarity loss, wherein the L1 loss is used to calculate the similarity between the encoded image and the original image, the mean square error (MSE) loss is used to measure the difference between the encoded information and the decoded information, the adversarial network loss is used to reflect whether the discriminator can distinguish the difference between the embedded images, and the structural similarity (SSIM) loss is used to maintain the structural similarity between the encoded image and the original image.

[0023] Furthermore, in step S3, a small rectangular area is cropped around the screened interest points to embed watermark information.

[0024] Further, in step S4, the watermark decoder uses ConvNext as a decoder;

[0025] Among them, a threshold is first set according to the statistical characteristics of the decoder output. During the actual watermark extraction, statistical data is calculated from the output tensor obtained by the decoder, and the statistical data is compared with the threshold to determine the watermarked and non-watermarked areas.

[0026] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the information hiding method based on deep learning is implemented.

[0027] The present invention has the following beneficial effects:

[0028] The present invention proposes an information hiding method based on deep learning, which realizes the high robustness embedding and extraction of watermarks in images through innovative information hiding network design and training strategies. The method of the present invention extracts and screens robust points of interest, embeds watermarks at the robust features of the image, and ensures the stability and accuracy of the watermark in the face of geometric distortion and high-intensity attacks. By introducing simulated noise and multiple attacks through adversarial training networks, the concealment of the watermark and its resistance to different attacks are further improved. In addition, the watermark decoder is combined with a sub-watermark block screening model to effectively extract watermark information from the encrypted image, ensuring the accuracy and consistency of the extraction process. The present invention can embed more information while maintaining high robustness, and has the ability to handle multiple tasks, and can embed multiple pieces of information of different lengths.

[0029] The advantages of the present invention compared with the prior art are:

[0030] 1. The large-capacity deep learning information hiding network proposed in the present invention can embed more information while ensuring the robustness of the network, and achieves extensive robustness against different attacks through one-time training.

[0031] 2. By designing a two-step screening procedure and a method for distinguishing decoded values, the present invention avoids omission and interaction of watermark sub-blocks, which greatly reduces the error rate and improves the extraction efficiency.

[0032] 3. The present invention further ensures the robustness of the algorithm by selecting sub-blocks centered on the point of interest to embed the watermark. By adding simulated noise, adjusting the HSV color space and other attacks in the adversarial network, the quality of the encoded image and its robustness to large-scale cropping and other geometric deformations are improved.

[0033] 4. Compared with the existing deep learning information hiding algorithms, the model proposed in this invention has the ability to handle multiple tasks. Specifically, the embedded information can be expanded to multiple different information, and the length of the information is also longer than that of the existing algorithms.

[0034] 5. The present invention can set a threshold of watermark intensity and thus determine whether a sample is a copy according to the degree of information loss during the copying process.

[0035] In general, the present invention performs well in terms of invisibility, robustness and information capacity, and provides an effective solution for copyright protection and secure transmission of digital image information.

[0036] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an algorithm framework diagram of the information hiding method based on deep learning in an embodiment of the present invention.

[0038] Figure 2 The median filtering experimental results of the embodiments of the present invention with kernels of different sizes on three data sets;

[0039] Figure 3 The box filtering experimental results of different sized kernels on three data sets according to the embodiment of the present invention are shown;

[0040] Figure 4 The bilateral filtering experimental results of the embodiments of the present invention with kernels of different sizes on three data sets;

[0041] Figure 5 Comparison of the number of feature points of the three algorithms under different sub-watermark block sizes;

[0042] Figure 6 Comparison of the accuracy before and after feature point selection and PSNR values ​​under different attacks in the embodiment of the present invention;

[0043] Figure 7 The accuracy results of different types of noise attacks under three data sets in the embodiment of the present invention are shown;

[0044] Figure 8 It is the image comparison at different stages in the whole process of the embodiment of the present invention;

[0045] Fig. 9 This is the point distribution generated by the feature point generation model of the embodiment of the present invention for ordinary image processing. DETAILED DESCRIPTION

[0046] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0047] The present invention proposes an information hiding method based on deep learning, and designs an information hiding network based on deep learning that has watermark regional adaptability, high robustness, and resistance to motion distortion. A deep learning watermark hiding scheme with regional adaptability and resistance to motion geometric distortion is implemented. The present invention provides an innovative watermark framework that achieves accuracy in the data embedding and extraction process, good hiding, and resistance to different types of attacks by hiding data at image robust features and screening.

[0048] See also Figure 1 , an embodiment of the present invention provides an information hiding method based on deep learning, comprising the following steps:

[0049] S1. Extract interest points of the image using an interest point extraction network;

[0050] S2. The extracted interest points are screened through an interest point screening network to obtain sparse and robust interest points, wherein the interest point screening network includes two steps of screening: the first step of screening is to calculate the Euclidean distance between the interest points and compare it with a preset threshold. If the Euclidean distance is less than the preset threshold, one of the interest points is discarded; the second step of screening is to delete unstable interest points by comparing the coordinate changes of the same interest point before and after the attack on the images after different attack processing;

[0051] S3. embedding watermark information in the area around the selected interest point through a watermark encoder; in the training process of the watermark encoder, the watermark encoder and a discriminator for attempting to identify the watermark form an adversarial training network, and introducing simulated noise and different attacks as distortion layers in the adversarial training network to improve the robustness of the watermark;

[0052] S4. The hidden watermark information is extracted from the encrypted image through the watermark decoder, and the potential sub-watermark blocks are further screened out through the sub-watermark block screening model.

[0053] In a preferred embodiment, in step S1, the interest point extraction network uses a Superpoint model, which includes a shared encoder for processing an input image and reducing its dimension; the shared encoder includes a convolutional layer, spatial downsampling and a nonlinear activation function. The interest point extraction network includes two decoders, which are used to generate interest points based on a probability map and descriptors for characterizing the features of interest points, respectively. The pixel value of each point in the image represents the probability of the point becoming an interest point. The watermark encoder adopts a structure similar to U-net, and uses a ConvBNRelu layer including a convolutional layer, a normalization layer and an activation layer to process the carrier image; wherein the preprocessed information, the upsampled carrier image and the original carrier image are stacked together in the channel dimension; the stacked image is sent to the ConvBNRelu layer for downsampling operation, the image size is reduced to the size of the original carrier image, and the number of channels is restored to 3 channels; the watermark information is embedded in the pixel map by upsampling first and then downsampling to enhance the hiding ability of the algorithm and the flexibility of selecting the embedding position.

[0054] In some embodiments, the ConvBNRelu layer includes a series of convolutional layers with a specific step size, a batch normalization method BN layer and an activation function. The stacking of the convolutional layers can control the up and down sampling to control the size and receptive field of the tensor. The BN layer uses normalization to make the gradient descent stable, so that the neural network can stabilize the selection of hyperparameters. The activation function enables the network to have the ability of nonlinear fitting.

[0055] In a preferred embodiment, in step S3, a small rectangular area is cropped around the screened interest points to embed watermark information.

[0056] The discriminator of the adversarial training network includes three ConvBNRelu layers with a step size of 1, a global average pooling and a linear connection layer, and the ConvBNRelu layer includes a convolution layer, a normalization layer and an activation layer.

[0057] In a preferred embodiment, in step S3, during the training process of the watermark encoder, by comparing the image output by the encoder with the original image, different types of losses are calculated to perform gradient descent, thereby better guiding the generation of images with hidden information. The loss function is a weighted combination of L1 loss, mean square error (MSE) loss, adversarial network loss, and structural similarity loss, wherein L1 loss is used to calculate the similarity between the encoded image and the original image, the mean square error (MSE) loss is used to measure the difference between the encoded information and the decoded information, the adversarial network loss is used to reflect whether the discriminator can distinguish the difference between the embedded images, and the structural similarity (SSIM) loss is used to maintain the structural similarity between the encoded image and the original image.

[0058] In a preferred embodiment, the watermark decoder uses ConvNext as a decoder. In step S4, a threshold is first set according to the statistical characteristics of the decoder output, and during actual watermark extraction, statistical data is calculated from the output tensor obtained by the decoder, and the statistical data is compared with the threshold to determine the watermarked and non-watermarked areas.

[0059] The information hiding network in the embodiment of the present invention obtains universal resistance to different attacks through one training, and performs well in geometric distortion and high-intensity attacks. First, the present invention uses Superpoint to extract the points of interest of the image, and designs two screening steps to obtain sparse and robust points of interest: the first step is to screen dense points with lower probability values ​​to avoid overlap between them, and the second step is to delete unstable points by calculating their displacement after being attacked, so as to ensure that the watermark can be correctly detected during the extraction process. Secondly, the present invention cuts a small rectangular area around the robust points of interest to embed the watermark. During the training process of the sub-block encoder, the present invention introduces simulated noise and different attacks as distortion layers in the adversarial network to further improve the robustness. In the watermark extraction process, the present invention uses ConvNext as a decoder, and sets a threshold according to the statistical results to obtain the decoding result, so as to ensure the accuracy and consistency of the result. At the same time, the information loss of the watermark with different intensity thresholds during the copying process of different times can be used to detect the copy of the printed image, thereby ensuring the security during use.

[0060] The embodiment of the present invention provides a deep learning information hiding method with local feature point positioning, large capacity and high robustness. Its innovations include: (1) an upsampling convolution design different from traditional deep learning is designed to expand the receptive field and reduce information loss to obtain a robust encoder, so that the sub-image after cutting can be embedded and accurately extracted. Fixed-bit encrypted information is embedded in a single channel during training. The upsampling convolution design is adopted to provide more space for information embedding. (2) A robust feature positioning network is designed and trained and a primary screening is performed to select the area with robust features in the carrier image, and a non-robust point elimination algorithm is performed by designing adjacent point screening and image attack to obtain the final robust feature point. The existing algorithm channel convolution is improved to a repeated operation of information scale, which not only reduces the model parameters but also expands the application scenario of the algorithm. (3) The robust feature point area is fixed-size cut and encoded by a decoder to obtain the entire encrypted image, and a potential decoding module is obtained by using a screening scheme during the decryption process, and then a decoding operation is performed. By adding guiding edge information into the training process, the threshold of watermark intensity can be adjusted to detect the degree of information loss in different copy times.

[0061] In the embodiment of the present invention, the feature point generation model is used to predict the original image, and the region with stable features is extracted and screened to embed the watermark information, thereby enhancing the robustness and hiding ability of the watermark. The information embedding network structure is changed according to the input image size, and a convolution method of upsampling first and then downsampling is adopted, and a specific size region around the feature point is used as input to achieve stable information embedding. The feature point extraction model and the targeted embedding extraction network can ensure that the information is embedded in the stable features of the pattern in a sparse manner, which improves the ability of the pattern to resist large-area cutting and composite noise attacks. At the same time, the design concept of the present invention also realizes a better local watermarking solution.

[0062] The specific embodiments of the present invention are further described below.

[0063] The present invention proposes a new watermark framework, which can obtain universal resistance to different attacks through one training, and performs well in geometric distortion and high-intensity attacks. First, the present invention extracts the points of interest of the image through Superpoint, and designs two screening steps to obtain sparse and robust points of interest: the first step is to screen dense points with low probability values ​​to avoid their overlap, and the second step is to remove unstable points by calculating the displacement after being attacked, which ensures that the watermark can be correctly detected during the extraction process. Secondly, the present invention crops small rectangular areas around the robust points of interest to embed the watermark. In the process of training the sub-block encoder, the present invention introduces simulated noise and different attacks as distortion layers in the adversarial network to further improve the robustness. In the watermark extraction process, the present invention uses ConvNext as a decoder, sets a threshold by statistical results, and obtains the decoding result. The decoding result is obtained by statistical results. This ensures the accuracy and consistency of the results.

[0064] The following introduces in detail the large-capacity, high-robustness deep learning hidden network proposed in the present invention from five parts: interest point generation network, interest point screening network, adversarial training network, encoder and decoder.

[0065] Point of interest generation part

[0066] The goal of this part is to reduce the loss of watermark information under attack and correctly extract the watermark without knowing the watermark coordinates. Therefore, the present invention reduces information loss by embedding the watermark into the robust feature area of ​​the image and locating the robust feature area by the generated interest points, which makes it possible to pass without the original image. Therefore, the present invention uses the Superpoint model to extract interest points, which is proven to be robust to noise, motion blur and brightness changes. The model has a shared encoder to process and reduce the dimensionality of the input image. The purpose of the encoder is to reduce the dimensionality of the input image.

[0067] The dimension of the input image. The input image passes through the convolution layer, spatial downsampling and nonlinear activation function. The input image is decoded by two decoders to generate interest points and descriptors. The probability that each pixel value of the output map corresponds to an interest point is generated, and the probability map is as follows Fig. 9 shown.

[0068] Points of interest filter section

[0069] The initial interest points extracted by the interest point extraction module are scattered, robust and numerous, but individual feature points may become unstable during attacks. Therefore, the present invention designs a screening network to eliminate unstable feature points to ensure the accuracy of decoding. This method designs a two-step screening method for the situation where some points are locally dense, causing the embedded watermark blocks to overlap, and a small number of points disappear or move after attacks of different intensities, so as to further improve the robustness of interest points and avoid point overlap.

[0070] In the first step of screening, some robust feature points are close to each other, which may cause the watermark blocks to overlap and thus cause decoding errors. In order to reduce the density of points and maintain the sparsity of the watermark, the present invention calculates the Euclidean distance D between two feature points and compares it with the threshold δ m Compare, if D is less than δ m If the point is not found, the point is discarded, thus retaining the only robust feature point in a certain area.

[0071] In the second step of screening, a small number of interest points will disappear or move after receiving the attack, resulting in omissions and decoding errors. In order to ensure the stability of the interest points, the present invention uses different types of attacks to process the original image before and after the watermark is embedded, and extracts robust interest points respectively. By measuring the coordinate changes of the same feature point after different attacks, the feature area that meets the conditions can be screened out.

[0072] Information embedding part

[0073] The encoder and decoder used in this method are trained separately, and an encoder with excellent encryption performance is obtained by training with image blocks of the same size as the sub-block randomly cropped from a public dataset.

[0074] This method adopts a structure similar to U-net. Due to the different sizes of input, the encoder consists of a convolution layer, a normalization layer and an activation layer. The present invention uses ConvBNRelu to process the carrier image. The preprocessed information, the upsampled carrier image and the original carrier image are stacked together in the channel dimension and sent to the ConvBNRelu layer for downsampling. This downsampling operation is similar to the upsampling operation of the carrier. The size of the image is downsampled to the size of the original carrier image, and the number of channels is restored to 3 channels. In this way, the sub-image block with the final embedded information is obtained. After processing, the information is secretly hidden in the cover image and is difficult to extract. Since this experiment adopts the method of upsampling first and then downsampling, more information can be embedded in the pixel map, so the algorithm has a stronger hiding ability and has the advantage of being able to freely choose the embedding location.

[0075] Discriminant network part

[0076] The discriminant network is mainly used to guide the embedding of watermark pixels during the training process so that the discriminator cannot correctly identify whether the image contains hidden information. The discriminator is mainly composed of three ConvBNRelu layers with a step size of 1, a global average pooling layer, and a linear connection layer. By comparing the output image with the original image, different types of losses can be obtained, so that gradient descent can be performed to better guide the generation of images with hidden information.

[0077] Information extraction part

[0078] After the encrypted image is obtained, the information is extracted through the decoder. In the decoding process, the potential sub-watermark blocks can be screened out using the same screening model, and the values ​​obtained by the decoder are screened. The present invention uses the ConvNext network to recover the hidden information of the encoded image. A 1×m tensor (m=2 in the experiment) is obtained from the input sub-image block, and the decoding result can be judged by comparing the value of the tensor. The decoding result is judged by comparing the value of the tensor and the calculated threshold. By randomly cutting 1000 data sets of size 64×64, the present invention calculates the mean and variance. Through the decoded value of the decoder network, according to the 3σ principle, the present invention sets two thresholds of γ1=0.01 and γ2=0.996, and the decoded value in (0-γ1) is determined to be 0, and when the value is higher than γ2, it is 1. When their decoded values ​​∈(γ1, γ2), these blocks are determined to be not embedded and are abandoned.

[0079] Loss function part

[0080] The present invention designs different loss functions in order to improve the invisibility and robustness of the image watermark, and adopts L1 loss. The loss of this method is to improve the accuracy of decoding and image quality. L1 calculates the similarity between the encoded image and the original image. It shows the average error range of the predicted value and the average error range of the predicted value, which can ensure the visual similarity between the encoded image and the original image at the pixel level. At the same time, the present invention adds mean square error loss to measure the difference between the encoded information and the decoded information. Since MSE is the average of the sum of the squares of the encoded information and the decoded information, the accuracy of decoding can be guaranteed during the training process. At the same time, the present invention adds an adversarial network loss, which also reflects whether the discriminator can distinguish the differences between the embedded images. The structural similarity (SSIM) loss is described as L SSIM , represents the similarity of image structure, and the final loss function is as follows: (λ1,λ A λ M ,λ s are the weights of their respective losses).

[0081] L=λ1L1+λ M L M +λ A L A +λ S L SSIM

[0082] According to existing research, it has been confirmed that areas of an image with rich texture information are more concealed than flat areas, because the human eye is less likely to perceive changes in image information. A common problem with deep learning is that the learning process lacks interpretability. Under the control of the loss function, deep learning models may learn in an unexplainable direction. In contrast, traditional information hiding algorithms have established many models with strong interpretability, which can achieve good imperceptibility and robustness through clever design. In addition, recent research results show that deep learning algorithms can achieve better results when combined with traditional features. Therefore, this innovation chooses to focus the embedded information on stable features based on the existing deep learning model, thereby achieving a good information embedding process through a two-stage steganography scheme.

[0083] Experimental Section

[0084] This section first introduces the image quality evaluation system, hardware facilities and data sets in different dimensions, and then attacks the watermarked image through different types of hybrid attacks to illustrate the rationality of the present invention through the decoding accuracy. This experiment compares with similar algorithms in terms of embedded watermark capacity, and measures the imperceptibility of the image through relevant indicators to reflect the advantages of this algorithm.

[0085] Evaluation indicators

[0086] The method of measuring the covert carrier generally uses the Peak Signal-to-Noise Ratio (PSNR) to compare the difference between the watermarked image and the original image. The Peak Signal-to-Noise Ratio is usually used to evaluate the imperceptibility of the image after embedding the watermark. The larger the PSNR value, the smaller the image distortion and the better the imperceptibility.

[0087]

[0088] Structural similarity (SSIM) evaluates the accuracy of extracted information. The present invention uses the bit error rate to measure it, that is, the number of erroneous bits in the extracted information compared with the original information. The lower the bit error rate, the more accurate the extracted information is and the better the robustness of the algorithm. l(x,y), c(x,y), s(x,y) are brightness similarity, contrast similarity and structural similarity respectively. α, β, γ are weighting coefficients, generally 1.

[0089] SSIM(x,y)=[l(x,y)] α [c(x,y)] β [s(x,y)] γ

[0090] The bit error rate (BER) is used to evaluate the difference between the extracted information and the original embedded information. The larger the bit error rate, the worse the algorithm effect.

[0091] Experimental parameters

[0092] The experimental platform of the present invention is implemented based on PyTorch and accelerated using a GeForce RTX 3090 graphics card. The present invention uses the COCO val2014 dataset and adopts 30,000 datasets for training, of which 85% are randomly selected for training and 15% are tested. The test set is performed by randomly cutting 64×64 sub-image blocks from the image, and the total number of iterations is 300 times. The mini-batch size is set to 16. The Adam optimizer is used in all experiments of the present invention, and the initial learning rate is 0.001. In the experiment, the present invention uses the ImageNet and Monet2phone datasets as comparisons to prove that the algorithm has good adaptability to different types of images. The algorithm has good adaptability to different types of images.

[0093] Experimental Results

[0094] Hybrid attack experiment:

[0095] In order to illustrate the robustness of this method under mixed attacks, this method uses different types of attacks to test the robustness of the image. The types of attacks are shown in the following table, which are divided into geometric attacks, noise attacks, filtering attacks and mixed attacks.

[0096] The mean filter is a common filter in image processing, mainly used to smooth noise. It uses the average value of surrounding pixels to achieve the effect of smoothing noise. Since most of the Monet2photo datasets are complex and colorful landscape images, more information will be lost when the kernel is larger. Its accuracy is 90.01%, the lowest among the three datasets, but still higher than 90%. In box filtering, the accuracy of the three datasets shows good consistency, and even when the noise intensity reaches 0.05, the accuracy exceeds 90%. Since bilateral filtering also produces more serious losses for complex images, as the filtering effect gradually increases, the accuracy of the Monet2photo dataset drops to 91.33%.

[0097] In all attacks, the accuracy of different data sets decreases with the increase of filtering strength, but overall, they all maintain good consistency. In addition, when the strength is 0.05, most of them can maintain an accuracy of more than 90%, showing the good robustness of the algorithm. Noise attack: In this experiment, the present invention uses three different noises to attack the data set to verify its robustness. The noise includes speckle noise, salt and pepper noise, and Gaussian noise, each of which is tested with different intensities. After the noise attack, the above-mentioned positioning network and decoding rules are used to extract the target information, and the accuracy of watermark extraction is the criterion for this experiment.

[0098] Table 1 shows the different accuracies under different conditions. According to the experimental results, the method has strong resistance to the three tested noises, and still maintains an accuracy of more than 89.6% when the intensity reaches 0.05. Since Gaussian noise on the image will cause interference of light and dark points, which greatly reduces the image quality and affects the image feature extraction, the recognition and extraction accuracy of the points of interest are most seriously affected. When the noise intensity is 0.01 and 0.02, the accuracy is most affected.

[0099] Table 1 Complex attack experiments

[0100] Attack Types HkDJ Two-stage GM DADW The present invention No attack 100.0 100.0 99.0 100.0 99.5 JPEG(Q=50) 63.0 76.2 75.9 81.7 85.2 Clipping (0.09) - 91.9 - 93.5 96.7 Clipping (0.35) 88 - - - 94.3 Gaussian blur (sigma = 2) 96.1 98.6 - 60.0 96.6 Salt and pepper noise (0.05) 61.2 - - - 92.9 Salt and pepper noise (0.05) 88.9 92.8 87 95.7 89.6 Gaussian noise (0.05) 61.6 91.6 86.1 - 91.9 Mean filter (d = 5) 82.1 91.1 79.2 - 94.3 Bilateral filtering (d=6) 81.3 90.6 - - 93.9 Box filter (d = 6) 82.3 90.1 - - 91.2 Size adjustment (p=0.5) - 92.8 - 67.1 93.2 Resizing + Salt and Pepper Noise 52.9 89.7 - - 90.5 Gaussian blur + bilateral filtering 44.8 88.3 - - 90.1 Cropping + box filtering 50.6 84.9 - - 89.6

[0101] Geometry attack experiment:

[0102] Since this method adopts local embedding of watermarks and uses feature points to perform geometric correction on the image during decoding, it can resist some complex geometric deformation attacks. In the following experiments, as shown in Table 2, the present invention adopts different angles of rotation and different scales of scaling to form complex geometric attacks, and tests the decoding accuracy. It can be seen that under a certain degree of angle rotation and a certain degree of scaling, the decoding accuracy remains above 90%, proving that under complex attacks, this method still maintains a certain decoding stability and good resistance to geometric attacks.

[0103] Table 2 Composite geometric attack experiments

[0104] Rotation 5° 10° 15° 20° 1 96.5 95.4 94.2 92.01 0.9 94.3 93.9 93.6 91.6 0.8 93.3 92.8 91.7 90.3 0.7 88.1 87.9 87.1 86.6

[0105] Noise attack experiment:

[0106] In order to illustrate the superiority of the algorithm's stable feature point embedding watermark method and its effectiveness in the screening step, the present invention uses noises of different types and intensities to test the accuracy of watermark decoding, such as Figure 7As shown in the figure. It can be seen that the accuracy rate Acc gradually decreases as the intensity of the same noise increases, but generally remains above 90% when the intensity is 0.05, and it can be seen that the algorithm maintains good resistance to three different types of noise. At the same time, this experiment uses three different types of data sets, namely COCO2014, ImageNet and Monet2photo. Some tests were done on images under different scenes, and it was found that there was not much difference between the randomly tested images, which proves that this algorithm has good adaptability to different scenes and types of images.

[0107] Similarly, in order to prove the effectiveness of this scheme in the feature screening process, Figure 6 As shown, this experiment compares the accuracy before and after the feature point screening. It can be seen that in the unscreened group (w / o), the average accuracy is about 80%, while the group after feature screening generally maintains an accuracy of more than 90% in various attacks. Therefore, the present invention proves that the screening link designed by the present invention effectively removes non-robust feature points, thereby improving the robustness and stability of the algorithm.

[0108] Replica detection experiment:

[0109] During the copying process of the image sample, due to the conversion of the RGB image to the CMYK image and the influence of the printer accuracy, the image information will be lost to a certain extent. When the watermark strength is low, the copy loss will cause detection failure. Therefore, by using this feature, the present invention sets the intensity threshold gradient of different watermark blocks, so that different numbers of watermark blocks will fail to detect in the copying process of different times, so as to test the number of copies of the copy.

[0110] Analysis of resistance to filtering attacks:

[0111] In order to further illustrate the effectiveness of this algorithm against different types of attacks, the present invention uses filter attacks of different sizes to test the stability of the algorithm, such as Figures 2 to 4 As shown. The test sets are from COCO2014, ImageNet and Monet2photo. After different types of experiments, it can be found that as the filter scale increases, the loss of the original image increases, and the decoding accuracy decreases to a certain extent. In the case of median filtering, when the filter kernel size reaches 9, the decoding accuracy remains above 90%. For box filtering, when the filter kernel reaches 6, the accuracy remains above 90%. For bilateral filtering attacks, when the filter kernel reaches 6, the decoding accuracy remains above 90%.

[0112] Calculation of computing resource consumption:

[0113] In order to illustrate the difference between resource consumption, the present invention adopts the parameter of comparing GFLOPs, so as to reflect the resource consumption of different algorithms, and therefore reflect the calculation speed under the hardware conditions of the same computing power. It can be seen in this experiment that compared with the Stegastamp algorithm, the algorithm of the present invention has a calculation amount of 0.25 of the former, a parameter amount of 0.22, and a calculation speed of 0.46 in the test computer. The above experimental results prove that the algorithm of the present invention has a great improvement in calculation speed and resource consumption over some existing algorithms.

[0114] Table 3 Computational amount comparison experiment

[0115] GFLOPs Params It(ms) Stegastamp 20.7 1.8MB 102 The present invention 5.2 0.4MB 47

[0116] Invisibility experiment comparison:

[0117] In order to further illustrate the superiority of this algorithm in terms of invisibility, this algorithm compares the invisibility parameters of three algorithms under the same watermark strength coefficient. In the table, the PSNR value is selected as the measurement indicator, and 100 images in the MSCOCO dataset are randomly selected as test objects to calculate the parameters of the two images before and after the watermark is embedded. It can be seen that under the same intensity, since this method adopts the method of local area watermark embedding, the watermark is less likely to be observed from the overall situation. At the same time, the PSNR value is the highest under different watermark strengths, which proves that the present invention is more superior in invisibility and hiding ability.

[0118] Table 4 Invisibility comparison results

[0119] Intensity Factor HkDJ TwoStage CS The present invention 1 37.3 33.5 35.99 38.6 1.5 / 30 / 36.9 2 / 27.5 / 31.8

[0120] Watermark capacity detection:

[0121] Watermark capacity is an important reference indicator of invisible watermark algorithms. The more embedded bits there are, the more information the algorithm carries and the stronger its ability to transmit information. In the same image, the present invention uses different feature point generation methods for horizontal comparison, such as Figure 8 As shown, the test data set is still MSCOCO. It can be seen that the average number of feature points generated by the common feature point extraction method is 10, while the average number of feature points of the feature point generation method adopted by the present invention is close to 20. It can be seen that the watermark capacity is twice that of the previous method, which shows that the method adopted by the present invention has better information capacity, thereby embedding more information while controlling the disturbance to the image.

[0122] In summary, the present invention proposes an information hiding method based on deep learning, the main principles of which are: (1) extracting interest points of an image through Superpoint. (2) Two screening steps are designed to obtain sparse and robust interest points: the first step is to screen dense points with low probability values ​​to avoid their overlap, and the second step is to remove unstable points by calculating the displacement after being attacked, which ensures that the watermark can be correctly detected during the extraction process. (3) The present invention crops a small rectangular area around the robust interest point to embed the watermark. In the process of training the sub-block encoder, the present invention introduces simulated noise and different attacks as distortion layers in the adversarial network to further improve the robustness. In the watermark extraction process, the present invention uses ConvNext as a decoder, sets a threshold by statistical results, and obtains the decoding result. The decoding result is obtained by statistical results. This ensures the accuracy and consistency of the results.

[0123] Compared with the traditional technology, the advantages of the present invention are:

[0124] (1) This paper designs a watermark embedding and extraction framework for points of interest obtained based on a deep learning model, and achieves extensive robustness against different attacks through one-time training.

[0125] (2) By designing a two-step screening procedure and a method for identifying decoded values, the present invention avoids omission and interaction of watermark sub-blocks, which greatly reduces the error rate and improves the extraction efficiency.

[0126] (3) The present invention further ensures the robustness of the algorithm by selecting sub-blocks centered on the point of interest to embed the watermark. By adding simulated noise, adjusting the HSV color space and other attacks in the adversarial network, the quality of the encoded image and its robustness to large-scale cropping and other geometric deformations are improved.

[0127] (4) Compared with the existing deep learning information hiding algorithms, the model proposed in the present invention has the ability to handle multiple tasks. Specifically, the embedded information can be expanded to multiple different information, and the length of the information is also longer than that of the existing algorithms.

[0128] (5) By adjusting the threshold of the watermark intensity and the influence of information loss during the copying process on the discrimination of the sub-watermark block, it is possible to determine whether the sample is a copy, thereby achieving the detection effect of the copy.

[0129] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0130] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0131] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0132] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0133] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0134] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0135] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0136] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0137] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0138] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0139] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0140] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0141] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A deep learning-based information hiding method, characterized in that: The following steps are involved: S1. Extract interest points of the image using an interest point extraction network; S2. The extracted interest points are screened by an interest point screening network to obtain sparse and robust interest points, wherein the interest point screening network includes two steps of screening: the first step of screening is to calculate the Euclidean distance between the interest points and compare it with a preset threshold, and if the Euclidean distance is less than the preset threshold, one of the interest points is discarded; The second step is screening. For images processed by different attacks, the coordinate changes of the same interest point before and after the attack are compared to delete unstable interest points. S3. embedding watermark information in the area around the selected interest point through a watermark encoder; in the training process of the watermark encoder, the watermark encoder and a discriminator for attempting to identify the watermark form an adversarial training network, and introducing simulated noise and different attacks as a distortion layer in the adversarial training network; S4. The hidden watermark information is extracted from the encrypted image through the watermark decoder, and the potential sub-watermark blocks are further screened out through the sub-watermark block screening model.

2. The information hiding method based on deep learning as claimed in claim 1, characterized in that: In step S1, the interest point extraction network uses a Superpoint model, which includes a shared encoder for processing an input image and reducing its dimension; the shared encoder includes a convolutional layer, spatial downsampling and a nonlinear activation function.

3. The information hiding method based on deep learning as claimed in claim 2, characterized in that: In step S1, the interest point extraction network includes two decoders, which are respectively used to generate interest points based on probability maps and descriptors for characterizing the features of interest points. The pixel value of each point in the image represents the probability of the point becoming an interest point.

4. The deep learning-based information hiding method according to any one of claims 1 to 3, characterized in that: In step S3, the watermark encoder adopts a structure similar to U-net, and uses a ConvBNRelu layer including a convolution layer, a normalization layer and an activation layer to process the carrier image; wherein the preprocessed information, the upsampled carrier image and the original carrier image are stacked together in the channel dimension; the stacked image is sent to the ConvBNRelu layer for downsampling operation, the image size is reduced to the size of the original carrier image, and the number of channels is restored to 3 channels; the watermark information is embedded in the pixel image by the method of upsampling first and then downsampling.

5. The deep learning-based information hiding method according to any one of claims 1 to 4, characterized in that: In step S3, the discriminator includes three ConvBNRelu layers with a step size of 1, a global average pooling layer and a linear connection layer, and the ConvBNRelu layer includes a convolution layer, a normalization layer and an activation layer.

6. The deep learning-based information hiding method according to any one of claims 1 to 5, characterized in that: In step S3, during the training process of the watermark encoder, the image output by the encoder is compared with the original image, and different types of losses are calculated to perform gradient descent, thereby better guiding the generation of images with hidden information.

7. The information hiding method based on deep learning as claimed in claim 6, characterized in that: In step S4, the loss function is a weighted combination of L1 loss, mean square error (MSE) loss, adversarial network loss, and structural similarity loss, wherein the L1 loss is used to calculate the similarity between the encoded image and the original image, the mean square error (MSE) loss is used to measure the difference between the encoded information and the decoded information, the adversarial network loss is used to reflect whether the discriminator can distinguish the difference between the embedded images, and the structural similarity (SSIM) loss is used to maintain the structural similarity between the encoded image and the original image.

8. The deep learning-based information hiding method according to any one of claims 1 to 7, characterized in that: In step S3, a small rectangular area is cropped around the screened interest points to embed watermark information.

9. The deep learning-based information hiding method according to any one of claims 1 to 8, characterized in that: In step S4, the watermark decoder uses ConvNext as a decoder; Among them, a threshold is first set according to the statistical characteristics of the decoder output. During the actual watermark extraction, statistical data is calculated from the output tensor obtained by the decoder, and the statistical data is compared with the threshold to determine the watermarked and non-watermarked areas.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the deep learning-based information hiding method according to any one of claims 1 to 9 is implemented.

Images