An information hiding method based on deep learning

Abstract

This paper proposes a deep learning-based information hiding method that effectively improves the robustness and concealment of digital image watermarks. The method includes the following key steps: first, using a point-of-interest extraction network to extract points of interest from the image; second, using a two-step screening network to ensure the sparsity and robustness of the points of interest; then, using a watermark encoder to embed watermark information around the robust points of interest, and introducing an adversarial training network during the training process to improve the robustness of the watermark; finally, using a watermark decoder to extract the hidden watermark information, and using a sub-watermark block screening model to screen potential sub-watermark blocks. By embedding data at robust features of the image and employing an adversarial training network to enhance the robustness of the model, this method effectively resists various attacks and ensures the accuracy and security of the watermark information.

Description

Technical Field

[0001] The present invention relates to information hiding technology, and in particular to an information hiding method based on deep learning. Background Art

[0002] The widespread adoption and application of multimedia technology has brought significant benefits to society, but it also poses numerous risks to information protection. Digital image information is widely used for communication, but this information is susceptible to illegal tampering. Digital image watermarking algorithms are a powerful tool for copyright protection. Existing watermarking methods can be categorized as spatial domain, frequency domain, hybrid algorithms, and learning-based models. Spatial domain watermarking hides information within invisible regions of the cover image, but can lead to decoding errors when attacked by attacks such as chromatic aberration due to information loss.

[0003] Information hiding is a technique that embeds confidential data into a carrier, such as an image, audio, or string, to conceal secret data. Unlike traditional cryptography, which conceals the true meaning, information hiding aims to conceal the existence of information and recover the hidden information through algorithms, thereby achieving the purpose of secret transmission. Depending on the hidden object, information hiding techniques can be categorized as string information, image information, and audio information. The image used to conceal the secret data is called the carrier image, while the information or image recovered from it is called the secret information or image. Information hiding technology has been widely used in various fields, including copyright protection and information authentication.

[0004] Attacks such as color shifts and cropping can cause decoding errors. Frequency-domain watermarking algorithms embed information by applying the watermark signal to the frequency domain. Discrete cosine transform (DCT)-based methods distribute the watermark across the entire spatial image after performing an inverse transform on the encoded image. These methods achieve excellent concealment but are less robust against cropping and compression.

[0005] For a hybrid watermarking algorithm based on SIFT (Scale Invariant Feature Transform) and ORB (Oriented Fast and Rotated Brief), they used ORB, an adaptive region selection method based on texture information and feature points, to locate texture information and feature points to locate regions for embedding DCT watermarks. This algorithm offers improved concealment and invisibility, but the watermark capacity is limited, and feature points often shift under attacks. However, the watermark capacity is limited, and feature points often shift under attacks. A watermarking algorithm based on deep learning significantly improves robustness and invisibility under different attacks.

[0006] Watermarks can be divided into spatial domain and frequency domain watermarks according to the embedding method.

[0007] Spatial-domain digital watermarking embeds watermark information by directly modifying the pixel values ​​of an image. Least significant bits (LSB) modification algorithms are widely used due to their simplicity and ease of operation. However, these algorithms are less robust to noise attacks and can be easily removed through attacks such as filtering. Frequency-domain methods primarily transform the image in the frequency domain and then modify it at specific frequencies, making the watermark more imperceptible and robust. This results in higher algorithmic complexity and greater resistance to various attacks. Commonly used methods include spatial-domain watermarking algorithms based on singular value decomposition (SVD) and frequency-domain watermarking algorithms based on the discrete cosine transform (DCT), which preprocess the watermark information and embed it into the image's singular values. Frequency-domain algorithms generally offer good hiding power but suffer from lower information capacity. With the advent of deep learning, the learning capabilities of neural networks have enabled significant advancements in watermarking technology. Watermarking algorithms based on learning models have made significant progress in both information hiding capacity and robustness against various noises. These algorithms are broadly categorized as generative, adversarial, and staged. Generative algorithms directly utilize adversarial networks to generate images, while staged algorithms achieve better embedding performance and robustness through phased training of encoders and decoders.

[0008] An excellent information hiding algorithm must excel in three key areas: embedding capacity, robustness, and imperceptibility. Embedding capacity is the ratio of the length of the embedded secret information to the carrier image. A larger ratio indicates greater embedding capacity and a stronger information-carrying capability. Robustness refers to the ability to extract secret information. The ultimate goal of information hiding is to convey information secretly. If the extracted information contains significant errors, or even the intended message cannot be recovered, the algorithm's robustness is very poor. Imperceptibility refers to the difference between the image after the algorithm embeds information and the original image without the embedded information. The smaller the difference, the less impact the embedded information has on the original image, and the better the imperceptibility. The more difficult it is to distinguish between the carrier image and the secret image, the better the algorithm's performance.

[0009] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0010] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide an information hiding method based on deep learning to solve the problems of poor resistance to motion distortion, difficulty in targeted local embedding and poor resistance to complex noise in deep learning hidden networks.

[0011] To achieve the above object, the present invention adopts the following technical solutions:

[0012] An information hiding method based on deep learning, comprising the following steps:

[0013] S1. Extract interest points from the image using an interest point extraction network;

[0014] S2. Filter the extracted interest points using an interest point screening network to obtain sparse and robust interest points. The interest point screening network includes two steps: the first step is to calculate the Euclidean distance between interest points and compare it with a preset threshold. If the Euclidean distance is less than the preset threshold, one of the interest points is discarded. The second step is to remove unstable interest points by comparing the coordinate changes of the same interest point before and after the attack on the image after different attack processing.

[0015] S3. embedding watermark information in the area around the selected point of interest using a watermark encoder; during the training process of the watermark encoder, the watermark encoder and a discriminator for attempting to identify the watermark form an adversarial training network, and introducing simulated noise and different attacks as distortion layers in the adversarial training network;

[0016] S4. The hidden watermark information is extracted from the encrypted image through the watermark decoder, and the potential sub-watermark blocks are further screened out through the sub-watermark block screening model.

[0017] Furthermore, in step S1, the interest point extraction network uses a Superpoint model, which includes a shared encoder for processing the input image and reducing its dimension; the shared encoder includes a convolutional layer, spatial downsampling and a nonlinear activation function.

[0018] Furthermore, in step S1, the interest point extraction network includes two decoders, which are respectively used to generate interest points based on the probability map and descriptors for characterizing the features of the interest points. The pixel value of each point in the image represents the probability of the point becoming an interest point.

[0019] Furthermore, in step S3, the watermark encoder adopts a structure similar to U-net, and uses a ConvBNRelu layer including a convolution layer, a normalization layer and an activation layer to process the carrier image; wherein the preprocessed information, the upsampled carrier image and the original carrier image are stacked together in the channel dimension; the stacked image is sent to the ConvBNRelu layer for downsampling operation, the image size is reduced to the size of the original carrier image, and the number of channels is restored to 3 channels; the watermark information is embedded in the pixel image by the method of upsampling first and then downsampling.

[0020] Furthermore, in step S3, the discriminator includes three ConvBNRelu layers with a step size of 1, a global average pooling layer and a linear connection layer, and the ConvBNRelu layer includes a convolution layer, a normalization layer and an activation layer.

[0021] Furthermore, in step S3, during the training of the watermark encoder, the image output by the encoder is compared with the original image, and different types of losses are calculated to perform gradient descent, thereby better guiding the generation of images with hidden information.

[0022] Furthermore, in step S4, the loss function is a weighted combination of L1 loss, mean square error (MSE) loss, adversarial network loss, and structural similarity loss, wherein the L1 loss is used to calculate the similarity between the encoded image and the original image, the mean square error (MSE) loss is used to measure the difference between the encoded information and the decoded information, the adversarial network loss is used to reflect whether the discriminator can distinguish the difference between the embedded images, and the structural similarity (SSIM) loss is used to maintain the structural similarity between the encoded image and the original image.

[0023] Furthermore, in step S3, a small rectangular area is cropped around the screened interest points to embed watermark information.

[0024] Furthermore, in step S4, the watermark decoder uses ConvNext as a decoder;

[0025] Among them, a threshold is first set according to the statistical characteristics of the decoder output. During the actual watermark extraction, statistical data is calculated from the output tensor obtained by the decoder, and the statistical data is compared with the threshold to determine the watermarked and non-watermarked areas.

[0026] A computer program product includes a computer program, which implements the deep learning-based information hiding method when executed by a processor.

[0027] The present invention has the following beneficial effects:

[0028] The present invention proposes an information hiding method based on deep learning, which realizes the highly robust embedding and extraction of watermarks in images through innovative information hiding network design and training strategies. The method of the present invention extracts and screens robust points of interest, embeds watermarks at the robust features of the image, and ensures the stability and accuracy of the watermark in the face of geometric distortion and high-intensity attacks. By introducing simulated noise and multiple attacks through adversarial training networks, the concealment of the watermark and its resistance to different attacks are further improved. In addition, the watermark decoder is combined with a sub-watermark block screening model to effectively extract watermark information from the encrypted image, ensuring the accuracy and consistency of the extraction process. The present invention can embed more information while maintaining high robustness, and has the ability to handle multiple tasks and can embed multiple pieces of information of different lengths.

[0029] The advantages of the present invention compared with the prior art are:

[0030] 1. The large-capacity deep learning information hiding network proposed in this invention can embed more information while ensuring the robustness of the network, and achieves extensive robustness against different attacks through one-time training.

[0031] 2. By designing a two-step screening procedure and a method for distinguishing decoded values, the present invention avoids omission and interaction of watermark sub-blocks, which greatly reduces the error rate and improves the extraction efficiency.

[0032] 3. This paper further ensures the robustness of the algorithm by embedding the watermark in sub-blocks centered around the point of interest. By adding simulated noise to the adversarial network, adjusting the HSV color space, and other attacks, the quality of the encoded image and its robustness to large-scale cropping and other geometric deformations are improved.

[0033] 4. Compared with existing deep learning information hiding algorithms, the model proposed in this invention has the ability to handle multiple tasks. Specifically, the embedded information can be expanded to multiple different pieces of information, and the length of the information is also longer than that of existing algorithms.

[0034] 5. The present invention can determine whether a sample is a copy by setting a threshold of watermark intensity and judging the degree of information loss during the copying process.

[0035] In general, the present invention performs well in terms of invisibility, robustness and information capacity, and provides an effective solution for copyright protection and secure transmission of digital image information.

[0036] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an algorithm framework diagram of the information hiding method based on deep learning in an embodiment of the present invention.

[0038] Figure 2 The experimental results of median filtering with kernels of different sizes on three data sets are shown in the following figure.

[0039] Figure 3 The following are the experimental results of box filtering with kernels of different sizes on three data sets according to the embodiment of the present invention;

[0040] Figure 4 The following are the experimental results of bilateral filtering with kernels of different sizes on three data sets according to the embodiment of the present invention;

[0041] Figure 5 Comparison of the number of feature points of the three algorithms under different sub-watermark block sizes;

[0042] Figure 6 Comparison of the accuracy before and after feature point selection and PSNR values ​​under different attacks in the embodiment of the present invention;

[0043] Figure 7 These are the accuracy results of different types of noise attacks under three data sets in the embodiment of the present invention;

[0044] Figure 8 This is a comparison of images at different stages of the entire process of the embodiment of the present invention;

[0045] Figure 9 This is the point distribution generated by the feature point generation model of the embodiment of the present invention for ordinary image processing. DETAILED DESCRIPTION

[0046] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0047] This paper proposes a deep learning-based information hiding method and designs a deep learning-based information hiding network that features regional watermark adaptation, high robustness, and resistance to motion distortion. This paper implements a deep learning-based watermark hiding scheme that is regionally adaptive and resistant to motion geometric distortion. This paper provides an innovative watermarking framework that achieves accuracy during data embedding and extraction, good concealment, and resistance to various attacks by hiding data in robust image features and filtering them.

[0048] See Figure 1 , an embodiment of the present invention provides an information hiding method based on deep learning, comprising the following steps:

[0049] S1. Extract interest points from the image using an interest point extraction network;

[0050] S2. Filter the extracted interest points using an interest point screening network to obtain sparse and robust interest points. The interest point screening network includes two steps: the first step is to calculate the Euclidean distance between interest points and compare it with a preset threshold. If the Euclidean distance is less than the preset threshold, one of the interest points is discarded. The second step is to remove unstable interest points by comparing the coordinate changes of the same interest point before and after the attack on the image after different attack processing.

[0051] S3. embedding watermark information in the area surrounding the selected point of interest using a watermark encoder; during the training process of the watermark encoder, the watermark encoder and a discriminator for attempting to identify the watermark form an adversarial training network, and introducing simulated noise and different attacks as distortion layers in the adversarial training network to improve the robustness of the watermark;

[0052] S4. The hidden watermark information is extracted from the encrypted image through the watermark decoder, and the potential sub-watermark blocks are further screened out through the sub-watermark block screening model.

[0053] In a preferred embodiment, in step S1, the interest point extraction network uses a Superpoint model, which includes a shared encoder for processing the input image and reducing its dimension; the shared encoder includes a convolutional layer, spatial downsampling, and a nonlinear activation function. The interest point extraction network includes two decoders, respectively used to generate interest points based on a probability map and a descriptor for characterizing the features of the interest points. The pixel value of each point in the image represents the probability of the point being an interest point. The watermark encoder adopts a structure similar to U-net, using a ConvBNRelu layer including a convolutional layer, a normalization layer, and an activation layer to process the carrier image; wherein the preprocessed information, the upsampled carrier image, and the original carrier image are stacked together in the channel dimension; the stacked image is sent to the ConvBNRelu layer for downsampling, reducing the image size to the size of the original carrier image and restoring the number of channels to 3 channels; the watermark information is embedded in the pixel image by upsampling followed by downsampling to enhance the algorithm's hiding ability and the flexibility of selecting the embedding position.

[0054] In some embodiments, the ConvBNRelu layer includes a series of convolutional layers with a specific stride, a batch normalization (BN) layer, and an activation function. The stacking of convolutional layers controls upsampling and downsampling, thereby controlling the size and receptive field of the tensor. The BN layer utilizes normalization to stabilize gradient descent, thus ensuring a stable selection of hyperparameters for the neural network. The activation function enables the network to perform nonlinear fitting.

[0055] In a preferred embodiment, in step S3, a small rectangular area is cropped around the screened interest points to embed watermark information.

[0056] The discriminator of the adversarial training network includes three ConvBNRelu layers with a step size of 1, a global average pooling layer and a linear connection layer. The ConvBNRelu layer includes a convolution layer, a normalization layer and an activation layer.

[0057] In a preferred embodiment, in step S3, during the training of the watermark encoder, the image output by the encoder is compared with the original image, and different types of losses are calculated to perform gradient descent, thereby better guiding the generation of the image with hidden information. The loss function is a weighted combination of L1 loss, mean square error (MSE) loss, adversarial network loss, and structural similarity loss, wherein the L1 loss is used to calculate the similarity between the encoded image and the original image, the mean square error (MSE) loss is used to measure the difference between the encoded information and the decoded information, the adversarial network loss is used to reflect whether the discriminator can distinguish the difference between the embedded images, and the structural similarity (SSIM) loss is used to maintain the structural similarity between the encoded image and the original image.

[0058] In a preferred embodiment, the watermark decoder uses ConvNext as a decoder. In step S4, a threshold is first set according to the statistical characteristics of the decoder output. During the actual watermark extraction, statistical data is calculated from the output tensor obtained by the decoder and compared with the threshold to determine the watermarked and non-watermarked areas.

[0059] The information hiding network in the embodiments of the present invention achieves universal resistance to various attacks through a single training session, with excellent performance against geometric distortions and high-intensity attacks. First, the present invention uses Superpoint to extract points of interest from an image and designs two screening steps to obtain sparse and robust points of interest: the first step is to screen dense points with low probability values ​​to avoid overlap, and the second step is to remove unstable points by calculating their displacement after being attacked, thereby ensuring that the watermark can be correctly detected during the extraction process. Secondly, the present invention crops small rectangular areas around the robust points of interest to embed the watermark. During the training of the sub-block encoder, the present invention introduces simulated noise and different attacks as distortion layers in the adversarial network to further improve robustness. During the watermark extraction process, the present invention uses ConvNext as a decoder and sets thresholds based on statistical results to obtain decoding results, thereby ensuring the accuracy and consistency of the results. Furthermore, by setting different intensity thresholds for the information loss of the watermark during different copying processes, the printed image can be copy-detected, ensuring security during use.

[0060] The embodiment of the present invention provides a deep learning information hiding method with local feature point positioning, large capacity, and high robustness. Its innovations include: (1) an upsampling convolution design different from traditional deep learning is designed to expand the receptive field and reduce information loss to obtain a robust encoder, so that the sub-image after cropping can be embedded with information and accurately extracted. Fixed-bit encrypted information is embedded in a single channel during training. The upsampling convolution design is adopted to provide more space for information embedding. (2) A robust feature positioning network is designed and trained and a preliminary screening is performed to select areas with robust features in the carrier image, and a non-robust point elimination algorithm is performed by designing adjacent point screening and image attack to obtain the final robust feature points. The existing algorithm channel convolution is improved to a repeated operation of information scale, which not only reduces the model parameters but also expands the application scenarios of the algorithm. (3) The robust feature point area is cropped to a fixed size and encoded by a decoder to obtain the entire encrypted image, and a screening scheme is used to obtain a potential decoding module during the decryption process, and then a decoding operation is performed. By adding guiding edge information into the training process, the threshold of watermark intensity can be adjusted to detect the degree of information loss in different copy times.

[0061] In an embodiment of the present invention, a feature point generation model is used to predict the original image, extracting and screening regions with stable features for embedding watermark information, thereby enhancing the robustness and concealment capability of the watermark. The information embedding network structure is changed according to the input image size, and a convolution method of upsampling followed by downsampling is adopted. A specific size region around the feature point is used as input to achieve stable information embedding. The feature point extraction model and the targeted embedding extraction network can ensure that information is sparsely embedded in the stable features of the pattern, improving the pattern's ability to resist large-area cropping and composite noise attacks. At the same time, the design concept of the present invention also realizes a better local watermarking solution.

[0062] Specific embodiments of the present invention are further described below.

[0063] The present invention proposes a new watermarking framework that can obtain universal resistance to different attacks through a single training, and performs well in geometric distortion and high-intensity attacks. First, the present invention extracts the points of interest of the image through Superpoint, and designs two screening steps to obtain sparse and robust points of interest: the first step is to screen dense points with low probability values ​​to avoid their overlap, and the second step is to remove unstable points by calculating the displacement after being attacked, which ensures that the watermark can be correctly detected during the extraction process. Secondly, the present invention crops small rectangular areas around the robust points of interest to embed watermarks. In the process of training the sub-block encoder, the present invention introduces simulated noise and different attacks as distortion layers in the adversarial network to further improve robustness. In the watermark extraction process, the present invention uses ConvNext as a decoder, sets a threshold through statistical results, and obtains the decoding result. The decoding result is obtained by statistical results. This ensures the accuracy and consistency of the results.

[0064] The following describes in detail the large-capacity, highly robust deep learning hidden network proposed in this invention from five parts: interest point generation network, interest point screening network, adversarial training network, encoder and decoder.

[0065] Point of interest generation part

[0066] The goal of this part is to reduce the loss of watermark information under attack and to correctly extract the watermark without knowing the watermark coordinates. Therefore, the present invention reduces information loss by embedding the watermark into the robust feature area of ​​the image and locating the robust feature area by generating interest points, which makes it possible to pass even without the original image. Therefore, the present invention uses the Superpoint model to extract interest points, which has been proven to be robust to noise, motion blur and brightness changes. The model has a shared encoder to process and reduce the dimensionality of the input image. The purpose of the encoder is to reduce the dimensionality of the input image.

[0067] The dimension of the input image. The input image passes through the convolution layer, spatial downsampling and nonlinear activation function. The input image is decoded by two decoders to generate interest points and descriptors. The probability of each pixel value in the output map corresponding to an interest point is generated, and the probability map is as follows: Figure 9 shown.

[0068] Points of interest filter section

[0069] The initial points of interest extracted by the interest point extraction module are dispersed, robust, and numerous. However, some feature points may become unstable during attacks. Therefore, this invention designs a screening network to eliminate unstable feature points to ensure decoding accuracy. This method designs a two-step screening method to further improve the robustness of interest points and avoid point overlap, as some points may become locally dense, causing embedded watermark blocks to overlap, and a small number of points may disappear or move after attacks of varying intensities.

[0070] In the first step of screening, some robust feature points are close to each other, which may cause the watermark blocks to overlap and thus result in decoding errors. In order to reduce the density of points and thus maintain the sparsity of the watermark, the present invention calculates the Euclidean distance D between two feature points and compares it with the threshold δ m Compare, if D is less than δ m If the point is not found, the point is discarded, thus retaining the only robust feature point in a certain area.

[0071] In the second step of screening, a small number of points of interest disappear or shift after being attacked, leading to missed detection and decoding errors. To ensure the stability of the points of interest, this paper uses different types of attacks to process the original image before and after watermarking, and extracts robust points of interest respectively. By measuring the coordinate changes of the same feature point after different attacks, it is possible to screen out qualified feature regions.

[0072] Information embedding part

[0073] The encoder and decoder used in this method are trained separately. The encoder with excellent encryption performance is obtained by training with image blocks of the same size as the sub-blocks randomly cropped from a public dataset.

[0074] This method adopts a structure similar to U-net. Due to the different sizes of the input, the encoder consists of a convolutional layer, a normalization layer, and an activation layer. The present invention uses ConvBNRelu to process the carrier image. The preprocessed information, the upsampled carrier image, and the original carrier image are stacked together in the channel dimension and sent to the ConvBNRelu layer for downsampling. This downsampling operation is similar to the upsampling operation of the carrier. The size of the image is downsampled to the size of the original carrier image, and the number of channels is restored to 3 channels. In this way, the sub-image block with the final embedded information is obtained. After processing, the information is secretly hidden in the cover image and is difficult to extract. Since this experiment adopts the method of upsampling first and then downsampling, more information can be embedded in the pixel map, so the algorithm has a stronger hiding ability and has the advantage of being able to freely choose the embedding location.

[0075] Discriminant network part

[0076] The discriminator network is primarily used to guide the embedding of watermarked pixels during training, preventing the discriminator from correctly identifying hidden information in the image. The discriminator consists primarily of three ConvBNReLU layers with a stride of 1, a global average pooling layer, and a linear connection layer. By comparing the output image with the original image, different types of loss can be obtained, enabling gradient descent to better guide the generation of images with hidden information.

[0077] Information extraction part

[0078] After obtaining the encrypted image, information is extracted through the decoder. During the decoding process, the same screening model is used to screen out potential sub-watermark blocks, and the values ​​obtained by the decoder are used for screening. The present invention uses a ConvNext network to recover the hidden information of the encoded image. A 1×m tensor (m=2 in the experiment) is obtained from the input sub-image block, and the decoding result can be judged by comparing the value of the tensor. The decoding result is judged by comparing the value of the tensor with the calculated threshold. By randomly cropping 1000 data sets of size 64×64, the present invention calculates the mean and variance. Based on the decoded values ​​of the decoder network, according to the 3σ principle, the present invention sets two thresholds of γ1=0.01 and γ2=0.996. The decoded values ​​in (0-γ1) are determined to be 0, and when the value is higher than γ2, it is 1. When their decoded values ​​∈(γ1, γ2), these blocks are determined to be not embedded and are discarded.

[0079] Loss function part

[0080] The present invention designs different loss functions in order to improve the invisibility and robustness of the image watermark, and adopts L1 loss. The loss of this method is to improve the accuracy of decoding and image quality. L1 calculates the similarity between the encoded image and the original image. It shows the average error range of the predicted value and the average error range of the predicted value, which can ensure the visual similarity between the encoded image and the original image at the pixel level. At the same time, the present invention adds the mean square error loss to measure the difference between the encoded information and the decoded information. Since MSE is the average of the sum of the squares of the encoded information and the decoded information, the accuracy of decoding can be guaranteed during the training process. At the same time, the present invention adds an adversarial network loss, which also reflects whether the discriminator can distinguish the differences between the embedded images. The structural similarity (SSIM) loss is described as L SSIM , represents the similarity of image structure, and the final loss function is as follows: (λ1,λ A λ M ,λ s are the weights of their respective losses).

[0081] L=λ1L1+λ M L M +λ A L A +λ S L SSIM

[0082] Existing research has established that image regions rich in texture information are more concealable than flat regions, as the human eye is less able to detect changes in image information. A common problem with deep learning is the lack of interpretability in the learning process. Under the control of the loss function, deep learning models may learn in unexplainable directions. In contrast, traditional information hiding algorithms have established many models with strong interpretability, which, through clever design, can achieve good imperceptibility and robustness. Furthermore, recent research results have shown that deep learning algorithms can achieve better results when combined with traditional features. Therefore, this innovation chooses to build on existing deep learning models by concentrating the embedded information on stable features, thereby achieving a good information embedding process through a two-stage steganography scheme.

[0083] Experimental part

[0084] This section first introduces image quality evaluation systems, hardware facilities, and datasets across different dimensions. It then demonstrates the validity of this invention by testing watermarked images using various hybrid attacks and analyzing decoding accuracy. This experiment compares the watermark capacity with similar algorithms and demonstrates the advantages of this algorithm by measuring image imperceptibility using relevant metrics.

[0085] Evaluation indicators

[0086] The Peak Signal-to-Noise Ratio (PSNR) is a common method for measuring covert vectors. This measure compares the difference between the watermarked image and the original image. The PSNR is often used to assess the imperceptibility of a watermarked image. A higher PSNR value indicates less image distortion and better imperceptibility.

[0087]

[0088] Structural similarity (SSIM) evaluates the accuracy of extracted information. This method uses the bit error rate (BER), which measures the number of erroneous bits in the extracted information compared to the original. A lower BER indicates more accurate extracted information and greater algorithm robustness. l(x, y), c(x, y), and s(x, y) represent brightness similarity, contrast similarity, and structural similarity, respectively. α, β, and γ are weighting coefficients, generally set to 1.

[0089] SSIM(x,y)=[l(x,y)] α [c(x,y)] β [s(x,y)] γ

[0090] The bit error rate (BER) is used to evaluate the difference between the extracted information and the original embedded information. The larger the bit error rate, the worse the algorithm effect.

[0091] Experimental parameters

[0092] The experimental platform of the present invention is implemented based on PyTorch and accelerated by GeForce RTX 3090 graphics card. The present invention uses COCO val2014 dataset and adopts 30,000 datasets for training, of which 85% are randomly selected for training and 15% for testing. The test set is performed by randomly cropping 64×64 sub-image blocks from the image, and the total number of iterations is 300 times. The mini-batch size is set to 16. The Adam optimizer is used in all experiments of the present invention, and the initial learning rate is 0.001. In the experiment, the present invention uses ImageNet and Monet2phone datasets as comparison to prove that the algorithm has good adaptability to different types of images. The algorithm has good adaptability to different types of images.

[0093] Experimental results

[0094] Hybrid attack experiment:

[0095] In order to illustrate the robustness of this method under mixed attacks, this method uses different types of attacks to test the robustness of images. The attack types are shown in the following table, which are divided into geometric attacks, noise attacks, filtering attacks and mixed attacks.

[0096] The mean filter is a common filter in image processing, primarily used to smooth noise. It uses the average value of surrounding pixels to achieve this effect. Since the Monet2photo dataset consists mostly of complex and colorful landscape images, a larger kernel size results in a loss of more information. Its accuracy is 90.01%, the lowest of the three datasets, but still above 90%. In the box filter, the accuracy across the three datasets shows good consistency, exceeding 90% even when the noise intensity reaches 0.05. Because bilateral filtering also produces more severe losses for complex images, the accuracy of the Monet2photo dataset drops to 91.33% as the filtering effect gradually increases.

[0097] In all attacks, the accuracy of different data sets decreases with the increase of filtering strength, but overall, they all maintain good consistency. In addition, when the strength is 0.05, most of them can maintain an accuracy of more than 90%, showing the good robustness of the algorithm. Noise attack: In this experiment, the present invention uses three different noises to attack the data set to verify its robustness. The noise includes speckle noise, salt and pepper noise, and Gaussian noise, and each noise is tested with different intensities. After the noise attack, the above-mentioned positioning network and decoding rules are used to extract the target information, and the accuracy of watermark extraction is the measurement criterion of this experiment.

[0098] Table 1 shows the different accuracy levels under different conditions. The experimental results show that this method is highly resistant to all three types of noise tested, maintaining an accuracy rate of over 89.6% even at an intensity of 0.05. Because Gaussian noise in images causes interference between bright and dark points, significantly degrading image quality and affecting feature extraction, the accuracy of point-of-interest identification and extraction is most severely affected. Accuracy is most severely impacted when the noise intensity is 0.01 and 0.02.

[0099] Table 1 Complex attack experiments

[0100] Attack Type HiDDeN Two-stage GM DADW The present invention No attack 100.0 100.0 99.0 100.0 99.5 JPEG (Q=50) 63.0 76.2 75.9 81.7 85.2 Clipping (0.09) - 91.9 - 93.5 96.7 Clipping (0.35) 88 - - - 94.3 Gaussian blur (sigma = 2) 96.1 98.6 - 60.0 96.6 Salt and pepper noise (0.05) 61.2 - - - 92.9 Salt and pepper noise (0.05) 88.9 92.8 87 95.7 89.6 Gaussian noise (0.05) 61.6 91.6 86.1 - 91.9 Mean filter (d = 5) 82.1 91.1 79.2 - 94.3 Bilateral filtering (d=6) 81.3 90.6 - - 93.9 Box filter (d=6) 82.3 90.1 - - 91.2 Size adjustment (p=0.5) - 92.8 - 67.1 93.2 Resizing + Salt and Pepper Noise 52.9 89.7 - - 90.5 Gaussian blur + bilateral filtering 44.8 88.3 - - 90.1 Cropping + box filtering 50.6 84.9 - - 89.6

[0101] Geometric attack experiment:

[0102] Because this method uses localized watermark embedding and geometrically corrects the image using feature points during decoding, it can resist attacks from complex geometric deformations. In the following experiments, as shown in Table 2, the present invention uses complex geometric attacks composed of rotations of different angles and scalings of different scales, and tests the decoding accuracy. It can be seen that under certain degrees of rotation and scaling, the decoding accuracy remains above 90%, demonstrating that even under complex attacks, this method still maintains a certain degree of decoding stability and good resistance to geometric attacks.

[0103] Table 2 Composite geometric attack experiments

[0104] Rotation 5° 10° 15° 20° 1 96.5 95.4 94.2 92.01 0.9 94.3 93.9 93.6 91.6 0.8 93.3 92.8 91.7 90.3 0.7 88.1 87.9 87.1 86.6

[0105] Noise attack experiment:

[0106] In order to illustrate the superiority of the algorithm's stable feature point embedding watermark method and its effectiveness in the screening step, the present invention uses noise of different types and intensities to test the accuracy of watermark decoding, such as Figure 7As shown in the figure, the accuracy rate (Acc) gradually decreases as the noise intensity increases, but generally remains above 90% at an intensity of 0.05. Furthermore, the algorithm maintains good resistance to all three types of noise. This experiment also used three different datasets: COCO2014, ImageNet, and Monet2photo. Testing was performed on images from various scenes, and no significant differences were observed between randomly tested images, demonstrating the algorithm's adaptability to diverse scenes and image types.

[0107] Similarly, in order to prove the effectiveness of this scheme in the feature screening process, Figure 6 As shown, this experiment compared the accuracy before and after feature point screening. It can be seen that in the unscreened group (without feature screening), the average accuracy rate is around 80%, while the group after feature screening generally maintains an accuracy rate of over 90% in various attacks. Therefore, this invention proves that the screening process designed by the present invention effectively removes non-robust feature points, thereby improving the robustness and stability of the algorithm.

[0108] Replica detection experiment:

[0109] During the copying process, image information is lost to a certain extent due to the RGB image conversion to CMYK and the influence of printer accuracy. When the watermark intensity is low, this loss can cause detection failure. Therefore, the present invention takes advantage of this feature by setting intensity threshold gradients for different watermark blocks. This ensures that different numbers of watermark blocks will fail detection during different copying processes, thereby measuring the number of copies a copy has undergone.

[0110] Analysis of resistance to filtering attacks:

[0111] In order to further illustrate the effectiveness of this algorithm against different types of attacks, the present invention uses filter attacks of different sizes to test the stability of the algorithm, such as Figures 2 to 4 As shown in the figure, the test sets are from COCO2014, ImageNet, and Monet2photo. A variety of experiments have shown that as the filter scale increases, the original image is lost more and the decoding accuracy decreases slightly. For median filtering, the decoding accuracy remains above 90% when the filter kernel size reaches 9. For box filtering, the accuracy remains above 90% when the filter kernel size reaches 6. For bilateral filtering attacks, the decoding accuracy remains above 90% when the filter kernel size reaches 6.

[0112] Calculation of computing resource consumption:

[0113] To illustrate the differences in resource consumption, this paper compares the GFLOPs parameter, which reflects the resource consumption of different algorithms and, therefore, the computational speed under hardware conditions with the same computing power. In this experiment, we can see that compared to the Stegastamp algorithm, the computational effort of our algorithm is 0.25 of the former, the number of parameters is 0.22, and the computing speed on the test computer is 0.46. These experimental results demonstrate that our algorithm significantly improves on existing algorithms in terms of computational speed and resource consumption.

[0114] Table 3 Computational amount comparison experiment

[0115] GFLOPs Params It(ms) Stegastamp 20.7 1.8MB 102 The present invention 5.2 0.4MB 47

[0116] Invisibility experiment comparison:

[0117] To further demonstrate the superiority of this algorithm in terms of invisibility, this algorithm compares the invisibility parameters of three algorithms at the same watermark strength coefficient. In the table, the PSNR value is selected as the measurement metric, and 100 images from the MSCOCO dataset are randomly selected as test objects to calculate the parameters of the two images before and after watermarking. It can be seen that at the same strength, due to the localized watermark embedding method adopted by this method, the watermark is less visible from an overall perspective. At the same time, the PSNR value is the highest at different watermark strengths, which proves that this method is superior in invisibility and hiding ability.

[0118] Table 4 Invisibility comparison results

[0119] Intensity Factor HiDDeN TwoStage CS The present invention 1 37.3 33.5 35.99 38.6 1.5 / 30 / 36.9 2 / 27.5 / 31.8

[0120] Watermark capacity detection:

[0121] Watermark capacity is an important reference indicator of invisible watermark algorithms. The more embedded bits, the more information the algorithm carries and the stronger its ability to transmit information. In the same image, the present invention uses different feature point generation methods for horizontal comparison, such as Figure 8 As shown, the test data set is still MSCOCO. It can be seen that the average number of feature points generated by the ordinary feature point extraction method is 10, while the average number of feature points of the feature point generation method adopted by the present invention is close to 20. It can be seen that the watermark capacity is twice that of the previous method, which shows that the method adopted by the present invention has better information capacity, thereby embedding more information while controlling the disturbance to the image.

[0122] In summary, the present invention proposes an information hiding method based on deep learning, the main principles of which are: (1) extracting points of interest from an image through Superpoint. (2) designing two screening steps to obtain sparse and robust points of interest: the first step is to screen dense points with low probability values ​​to avoid their overlap, and the second step is to remove unstable points by calculating the displacement after being attacked, which ensures that the watermark can be correctly detected during the extraction process. (3) The present invention crops a small rectangular area around the robust points of interest to embed the watermark. In the process of training the sub-block encoder, the present invention introduces simulated noise and different attacks as distortion layers in the adversarial network to further improve the robustness. In the watermark extraction process, the present invention uses ConvNext as a decoder, sets a threshold by statistical results, and obtains the decoding result. The decoding result is obtained by statistical results. This ensures the accuracy and consistency of the results.

[0123] Compared with the traditional technology, the advantages of the present invention are:

[0124] (1) This paper designs a watermark embedding and extraction framework for points of interest obtained based on a deep learning model, and achieves extensive robustness against different attacks through one-time training.

[0125] (2) By designing a two-step screening procedure and a method for distinguishing decoded values, the present invention avoids omission and interaction of watermark sub-blocks, which greatly reduces the error rate and improves the extraction efficiency.

[0126] (3) The present invention further ensures the robustness of the algorithm by selecting sub-blocks centered on the point of interest to embed the watermark. By adding simulated noise, adjusting the HSV color space, and other attacks to the adversarial network, the quality of the encoded image and its robustness to large-scale cropping and other geometric deformations are improved.

[0127] (4) Compared with the existing deep learning information hiding algorithms, the model proposed in this invention has the ability to handle multiple tasks. Specifically, the embedded information can be expanded to multiple different pieces of information, and the length of the information is also longer than that of the existing algorithms.

[0128] (5) By adjusting the threshold of the watermark intensity and the influence of information loss on the discrimination of the sub-watermark block during the copying process, it is possible to determine whether the sample is a copy, thereby achieving the detection effect of the copy.

[0129] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0130] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0131] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.

[0132] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a read-only optical disc (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0133] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0134] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0136] Those skilled in the art will understand that all or part of the steps of the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. Various media that can store program codes.

[0137] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0138] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0139] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0140] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0141] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.

Claims

1. A deep learning-based information hiding method, characterized in that: The following steps are involved: S1. Extracting interest points from an image using an interest point extraction network; S2. Filter the extracted POIs using an POI screening network to obtain sparse and robust POIs. The POI screening network includes two steps: First, the Euclidean distance between POIs is calculated and compared with a preset threshold. If the Euclidean distance is less than the preset threshold, one of the POIs is discarded. The second step is screening. For images processed with different attacks, the coordinate changes of the same interest point before and after the attack are compared to remove unstable interest points. S3. Embed watermark information in the area surrounding the selected POI using a watermark encoder. During training of the watermark encoder, the watermark encoder and a discriminator for attempting to identify the watermark form an adversarial training network. Simulated noise and different attacks are introduced as distortion layers in the adversarial training network. S4. The hidden watermark information is extracted from the encrypted image through the watermark decoder, and the potential sub-watermark blocks are further screened out through the sub-watermark block screening model.

2. The information hiding method based on deep learning according to claim 1, characterized in that In step S1, the interest point extraction network uses a Superpoint model, which includes a shared encoder for processing the input image and reducing its dimension; the shared encoder includes a convolutional layer, spatial downsampling and a nonlinear activation function.

3. The information hiding method based on deep learning according to claim 2, characterized in that: In step S1, the interest point extraction network includes two decoders, which are used to generate interest points based on the probability map and descriptors for characterizing the features of interest points. The pixel value of each point in the image represents the probability of the point becoming an interest point.

4. The deep learning-based information hiding method according to any one of claims 1 to 3, wherein: In step S3, the watermark encoder adopts a structure similar to U-net and uses a ConvBNRelu layer including a convolution layer, a normalization layer and an activation layer to process the carrier image; wherein the preprocessed information, the upsampled carrier image and the original carrier image are stacked together in the channel dimension; the stacked image is sent to the ConvBNRelu layer for downsampling operation, the image size is reduced to the size of the original carrier image, and the number of channels is restored to 3 channels; the watermark information is embedded in the pixel image by upsampling first and then downsampling.

5. The deep learning-based information hiding method according to any one of claims 1 to 3, wherein: In step S3, the discriminator includes three ConvBNRelu layers with a step size of 1, a global average pooling layer and a linear connection layer, and the ConvBNRelu layer includes a convolution layer, a normalization layer and an activation layer.

6. The deep learning-based information hiding method according to any one of claims 1 to 3, wherein: In step S3, during the training of the watermark encoder, the image output by the encoder is compared with the original image, and different types of losses are calculated to perform gradient descent, thereby better guiding the generation of images with hidden information.

7. The information hiding method based on deep learning according to claim 6, characterized in that: In step S3, the loss function is a weighted combination of L1 loss, mean square error (MSE) loss, adversarial network loss, and structural similarity loss, where L1 loss is used to calculate the similarity between the encoded image and the original image, the mean square error (MSE) loss is used to measure the difference between the encoded information and the decoded information, the adversarial network loss is used to reflect whether the discriminator can distinguish the difference between the embedded images, and the structural similarity (SSIM) loss is used to maintain the structural similarity between the encoded image and the original image.

8. The deep learning-based information hiding method according to any one of claims 1 to 3, wherein: In step S3, a small rectangular area is cropped around the screened interest points to embed watermark information.

9. The deep learning-based information hiding method according to any one of claims 1 to 3, wherein: In step S4, the watermark decoder uses ConvNext as a decoder; Among them, a threshold is first set according to the statistical characteristics of the decoder output. During the actual watermark extraction, statistical data is calculated from the output tensor obtained by the decoder, and the statistical data is compared with the threshold to determine the watermarked and non-watermarked areas.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the deep learning-based information hiding method according to any one of claims 1 to 9 is implemented.

Images