Digital image mixed watermarking method based on time domain and frequency domain synchronization

Abstract

The invention relates to the technical field of digital image processing, in particular to a digital image mixed watermarking method based on time domain and frequency domain synchronization. According to the method, a time domain and frequency domain synchronous watermark embedding algorithm and a multi-stage watermark extraction algorithm are innovatively combined, so that the concealment and robustness of the watermark are remarkably improved, and the problems of digital image copyright protection in the face of printing and scanning attacks and network attacks are effectively solved; the method ensures the security and reliability of digital image copyright protection, provides a more advanced and effective technical means for digital image copyright protection, has the advantages that a time domain and frequency domain synchronous watermark embedding algorithm enables watermarks to be fused into image details and is high in concealment, the watermarks are effectively protected in the face of various attacks, and the security and reliability of digital image copyright protection are improved. The multi-stage watermark extraction algorithm ensures accurate extraction when the image quality is reduced, and the watermark performance is integrally improved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and in particular to a digital image hybrid watermarking method based on time domain and frequency domain synchronization. Background Art

[0002] In order to meet the urgent need for digital image copyright protection, digital watermark technology has been widely studied and applied as a potential solution. The basic principle of digital watermark technology is to embed specific information (watermark) into digital images in an invisible or imperceptible way, thereby achieving the identification and protection of image copyright without affecting the normal use of the image.

[0003] Existing digital watermarking technologies are divided into two categories: spatial domain and transform domain. Spatial domain watermarking technology directly operates pixel values ​​to embed watermarks. Although it has low computational complexity, simple implementation and fast embedding speed, the watermark is easily damaged by slight changes in the image and has poor robustness. Transform domain watermarking technology first transforms the image to a specific domain, then embeds the watermark and then inversely transforms it back to the spatial domain. It can use the characteristics of the transform domain to make the watermark resistant to common operations and can be embedded in the low-frequency coefficients of the discrete cosine transform domain. However, the calculation is complex, and the watermark validity is difficult to maintain in the face of printing, scanning and network attacks, and it cannot reliably protect copyright.

[0004] The existing technology has some limitations and shortcomings in digital image hybrid watermarking:

[0005] Secondly, the adaptability of watermark is limited: the watermark embedding method is simple and lacks consideration of the image content characteristics. The watermark embedding strategy cannot be flexibly adjusted according to the importance and characteristics of different areas of the image, resulting in unreasonable distribution of watermarks in the image and easy to be destroyed;

[0006] Furthermore, the watermark protection effect is not good: when faced with complex attack methods (such as printing and scanning attacks that introduce multiple noises, change resolution, geometric distortion, noise addition, excessive compression, cropping and splicing in network attacks, etc.), the distribution pattern of watermark information is easily destroyed, resulting in the watermark extraction algorithm being difficult to accurately identify and restore the watermark, and unable to provide reliable copyright protection for digital images. Summary of the invention

[0007] The purpose of the present invention is to provide a digital image hybrid watermarking method based on time domain and frequency domain synchronization to solve the problem that the prior art mentioned in the above background technology has some limitations and shortcomings in digital image hybrid watermarking.

[0008] To achieve the above object, the present invention provides a digital image hybrid watermarking method based on time domain and frequency domain synchronization, comprising the following steps:

[0009] S1, read the digital image that needs to be watermarked and pre-process the image;

[0010] S2. Determine the watermark information content according to the copyright protection requirements and convert the information into a format suitable for embedding into the image;

[0011] S3, using the time domain and frequency domain synchronous watermark embedding algorithm, monitor the changes in the time domain of the image, and select the appropriate frequency in the frequency domain to embed the watermark;

[0012] S4, saving the processed image with embedded watermark, and the saving format can be selected according to actual needs;

[0013] S5. Use a deep learning-based extraction framework to identify watermark features in damaged images by training the model.

[0014] As a further improvement of the technical solution, the step S1 of reading the digital image to be watermarked and preprocessing the image specifically includes the following steps:

[0015] S11, read the image and obtain basic information;

[0016] S12, format conversion;

[0017] S13, size adjustment;

[0018] S14: Normalization processing.

[0019] As a further improvement of the technical solution, in step S2, the watermark information content is determined according to the copyright protection requirements, and the conversion of the watermark information into a format suitable for embedding into an image specifically includes the following steps:

[0020] S21, determining the watermark information content;

[0021] S22, information coding conversion;

[0022] S23. Coding review and testing.

[0023] As a further improvement of the technical solution, in step S3, the changes in the time domain of the image are monitored by using a time domain and frequency domain synchronous watermark embedding algorithm, and a suitable frequency is selected in the frequency domain to embed the watermark, which specifically includes the following steps:

[0024] S31, establish an image change monitoring module to track the changes in the pixel values ​​of the image in real time;

[0025] S32, using fast Fourier transform to convert the image into frequency domain space, analyzing the frequency spectrum characteristics of the image, and selecting appropriate frequency components as watermark embedding positions according to the frequency characteristics of the watermark information and the distribution of the image spectrum;

[0026] S33, using a specific encoding method to embed the watermark information into the selected frequency domain coefficients;

[0027] S34. During the embedding process, a correlation model between the time domain and the frequency domain is established to achieve synchronous embedding of the watermark in the time domain and the frequency domain, thereby enhancing the stability and recoverability of the watermark when it is attacked.

[0028] As a further improvement of the present technical solution, step S32 specifically includes the following steps:

[0029] S321, converting the image from the time domain space to the frequency domain space by using fast Fourier transform;

[0030] S322, providing a basis for selecting a suitable watermark embedding position by calculating the distribution of image energy in the frequency domain;

[0031] S323. Select appropriate frequency components as watermark embedding positions according to the frequency characteristics of the watermark information and the distribution of the image spectrum.

[0032] As a further improvement of the present technical solution, step S33 specifically includes the following steps:

[0033] S331, before embedding the watermark, pre-analyze the image, and adaptively adjust the watermark embedding strength according to the local complexity and visual importance area of ​​the image;

[0034] S332, a phase modulation-based coding method utilizes the sensitivity of the phase of the frequency domain coefficient to embed watermark information;

[0035] S333, amplitude modulation coding adjusts the amplitude of the frequency domain coefficient according to the value of the watermark information;

[0036] S334. After embedding the watermark, post-process the image to make the watermark-embedded image as visually similar as possible to the original image.

[0037] As a further improvement of the present technical solution, in step S4, the processed image with embedded watermark is saved, and the saving format can be selected according to actual needs. The specific operation steps are: if the image needs to be transmitted quickly on the network and has high requirements for storage space, select the JPEG format; if the image quality requirements are high and support for transparent background is required, select the PNG format.

[0038] As a further improvement of the technical solution, the extraction framework based on deep learning is adopted in step S5, and the watermark features in the damaged image are identified by training the model, which specifically includes the following steps:

[0039] S51, the anti-noise and anti-compression feature extraction layer suppresses the noise in the image and compensates for the information loss caused by compression through the pre-learned noise and compression patterns;

[0040] S52, inputting the image processed by anti-noise and anti-compression features into a pre-trained convolutional neural network, and the convolutional neural network automatically extracts potential watermark features in the image by using its convolutional layer and pooling layer and other structures;

[0041] S53, inputting the features extracted by the convolutional neural network into a multilayer perceptron, and the multilayer perceptron further processes and analyzes the features through its multiple hidden layers;

[0042] S54, decoding and identifying the extracted watermark features, restoring them to the original watermark information, and comparing them with the watermark information registered in the database to determine the copyright of the image.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. In the present invention, a powerful watermark defense mechanism is constructed by using a time domain and frequency domain synchronous watermark embedding algorithm and utilizing the complementary characteristics between the time domain and the frequency domain. When facing various common image processing attacks (such as cropping, compression, noise interference, printing and scanning, etc.), whether it is image changes in the time domain or coefficient disturbances in the frequency domain, the synergy of the time domain and the frequency domain can effectively protect the watermark information and ensure that the watermark can maintain a high integrity after being attacked.

[0045] 2. In the present invention, the anti-noise and anti-compression feature extraction layer in the multi-level watermark extraction algorithm provides a solid guarantee for watermark extraction, so that it can still accurately extract the watermark even when the image quality is severely degraded, which greatly improves the robustness of the watermark in complex environments and provides reliable technical support for the protection of digital image copyright. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a schematic diagram of the overall steps of the digital image hybrid watermarking method based on time domain and frequency domain synchronization of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In a specific embodiment, Figure 1 As shown, the present invention provides a digital image hybrid watermarking method based on time domain and frequency domain synchronization, comprising the following steps:

[0049] 1. Time domain and frequency domain synchronous watermark embedding (TFSW): Establish a time domain image change monitoring mechanism, convert the image to the frequency domain through fast Fourier transform (FFT) and select the appropriate frequency components, and use the complementary relationship between time domain and frequency domain to embed the watermark to achieve synchronous embedding;

[0050] 2. Multi-level watermark extraction: Get the image to be detected, first use the convolutional neural network (CNN) to extract the potential watermark features, and then use the multi-layer perceptron (MLP) combined with the anti-noise and anti-compression feature extraction layer to accurately extract the watermark and determine the copyright ownership.

[0051] The detailed algorithm steps are as follows:

[0052] Step 1: Read the digital image to be watermarked and preprocess the image.

[0053] Step 1.1: Read the image and get basic information.

[0054] 1) Read the image: Assume that the reading function using the image reading library is , where path is the storage path of the image or the network data source address, and I is the read image object (data structure).

[0055] 2) Get image format: define the function to get image format , the value range of F is {JPEG,PNG,BMP,...}.

[0056] 3) Get the original size: Let the function for getting the width and height of the image be W, , where W represents width and H represents height, in pixels.

[0057] 4) Get color mode: define the function to get color mode , the value range of C is {GRAYSCALE,RGB,CMYK,...}.

[0058] Step 1.2: Format conversion.

[0059] 1) Assume that the default universal format is target_format (RGB). If F≠target_format, perform format conversion.

[0060] 2) For grayscale images converted to RGB format (assuming the grayscale value is G): the converted RGB pixel values ​​R=G, G=G, B=G (that is, the value of each channel is equal to the grayscale value);

[0061] 3) For CMYK mode images converted to RGB format (assuming that C, M, Y, and K are the values ​​of the CMYK channels respectively): according to the color space conversion formula R=255×(1 and C)×(1 and K), G=255×(1 and M)×(1 and K), B=255×(1 and Y)×(1 and K).

[0062] Step 1.3: Sizing.

[0063] 1) Determine the standard size: Assume that the preset standard size is std_width and std_height (std_width=256, std_height=256).

[0064] 2) Calculate the scaling ratio: Width scaling ratio , height scaling .

[0065] 3) Image scaling (taking bilinear interpolation scaling as an example): The bilinear interpolation scaling function is ,in is the scaled image.

[0066] Step 1.4: Normalization.

[0067] 1) Calculate the conversion ratio (taking mapping to the [0, 1] interval as an example): Assume that the maximum value that the image pixel value data type can represent is (For 8-bit unsigned integers, ), then the conversion ratio .

[0068] 2) Normalization operation: traverse each pixel (x, y) of the image, set the original pixel value to p(x, y), and the normalized pixel value to , so that If you want to map to the interval [and 1,1], you can first perform the above mapping to the interval [0,1], and then use the formula to convert.

[0069] Step 2: According to the requirements of copyright protection, determine the content of watermark information and convert this information into a format suitable for embedding into the image.

[0070] Step 2.1: Determine the watermark information content.

[0071] Let the watermark information be W, which is a set of multiple information items, which can be expressed as W={owner,reg_no,create_date,...}, where owner is the full name of the copyright owner, reg_no is the copyright registration number, and create_date is the creation date. Other information that may be added, such as the work version number version and the authorized use range auth_range, can also be included in this set.

[0072] Step 2.2: Information encoding conversion.

[0073] 1) Convert character encoding to digital encoding (taking ASCII code as an example): Let the ASCII code value of character c be a(c). For each character in the watermark information (i=1,2,...,n, n is the total number of watermark information characters), the corresponding digital code ).

[0074] 2) Convert digital code to binary: Let the function that converts a number m into k-bit binary be b(m,k). For each digital code , converted to binary is b( ,8) (Assuming 8-bit binary representation is used, it can be adjusted according to actual needs). Connecting the binary codes of all characters to obtain the binary sequence B of the watermark information can be expressed as ,in Represents a join operation.

[0075] Step 2.3: Coding review and testing.

[0076] Suppose the compatibility check function is check_compatibility(B, embedding_algorithm), and its return value is True or False, indicating whether the watermark information encoding B is compatible with the watermark embedding algorithm embedding_algorithm.

[0077] 1) Test by simulating the watermark embedding and extraction process. Suppose the simulated embedding function is simulate_embed(I,B), where I is the test image and B is the binary sequence of the watermark information. The function returns the image after embedding the watermark. .

[0078] 2) Set the simulation extraction function to simulate_extract( ), which is obtained from the image after embedding the watermark Extract watermark information binary sequence .

[0079] 3) Check whether the extracted watermark information is consistent with the original watermark information. You can define a comparison function compare(B, ), if compare(B, )=True, it means that the watermark information can be accurately embedded and completely restored in this test case.

[0080] Step 3: Through the synchronous watermark embedding algorithm in time domain and frequency domain, the changes of the image are monitored in the time domain, and the appropriate frequency is selected in the frequency domain to embed the watermark.

[0081] Step 3.1: Establish an image change monitoring module to track the changes in the pixel values ​​of the image in real time.

[0082] By calculating the difference in pixel values ​​between adjacent frames or adjacent areas, it detects whether the image has been cropped, modified, translated, etc., while monitoring changes in image size and recording information such as image scaling. The image change monitoring module is built based on the precise analysis of image pixel values. Its core principle is to detect any changes in the image in the time domain by comparing pixel values ​​at different times or in different areas. For a video image sequence, the relationship between adjacent frames can reflect the dynamic changes in the image content; for a single image, the comparison of pixel values ​​after dividing the area can reveal local modifications inside the image.

[0083] Calculation of pixel value differences between adjacent frames (video image sequence).

[0084] Suppose the first The frame image is , No. The frame image is , for the image coordinates ) pixel, the difference between the pixel values ​​of adjacent frames .

[0085] Assume the total number of pixels is , the statistical difference exceeds the threshold The number of pixels is , then calculate the ratio .like , it is preliminarily determined that the image may have been cropped, modified, or translated.

[0086] Calculation of mean difference of pixel values ​​in adjacent regions (single image).

[0087] For a single image , divided into small areas of 8×8 pixels, set area The coordinates of the upper left corner are ,area The mean of the inner pixel values .

[0088] Set adjacent area and , whose means are and , then the difference between the means of adjacent regions is .Will With threshold For comparison, if , we further analyze the changes in this area.

[0089] Set adjacent area and , whose means are and , then the difference between the means of adjacent regions is .Will With threshold For comparison, if , we further analyze the changes in this area.

[0090] Image scaling calculation.

[0091] Suppose the width of the image at a certain moment is , the height is , and at another moment the width is , the height is , the width scaling ratio , height scaling The scaling uniformity index can be further defined ,like If it is larger, it means that the image has been non-proportionally scaled.

[0092] Step 3.2: Use Fast Fourier Transform (FFT) to transform the image into frequency domain space, analyze the spectral characteristics of the image, and select the appropriate frequency component as the watermark embedding position according to the frequency characteristics of the watermark information and the distribution of the image spectrum.

[0093] The use of fast Fourier transform (FFT) to convert an image from time domain to frequency domain is based on the principle of Fourier analysis. , FFT decomposes it into a combination of sine and cosine waves of different frequencies.

[0094] Suppose the two-dimensional digital image is , whose size is M×N (M is the number of rows and N is the number of columns).

[0095] First, perform a one-dimensional FFT calculation on each row of the image. Row (0≤ <M), its FFT result (u=0,1,...N and 1), where .

[0096] Then perform a one-dimensional FFT calculation on each column of these intermediate results to finally obtain the spectrum of the image (u=0,1,...N and 1, v=0,1,...M and 1).

[0097] By calculating the distribution of image energy in the frequency domain, a basis is provided for selecting the appropriate watermark embedding position.

[0098] When calculating the spectrum energy distribution, the formula Calculate each frequency component energy.

[0099] Assume that the frequency domain is divided into K frequency bands (for example, three frequency bands of low frequency, medium frequency, and high frequency, which are set as K for general representation here), and the frequency range of frequency band k (1≤k≤K) is , then the energy in frequency band k , the ratio of the energy in this frequency band to the total energy .

[0100] Selecting appropriate frequency components as the watermark embedding position according to the frequency characteristics of the watermark information and the distribution of the image spectrum is the key to ensuring the effectiveness and stability of the watermark.

[0101] Suppose the frequency characteristic function of the watermark information is W(f), which represents the characteristics of the watermark information at the frequency, such as the intensity or importance (for low-frequency signals, the W(f) value is larger in the low-frequency area; for watermarks containing high-frequency components, there is a certain value in the corresponding high-frequency area).

[0102] When selecting the watermark embedding position, calculate each frequency component Weight ) (Here it is assumed that the frequency is in the two-dimensional frequency domain and the coordinate The relationship is , which can actually be adjusted according to the specific frequency definition method).

[0103] Then choose the weight The larger frequency component is used as the watermark embedding position, for example, ( is the threshold value, which is determined based on experiments and watermark characteristics) Position embedded watermark.

[0104] Step 3.3: Use a specific encoding method to embed the watermark information into the selected frequency domain coefficients.

[0105] Before embedding the watermark, the image is pre-analyzed and the watermark embedding strength is adaptively adjusted according to the local complexity and visually important areas of the image.

[0106] Assume the local complexity of the image is ( is the image pixel coordinate), the visual importance region function is (In important areas The value is larger in the non-important area, and the value is smaller in the non-important area), then the watermark embedding strength adjustment factor ( is the initial strength factor, which can be set based on experience); For phase modulation based coding, the actual phase adjustment amount during embedding is actual( is the basic phase adjustment amount mentioned above ), then the modified phase is actually , the new frequency domain coefficients .

[0107] For amplitude modulation coding, the amplitude adjustment coefficient is actually embedded ( Adjust the basic coefficients for amplitude, such as , corresponding to the basic 1.2 and 0.8 adjustments), when When the modified amplitude , the new frequency domain coefficients ;when hour, , .

[0108] The coding method based on phase modulation utilizes the phase sensitivity of frequency domain coefficients to embed watermark information.

[0109] The coding method based on phase modulation has the advantage of having relatively little impact on the visual quality of the image, because the human eye is less sensitive to phase changes than amplitude changes.

[0110] Assume the binary bits of watermark information are ( =1,2,...,K, K is the length of the watermark sequence), for the frequency domain coefficient ,when When the phase increase , then the modified phase , the new frequency domain coefficients .

[0111] when When the phase decreases , then the modified phase , the new frequency domain coefficients .

[0112] Amplitude modulation coding adjusts the amplitude of the frequency domain coefficients according to the value of the watermark information.

[0113] The advantage of amplitude modulation coding is that watermark extraction is relatively simple and direct, but it is easy to affect the visual quality of the image, especially when the embedding strength is large.

[0114] When the watermark information When the frequency domain coefficient , the modified amplitude , the new frequency domain coefficients .

[0115] when When the modified amplitude , the new frequency domain coefficients .

[0116] After embedding the watermark, the image is post-processed to make the watermarked image as visually similar to the original image as possible, thereby improving the concealment of the watermark.

[0117] Assume that the original pixel value of the image at the coordinate (x, y) is I(x, y), and the average pixel value of the local area (set as a 3×3 neighborhood N(x, y)) is , the standard deviation is .

[0118] Enhanced pixel value ( is the contrast enhancement factor, which is determined according to the image characteristics and visual quality requirements, such as ), in this way, the local contrast of the image is improved, the visual quality of the image after watermark embedding is better, and the concealment of the watermark is improved.

[0119] Step 3.4: During the embedding process, a correlation model between the time domain and the frequency domain is established to achieve synchronous embedding of the watermark in the time domain and the frequency domain, thereby enhancing the stability and recoverability of the watermark when it is attacked.

[0120] Establishing the mapping function M between time domain and frequency domain is the key to achieve synchronous watermark embedding.

[0121] This mapping function needs to be constructed based on the type of image changes in the time domain and the characteristics of the frequency domain.

[0122] Assume that the image in the time domain is cropped horizontally Pixel, the original embedding position of the watermark in the horizontal frequency direction in the frequency domain is , according to the mapping function M, the position after translation .here is the scale factor, which is determined as follows: Suppose a series of test images are cropped with different numbers of horizontal pixels ( , is the number of experiments), and records the translation amount in the corresponding frequency domain when the watermark extraction effect is the best . Through linear regression and other methods to fit The value of satisfy .

[0123] For phase modulation, let the original phase adjustment be (For example, when the watermark is embedded, the watermark information is set to 1 or 0. ), the cropping ratio is ( is the original image width), then the adjusted phase change At this time, when the embedded watermark is 1, the modified phase becomes , the new frequency domain coefficients ; When the embedded watermark is 0, , .

[0124] For amplitude modulation, let the original amplitude adjustment coefficient be (such as 1.2 or 0.8), then the adjusted amplitude coefficient When the watermark information bit is 1, the modified amplitude becomes , the new frequency domain coefficients ; When the watermark information bit is 0, , .

[0125] Establishing constraints is crucial to ensure the stability of watermark embedding in both time and frequency domains.

[0126] Ensuring the conservation of watermark energy in the time domain and frequency domain is an important constraint. When the watermark energy changes due to operations such as scaling the image in the time domain, the watermark in the frequency domain adjusts the embedding coefficient to keep the total energy unchanged or fluctuate within a certain range.

[0127] Assume that the image is scaled in the time domain, and the scaling ratio is ( Indicates shrinkage, represents amplification), when amplitude modulation coding is used, the amplitude of the original watermark after embedding is .

[0128] For the downscaling case, the amplitude of the watermark in the frequency domain after scaling should be adjusted to , at this time the watermark energy in the frequency domain ( is the amplitude of the frequency domain coefficient after scaling), and the original watermark energy Compared with, satisfying (Within a certain error range, it can be set according to actual needs) to achieve watermark energy conservation.

[0129] For the case of magnification, the amplitude of the watermark in the frequency domain after scaling should be adjusted to , it can also be verified that the energy relationship meets the conservation requirements.

[0130] At the same time, other constraints can also be established, such as the consistency constraint of the watermark information in the time domain and the frequency domain, to ensure that the watermark information will not be lost or erroneous during the conversion process between the time domain and the frequency domain.

[0131] At the same time, other constraints can also be established, such as the consistency constraint of the watermark information in the time domain and the frequency domain, to ensure that the watermark information will not be lost or erroneous during the conversion process between the time domain and the frequency domain.

[0132] Assume that the watermark information embedded in the time domain is ( is the watermark sequence length), the watermark information extracted in the frequency domain is . Define the consistency function , in an ideal situation, , indicating that the watermark information is not lost or erroneous during the conversion process between the time domain and the frequency domain. In practical applications, a threshold is set (For example ),when When , it is considered that the watermark information remains consistent in the time domain and frequency domain, satisfying the constraints.

[0133] By establishing a time domain and frequency domain correlation model, the watermark is embedded synchronously in the time domain and frequency domain, which greatly enhances the stability and recoverability of the watermark when it is attacked. However, due to the synchronous embedding of the watermark in the time domain and frequency domain, in the time domain, the watermark can be adaptively adjusted according to the noise and color changes of the image; in the frequency domain, the watermark can adapt to the change of resolution. Through the corresponding extraction algorithm, the watermark information can be accurately restored.

[0134] Step 4: Save the image with watermark embedded after the above series of processing. The saving format can be selected according to actual needs, such as common JPEG, PNG and other formats.

[0135] After a series of processing such as the above-mentioned improved singular value decomposition (ISVD), time domain and frequency domain synchronous watermark embedding (TFSW), etc., an image with embedded watermark is obtained. When saving the image, select the appropriate image format according to the actual application scenario and needs. If the image needs to be transmitted quickly on the network and has high requirements for storage space, the JPEG format can be selected. The JPEG format uses a lossy compression algorithm, which can reduce the image file size to a large extent, but will lose a certain amount of image quality. When saving in JPEG format, it is necessary to pay attention to setting a suitable compression ratio, which can generally be determined through experiments. If the image quality requirements are high and features such as transparent background need to be supported, such as for design materials or professional image editing scenarios, the PNG format can be selected. The PNG format uses lossless compression, which can better preserve image details and watermark information, but the file size is relatively large.

[0136] During the saving process, ensure that the relevant metadata information of the image is fully preserved. For watermark embedding parameters, include the size of the image block in the ISVD operation (such as 8×8 pixels or 16×16 pixels), the strength factor of the singular value modification (such as the value used to modify the singular value size when embedding the watermark), the watermark embedding frequency component selected in the TFSW stage (such as the specific frequency range or specific frequency band in the low-frequency area), etc. At the same time, clearly record copyright information, such as the name of the copyright owner, copyright registration number, creation date, etc. These metadata information can be stored in the header of the image file or a dedicated metadata area, and stored in a specific encoding format for easy subsequent reading and parsing.

[0137] Step 5: Use a deep learning-based extraction framework to identify watermark features in damaged images by training the model.

[0138] Step 5.1: The noise-resistant and compression-resistant feature extraction layer suppresses the noise in the image and compensates for the information loss caused by compression through the pre-learned noise and compression patterns, so that the watermark features can still be accurately extracted even when the image quality is degraded.

[0139] Noise suppression (Gaussian noise and mean filter as examples).

[0140] Set image The neighborhood of pixel (x, y) is N(x, y), and the kernel of the mean filter is .

[0141] Pixel value after passing through the mean filter .

[0142] For Gaussian noise, the probability density function is ( is the mean, is the standard deviation), the anti-noise feature extraction layer estimates the noise parameters by learning a large number of images with Gaussian noise added, so as to suppress the noise in actual detection.

[0143] Compensation for compressed information (taking JPEG compression and bilinear interpolation as examples).

[0144] Suppose the frequency coefficient matrix of a 8×8 block of the JPEG compressed image after quantization is C, and its low-frequency part (assuming the upper left corner) and the high frequency part (lower right corner). When high-frequency information is lost, low-frequency information is used to restore high-frequency information.

[0145] Let the bilinear interpolation function be , for the pixel position where high-frequency information is lost , the restored pixel value (here The function calculates the target pixel value through bilinear interpolation based on the surrounding low-frequency pixel values).

[0146] Step 5.2: Input the image processed by anti-noise and anti-compression features into the pre-trained convolutional neural network (CNN). CNN uses its convolutional layer and pooling layer structures to automatically extract potential watermark features in the image.

[0147] Convolutional layer operation (taking 3×3 convolution kernel as an example).

[0148] Let the input image be I and the convolution kernel be , for each pixel (x, y) in the image, the output eigenvalue F(x, y) of the convolutional layer (on a certain feature channel) is calculated as follows:

[0149] .

[0150] here Indicates that the convolution kernel is The weight value of the position. By sliding the convolution kernel on the image, the feature map corresponding to the convolution kernel can be obtained. Different convolution kernels (such as 3×3 and 5×3, etc.) can obtain multiple feature maps through similar calculations. These feature maps together constitute the convolution layer output, which contains feature information of different scales of the image.

[0151] Pooling layer operation (taking 2×2 maximum pooling as an example).

[0152] Assume the pooling window size is 2×2, for the feature map of the pooling layer input , output feature map The calculation of is as follows:

[0153] .

[0154] That is, the maximum value in each 2×2 window is selected as the output, achieving the purpose of dimensionality reduction and retaining key features.

[0155] Step 5.3: The features extracted by CNN are input into the Multi-Layer Perceptron (MLP), which further processes and analyzes the features through its multiple hidden layers, and gradually extracts higher-level and more abstract watermark feature representations.

[0156] Neuron weighted summation and nonlinear transformation (with the first hidden layers as an example).

[0157] Set up The feature vector output by the layer is ) (n is the number of features), The connection weight matrix of the layer is ( =1,2, Indicates The number of neurons in a layer, =1,2, ), the bias vector is .

[0158] No. Layer of neurons The weighted sum of input .

[0159] Output after ReLU activation function .

[0160] Through multiple layers of such calculations (from the input layer to multiple hidden layers), the original image features are gradually transformed into more abstract watermark feature representations.

[0161] Step 5.4: Decode and identify the extracted watermark features, restore them to the original watermark information, and compare them with the watermark information registered in the database to accurately determine the copyright of the image, thereby achieving effective protection of digital image copyright.

[0162] Watermark decoding (taking binary sequence decoding as an example).

[0163] Suppose the extracted feature vector is , if the watermark is embedded by modifying the singular value size during encoding, let the encoding rule be ( is a decoding function determined according to the embedding rule, for example, the original watermark bits are calculated in reverse according to the singular value modification rule).

[0164] Get the decoded binary sequence .

[0165] Copyright ownership judgment (taking Hamming distance as an example).

[0166] Suppose the watermark information registered in the database is , the extracted watermark information is .

[0167] Hamming distance .

[0168] like ( is the set threshold), the copyright of the image is considered to belong to the owner of the registered watermark information; otherwise, it is considered that the image may have copyright issues.

[0169] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A digital image hybrid watermarking method based on time domain and frequency domain synchronization, characterized in that: The following steps are involved: S1, read the digital image that needs to be watermarked and pre-process the image; S2. Determine the watermark information content according to the copyright protection requirements and convert the information into a format suitable for embedding into the image; S3, using the time domain and frequency domain synchronous watermark embedding algorithm, monitor the changes in the time domain of the image, and select the appropriate frequency in the frequency domain to embed the watermark; S4, saving the processed image with embedded watermark, and the saving format can be selected according to actual needs; S5. Use a deep learning-based extraction framework to identify watermark features in damaged images by training the model.

2. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 1 is characterized in that: The step S1 of reading the digital image to be watermarked and preprocessing the image specifically includes the following steps: S11, read the image and obtain basic information; S12, format conversion; S13, size adjustment; S14: Normalization processing.

3. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 1 is characterized in that: In step S2, the watermark information content is determined according to the copyright protection requirements, and the conversion of the watermark information into a format suitable for embedding into an image specifically includes the following steps: S21, determining the watermark information content; S22, information coding conversion; S23. Coding review and testing.

4. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 1 is characterized in that: In step S3, the watermark embedding algorithm in the time domain and the frequency domain is synchronized to monitor the change in the time domain of the image, and the watermark embedding process is performed by selecting a suitable frequency in the frequency domain. Specifically, the following steps are included: S31, establish an image change monitoring module to track the changes in the pixel values ​​of the image in real time; S32, using fast Fourier transform to convert the image into frequency domain space, analyzing the frequency spectrum characteristics of the image, and selecting appropriate frequency components as watermark embedding positions according to the frequency characteristics of the watermark information and the distribution of the image spectrum; S33, using a specific encoding method to embed the watermark information into the selected frequency domain coefficients; S34. During the embedding process, a correlation model between the time domain and the frequency domain is established to achieve synchronous embedding of the watermark in the time domain and the frequency domain, thereby enhancing the stability and recoverability of the watermark when it is attacked.

5. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 4 is characterized in that: The step S32 specifically includes the following steps: S321, converting the image from the time domain space to the frequency domain space by using fast Fourier transform; S322, providing a basis for selecting a suitable watermark embedding position by calculating the distribution of image energy in the frequency domain; S323. Select appropriate frequency components as watermark embedding positions according to the frequency characteristics of the watermark information and the distribution of the image spectrum.

6. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 4 is characterized in that: The step S33 specifically includes the following steps: S331, before embedding the watermark, pre-analyze the image, and adaptively adjust the watermark embedding strength according to the local complexity and visual importance area of ​​the image; S332, a phase modulation-based coding method utilizes the sensitivity of the phase of the frequency domain coefficient to embed watermark information; S333, amplitude modulation coding adjusts the amplitude of the frequency domain coefficient according to the value of the watermark information; S334. After embedding the watermark, post-process the image to make the watermark-embedded image as visually similar as possible to the original image.

7. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 1 is characterized in that: In step S4, the processed image with embedded watermark is saved, and the saving format can be selected according to actual needs. The specific operation steps are: if the image needs to be transmitted quickly on the network and has high requirements for storage space, select the JPEG format; if the image quality requirements are high and support for transparent background is required, select the PNG format.

8. The digital image hybrid watermarking method based on time domain and frequency domain synchronization according to claim 1 is characterized in that: In step S5, the extraction framework based on deep learning is adopted, and the watermark features in the damaged image are identified by training the model, which specifically includes the following steps: S51, the anti-noise and anti-compression feature extraction layer suppresses the noise in the image and compensates for the information loss caused by compression through the pre-learned noise and compression patterns; S52, inputting the image processed by anti-noise and anti-compression features into a pre-trained convolutional neural network, and the convolutional neural network automatically extracts potential watermark features in the image by using its convolutional layer and pooling layer and other structures; S53, inputting the features extracted by the convolutional neural network into a multilayer perceptron, and the multilayer perceptron further processes and analyzes the features through its multiple hidden layers; S54, decoding and identifying the extracted watermark features, restoring them to the original watermark information, and comparing them with the watermark information registered in the database to determine the copyright of the image.

Images