Image processing method and device thereof

By determining non-significant areas in the frequency domain image of the video image for watermark embedding and extraction, the encoding efficiency reduction and alignment of the embedding end and the detection end caused by watermark embedding position transmission in the prior art is solved, and efficient watermark embedding and extract.

CN113763224BActive Publication Date: 2024-11-08ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010494066.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-03
Publication Date
2024-11-08
Estimated Expiration
2040-06-03

AI Technical Summary

Technical Problem

In the prior art, when embedding watermarks in video images, the watermark embedding position information is required, which increases the encoding bandwidth and network load, reduces the encoding efficiency, and is difficult to solve the problem of significant area alignment between the embedded end and the detection end.

Method used

By acquiring the frequency domain image of the image to be processed, determining the non-significant area, selecting the embedding block to perform watermark embedding, and performing watermark embedding and extraction without relying on the watermark embedding position, realizing adaptive selection of embedding blocks to reduce the embedding end Align with the significance area at the detection end.

Benefits of technology

It effectively reduces the problem of the significant area alignment between the watermark at the embedded end and the detection end, improves the consistency between the embedded end and the detection end, and reduces the encoding bandwidth and network load.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113763224B_ABST
    Figure CN113763224B_ABST
Patent Text Reader

Abstract

The present application discloses an image processing method and a device thereof, the method comprising: obtaining a frequency domain image corresponding to an initial image to be processed; determining a non-significant area in the frequency domain image; determining an embedding block for watermark embedding from the non-significant area; and embedding the watermark on the embedding block to generate a watermarked image. By adopting the present application, the consistency between the embedding end and the detection end can be improved by determining the embedding block for watermark embedding from the non-significant area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method and device thereof. Background Art

[0002] With the continuous development of digital media technology and networks, including the increasing popularity of 5G, digital data, especially media data such as digital images and videos, has shown explosive growth. As an advanced and effective security technology, digital watermarking technology has become an important research direction in the field of data security and intellectual property protection.

[0003] In the related art, in order to ensure the synchronization between the watermark embedding end and the detection end, the watermark embedding end embeds the watermark image at a specified position (i.e., the watermark embedding position) of the video image and transmits the position information of the watermark embedding position to the watermark detection end. However, transmitting the watermark embedding position will bring additional bit rate to the video encoding, increase the bandwidth and network load, and thus reduce the encoding efficiency. Therefore, the related art requires a technical solution that does not rely on the watermark embedding position and can ensure the synchronization of the horizontal embedding end and the detection segment. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and a device thereof, which at least solve the above-mentioned technical problems.

[0005] An embodiment of the present application also provides an image processing method, which includes: obtaining a frequency domain image corresponding to an initial image to be processed; determining a non-significant area in the frequency domain image; determining an embedding block for performing watermark embedding from the non-significant area; and performing watermark embedding on the embedding block to generate a watermarked image.

[0006] An embodiment of the present application also provides an image processing method, which includes: obtaining a frequency domain image corresponding to an initial image to be processed; determining a non-significant area in the frequency domain image; determining an embedding block for performing watermark embedding from the non-significant area; and extracting watermark information from the embedding block.

[0007] An embodiment of the present application also provides a computer-readable storage medium on which computer instructions are stored, characterized in that the above method is implemented when the instructions are executed.

[0008] An embodiment of the present application further provides an image processing device, which includes: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, enable the processor to perform the above method.

[0009] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0010] The method realizes adaptive selection of embedding blocks by determining the embedding blocks for watermark embedding from the non-salient areas, thereby effectively alleviating the problem of aligning the salient areas of the watermark at the embedding end and the detection end. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 is a scene graph for image processing according to an exemplary embodiment of the present application;

[0013] Figure 2 is a flowchart of steps of an image processing method according to an exemplary embodiment of the present application;

[0014] Figure 3 is a framework diagram of an image processing method according to another exemplary embodiment of the present application;

[0015] Figure 4 is a diagram showing that an embedding end embeds watermark information according to an image processing method of an exemplary embodiment of the present application;

[0016] Figure 5 is a diagram showing that a detection end extracts watermark information according to an image processing method of an exemplary embodiment of the present application;

[0017] Figure 6 is a comparison diagram of the effects of an embedding end and a detection end on a saliency mask according to an exemplary embodiment of the present application;

[0018] Figure 7 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0021] In order to better illustrate this application, the following Figure 1Describing the scene graph of image processing, it should be noted that the image processed by the image processing method according to the exemplary embodiment of the present application is much larger than Figure 1 As shown in Figure 1 The main components are described only to illustrate the application scenario of this application. This application is applicable to Figure 1 various scenes.

[0022] like Figure 1 As shown, user A creates a video image 10 using an electronic terminal 100. Subsequently, as a creator, user A can request image protection from a server 101 that provides image copyright protection. Subsequently, server 101 can perform image processing on the image to provide copyright protection. In other words, a watermark image is embedded in the video image 10 to provide copyright protection.

[0023] Subsequently, the server 101 can store the image with the embedded digital watermark. Figure 1 In the embodiment, server 101 is used to store images embedded with digital watermarks. However, in implementation, a server different from 101 can be used for storage. For example, a user creates an image on an application (e.g., a cartoon website). The server corresponding to the application can apply for copyright protection from server 101 dedicated to implementing image copyright protection. Server 101 can embed a digital watermark into the image as described above. Subsequently, the server corresponding to the application can obtain the image embedded with the digital watermark from server 101 and store the image. It can be seen that in this example, the server that performs copyright protection and the server that stores the image can be different.

[0024] Subsequently, the server 101 may provide the image 10 to other servers 200 according to the settings of the user A (for example, the user A may set the image 10 to be presentable but not downloadable). In this case, when the user B views the image 20 similar to the image 10 through the electronic device 300, the server 101 may extract the image 20, and then extract the digital watermark from the image 20 to determine whether the image 20 is an unauthorized image 10.

[0025] It should be noted that the electronic device according to the present application (hereinafter referred to as an image processing device that performs an exemplary embodiment of the present application) is a device including a display unit, and may include but is not limited to any of the following devices: a personal computer (PC), a mobile device (such as a cellular phone, a personal digital assistant (PDA), a digital camera, a portable game console, an MP3 player, a portable / personal multimedia player (PMP), a handheld e-book, a tablet PC, a portable laptop PC, and a global positioning system (GPS) navigator), a smart TV, etc. Although in Figure 1 The server 101 is a specific server, but in practice it may be implemented by multiple servers.

[0026] In the prior art, watermark embedding methods can be divided into blind processing and non-blind processing. The watermark embedding method using non-blind processing requires the exact location of the embedded watermark, which will increase the network burden of transmitting images. The watermark embedding method using blind processing usually uses a non-overlapping sliding window after determining the area to embed the watermark, and performs watermark embedding on different areas according to different embedding strengths, but this method is prone to misalignment between the embedding end and the detection end.

[0027] Based on this, the image processing method of the exemplary embodiment of the present application, while adopting blind processing, performs screening on the non-significant areas in the frequency domain image, determines the embedding blocks in which watermark embedding can be performed, and performs watermark embedding on the embedding blocks, thereby improving the consistency between the embedding end and the detection end.

[0028] The following will be combined with Figures 2 to 4 The technical solutions involved are described in detail. Figure 2 A flowchart showing the steps of an image processing method according to an exemplary embodiment of the present application is shown.

[0029] like Figure 2 As shown, in step S210, a frequency domain image corresponding to the initial image to be processed is obtained. In implementation, the initial image to be processed may be a single-frame image, including but not limited to images created by users (e.g., cartoons, paintings, etc.) or documents in various formats, or a single-frame video image in a video.

[0030] Specifically, after the initial image is obtained, the channel image corresponding to the brightness information can be obtained by extracting the brightness information of the initial image. As an example, the initial image can be a YUV image, which is a color encoding method mainly used in analog video and television systems, where Y represents brightness information (Luminance or Luma), and U and V represent chrominance (Chrominance or Chroma). In the case where the initial image is a YUV image, the Y channel in the initial image is extracted to obtain the channel image corresponding to the Y channel.

[0031] Subsequently, the channel image can be transformed in the frequency domain. In implementation, a discrete wavelet transform can be performed on the channel image to generate multiple sub-images corresponding to the initial image. Wavelet transform (discrete wavelet transform, DWT) uses a series of wavelets of different scales to decompose the original function, and the coefficients of the original function under wavelets of different scales are obtained after the transformation.

[0032] In an exemplary embodiment of the present application, wavelet transform can be performed on the channel image in the following manner: first, discrete wavelet transform is performed on each row of the channel image to obtain the low-frequency component L and high-frequency component H of the channel image in the horizontal direction, and then discrete wavelet transform is performed on each column of the transformed data, and finally multiple sub-images of the channel image are obtained, including: a low-frequency component sub-image (i.e., a coarse sub-image) LL in the horizontal and vertical directions, a low-frequency sub-image in the horizontal direction and a high-frequency sub-image in the vertical direction (horizontal detail sub-image) LH, a high-frequency sub-image in the horizontal direction and a low-frequency sub-image in the vertical direction (i.e., a vertical detail sub-image) HL, and a high-frequency component sub-image in the horizontal and vertical directions (i.e., a diagonal detail sub-image) HH.

[0033] In an embodiment of the present application, a sub-image including detail information may be selected from the sub-images as a frequency domain image, and preferably, a horizontal detail sub-image may be selected from the plurality of sub-images as a frequency domain image. Optionally, both the horizontal detail sub-image and the diagonal detail sub-image may be used as frequency domain images.

[0034] Step S220 is executed to determine a non-salient area in the frequency domain image.

[0035] Specifically, the visual attention model can be used to perform attention region segmentation on the initial image to obtain a saliency mask corresponding to a saliency region, wherein the visual attention model refers to simulating a human attention mechanism to obtain an image of interest in the image.

[0036] In addition, in the implementation, other methods may be used to determine the non-significant area in the frequency domain image. For example, the image name may be used to determine the non-significant area in the frequency domain image. Specifically, after the image name of the image is identified, the image recognition technology may be used to identify the area corresponding to the image name, and the area is determined as a significant area, and the area other than the area in the image is a non-significant area. Alternatively, the text area in the image may be identified, and the text area is determined as a non-significant area, and the area other than the text area in the image is a significant area.

[0037] In an exemplary embodiment of the present application, a visual attention model based on texture and brightness may be used to generate the attention area of ​​the initial image. Subsequently, a binarized Otsu threshold segmentation process may be performed on the attention area to obtain the saliency mask.

[0038] The mask mentioned here refers to the binary classification marking of the pixels on the processed image to control the area of ​​image processing. For example, by using the preset region of interest mask and the corresponding image on each pixel one by one, it can be known whether each pixel on the image is in the region of interest.

[0039] In an exemplary embodiment of the present application, in order to obtain a saliency mask, the maximum inter-class variance method may be used to process the attention area, thereby obtaining the saliency mask. The maximum inter-class variance method may also be called the Otsu algorithm (i.e., OTSU), which is an algorithm for determining an image segmentation threshold. In short, the image is divided into two parts, background and target, according to the grayscale distribution of the image, so that the inter-class variance between the background and the target is maximized, wherein the definition of the inter-class variance is as shown in Formula 1:

[0040]

[0041] Among them, σ 2 (k) is the inter-class variance, k* is the optimal classification threshold that maximizes the inter-class variance. ω0 and ω1 are the probabilities of the two classes, μ0 and μ1 are the mathematical expectations of the two classes. Applying the Otsu threshold segmentation method to the segmentation of the attention area can maximize the difference between the segmented non-significant area and the significant area, making the non-significant area less prominent.

[0042] Subsequently, the non-salient region is obtained by combining the saliency mask with the frequency domain image. Specifically, the saliency mask is downsampled to obtain the downsampled saliency mask, so that the resolution of the downsampled saliency mask is the same as the resolution of the frequency domain image, and then the downsampled saliency mask is matched with the frequency domain image in one-to-one correspondence at each pixel point to obtain the non-salient region.

[0043] Step S230 is executed to determine an embedding block for performing watermark embedding from the non-salient area. As an example, the non-salient area may be divided into sub-blocks of a predetermined size, and then the embedding block for performing watermark embedding is selected from the sub-blocks.

[0044] Specifically, the non-salient area can be divided into multiple non-overlapping sub-blocks of 8x8. In implementation, if the length or width of the non-salient area cannot be divided by 8, the pixels of the salient area need to be included so that the non-salient area can be completely divided into blocks.

[0045] For each sub-block, the number of salient pixel points in the salient mask corresponding to each sub-block may be determined, and the sub-blocks with the number less than or equal to a predetermined threshold are determined as embedding blocks for watermark embedding. Specifically, as shown in the following formula 2:

[0046]

[0047] Mask m It refers to the saliency mask corresponding to the mth sub-block, and the value f after summing its 64 pixels m represents the size of the intersection of the sub-block area and the salient area, and L is the preset threshold. mDetermine the sub-block for embedding watermark, that is, if the number of significant pixels contained in the 64 pixels corresponding to the sub-block is less than or equal to L, then the sub-block is considered to be an embedded block for embedding watermark, otherwise the current sub-block is considered to be in a significant sub-block and no watermark is embedded.

[0048] Step S240 is executed to embed a watermark on the embedding block to generate a watermarked image.

[0049] Before watermark embedding is performed, watermark information corresponding to the initial image may be generated. Usually, the identification information of the initial image may be used as the watermark information, and in order to allow other users to intuitively know whether the initial image is infringed, the identification information of the author of the initial image (for example, the author's name, identity identification number, etc.) may be used as the watermark information. Subsequently, the identification information may be converted into a binary image, wherein the binary image means that each pixel on the image has only two possible values, for example, a black and white image, and then the binary image may be subjected to image scrambling processing to generate a scrambled binary image as the watermark information, wherein the image scrambling processing may be Arnold scrambling, and in addition, the scrambled binary image may be subjected to a redundancy mechanism, and redundant information may be generated, and then the redundant information may be embedded into the image using a subsequent algorithm, for example, when the watermark information is 00001111, the redundant information is 000011110000111100001111. This makes it impossible for other users to obtain the real watermark information.

[0050] In the case where the watermark information has been obtained, watermark embedding can be performed on the embedding block. Discrete cosine transform is performed for each embedding block to obtain a discrete cosine coefficient matrix, and then a discrete cosine coefficient pair is selected from the discrete cosine coefficient matrix. Preferably, three pairs of discrete cosine coefficient pairs can be selected. In the 8*8 matrix, each coefficient can be sorted in a preset manner, for example, it can be sorted according to the energy size, and then three pairs of energy intermediate frequency coefficients can be selected and their positions in the matrix can be recorded.

[0051] Subsequently, the watermark is embedded in the embedding block using the discrete cosine coefficient pairs to generate a watermarked image. Taking three pairs of discrete cosine coefficient pairs as an example, the three pairs of coefficients can be arranged into one-dimensional vectors (C0, C1, C2, C3, C4, C5), and the six vectors are the even-numbered coefficients C0, C2, C4, and the odd-numbered coefficients C1, C3, C5.

[0052] If the watermark information is 1, the absolute values ​​of coefficients C0, C2, and C4 are set to be larger, greater than the absolute values ​​of C1, C3, and C5. That is, according to the positive or negative nature of the even-digit coefficients, an offset T is added or subtracted from their original values ​​to increase their absolute values; if the watermark is 0, the absolute values ​​of coefficients C0, C2, and C4 are set to be smaller, smaller than the absolute values ​​of C1, C3, and C5. The offset T indicates the watermark embedding strength.

[0053] Through the above processing, a frequency domain image embedded with a watermark can be obtained, and then the frequency domain to time domain conversion processing can be performed on the frequency domain image embedded with a watermark to obtain a watermarked image. Specifically, the embedded block embedded with a watermark can be subjected to an inverse discrete cosine transform, and then the frequency domain image embedded with a watermark is subjected to an inverse wavelet transform to obtain a watermarked image.

[0054] In summary, the image processing method according to the exemplary embodiment of the present application can determine the non-significant area from the frequency domain image corresponding to the initial image, and determine the embedding block for performing watermark embedding from the non-significant area without affecting the main quality of the image, thereby realizing adaptive selection of the embedding block without knowing the position information of the embedding block, thereby effectively alleviating the problem of alignment of the significant areas of the watermark at the embedding end and the detection end.

[0055] In addition, the above description is made by taking a single frame image as an example. In implementation, the above processing may be performed on each frame image in the video, thereby achieving the purpose of copyright protection of the video.

[0056] In order to better describe the image processing method of the exemplary embodiment of the present application, the following will be combined with Figure 3 Preferred embodiments are described. Figure 3 is a framework diagram of an image processing method according to another exemplary embodiment of the present application.

[0057] like Figure 3 As shown in , first, the Y channel is extracted from the initial image to obtain the image corresponding to the Y channel. It should be noted that when the initial image is a grayscale image, this step can be omitted.

[0058] Subsequently, the Y channel image is transformed by DWT. Wavelet decomposition can reduce the dimension of high-resolution video, play a role in concentrating signal energy, and embedding watermarks in high-frequency subbands makes the watermarks more invisible, and the video still has good objective quality after embedding watermarks. Considering the objective quality of the watermarked image and the performance of detecting watermarks, LH is selected as the frequency domain image for embedding watermarks.

[0059] At this point, based on the visual attention model, the saliency mask corresponding to the initial image can be obtained, such as Figure 3As shown in FIG. 1 , the non-salient region in the frequency domain image can be obtained by using the saliency mask and the frequency domain image. Subsequently, the embedding block that can be used to embed the watermark can be determined from the non-salient region. Finally, the watermark is embedded into the embedding block and the frequency domain image with the embedded watermark information is inversely transformed to obtain the image with the watermark information.

[0060] The following will give the initial image as follows Figure 4 The steps shown perform processing, that is, at the embedding end, watermark embedding is performed on the initial image 410 according to the image processing method of the exemplary embodiment of the present application.

[0061] Figure 4 is a diagram showing that the embedding end embeds watermark information according to the image processing method of the exemplary embodiment of the present application. Figure 4 As described above, the Y channel of the initial image 410 may be read to generate the image 420. The image 420 may be subjected to a wavelet transform to generate sub-images LL, HL, LH, and HH, and the like. Figure 4 As shown, LH may be selected as the image 430 for subsequent watermark embedding.

[0062] In addition, the visual attention model described above can be used to process the initial image 410 to generate an image 440, from which a salient area can be identified. Subsequently, in order to obtain a mask of the salient area, the image 440 can be segmented by Otsu thresholding and downsampled to the size of the image 430 to obtain a salient mask 450. Subsequently, the non-salient area sub-blocks in the image 430 can be screened in combination with the above-mentioned formula 2 and the salient mask 450 to determine the embedding block for watermark embedding.

[0063] In the case where the embedded watermark image has been determined, Arnold can be used to scramble the watermark image to generate watermark information as described above, and then the watermark information can be embedded into the corresponding position in the image 460.

[0064] Subsequently, IDCT and IDWT transformations are performed on the image 460 in which the watermark information has been embedded to obtain a watermarked image 470. Finally, the watermarked image 470 can be restored to an image 480 having the same image format as the original image.

[0065] The above has described the watermark embedding process performed by the embedding end on the initial image 410. Figure 5 Describe the watermark extraction process performed by the detection end on the watermarked image 510. Figure 5 is a diagram showing that a detection end extracts watermark information according to an image processing method of an exemplary embodiment of the present application.

[0066] like Figure 5 As shown, for the watermarked image 510, the following Figure 4The steps shown are used to obtain the embedded block for watermark embedding. Subsequently, the watermark information can be extracted from the embedded block in the above-mentioned manner using discrete cosine coefficient pairs.

[0067] Specifically, three pairs of coefficients can be selected from the intermediate frequency part of the embedded block in the 8x8DCT block, and these three pairs of coefficients can be arranged into one-dimensional vectors (C0, C1, C2, C3, C4, C5). These six vectors are the even-numbered coefficients C0, C2, C4, and the odd-numbered coefficients C1, C3, C5. Then, the C in each sub-block is determined. k With C k+1 The absolute value of C k The absolute value is greater than C k+1 , the embedded watermark is 1, otherwise it is 0. Finally, the watermark image is obtained by using Arnold inverse mapping.

[0068] The following will refer to Figure 6 A comparison diagram of the effects of an embedding end and a detection end on a saliency mask according to an exemplary embodiment of the present application is described. Figure 6 (a) in the figure is the saliency mask obtained at the embedding end. Figure 6 (b) in the figure is the saliency mask obtained by the detection end. Figure 6 (c) in the figure is the difference between the two. It can be seen that after the watermark is inserted into the initial image according to the image processing method of the exemplary embodiment of the present application, the synchronization of the watermark position between the embedding end and the detection end is effectively improved.

[0069] In addition, in the exemplary embodiment of the present application, the threshold value pre-set for determining the embedded block will have a significant impact on the synchronization between the embedding end and the detection end. Different threshold values ​​will affect the main quality of the watermarked image. In the implementation, it can be determined that as the threshold value increases, the error of the detected watermarked image gradually decreases, and as the threshold value increases, the difference between the embedding end and the detection end in determining the position of the embedded block decreases, thereby improving the robustness.

[0070] Figure 7 In order to more clearly understand the inventive concept of the exemplary embodiments of the present application, the following will refer to Figure 7 A block diagram of a data processing device describing an exemplary embodiment of the present application. A person of ordinary skill in the art will understand that: Figure 7 The device in the embodiment only shows the components related to the exemplary embodiment, and the device also includes Figure 7 Common components other than those shown in .

[0071] Figure 7 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present application. It should be noted that the image processing apparatus may be a processor that performs Figure 1 The image processing method is shown in the server.

[0072] refer to Figure 7 At the hardware level, the device includes a processor, an internal bus and a computer-readable storage medium, wherein the computer-readable storage medium includes a volatile memory and a non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory and then runs it. Of course, in addition to the software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0073] Specifically, the processor performs the following operations: obtaining a frequency domain image corresponding to an initial image to be processed; determining a non-significant area in the frequency domain image; determining an embedding block for performing watermark embedding from the non-significant area; and performing watermark embedding on the embedding block to generate a watermarked image.

[0074] Optionally, the processor executes the step of obtaining a frequency domain image corresponding to the initial image to be processed, including: extracting brightness information of the initial image to obtain a channel image corresponding to the brightness information; performing a wavelet transform on the channel image to generate a plurality of sub-images corresponding to the initial image; and determining the frequency domain image from the plurality of sub-images.

[0075] Optionally, the processor determines the frequency domain image from the multiple sub-images during the execution step, including: selecting a vertical detail sub-image from the multiple sub-images as the frequency domain image.

[0076] Optionally, the processor determines the non-salient area in the frequency domain image during the execution step, including: using a visual attention model to perform attention area segmentation on the initial image to obtain a saliency mask corresponding to the salient area; and obtaining the non-salient area by combining the saliency mask with the frequency domain image.

[0077] Optionally, the processor obtains the non-significant area by combining the saliency mask with the frequency domain image during the execution step, including: performing downsampling on the saliency mask to obtain the downsampled saliency mask, so that the resolution of the downsampled saliency mask is the same as the resolution of the frequency domain image; and obtaining the non-significant area by making a one-to-one correspondence between the downsampled saliency mask and the frequency domain image at each pixel point.

[0078] Optionally, the processor uses the visual attention model to perform attention area segmentation on the initial image to obtain a saliency mask corresponding to the saliency area during the execution step, including: using the visual attention model to generate the attention area of ​​the initial image; based on the maximum inter-class variance method, segmenting the attention area to obtain the saliency mask.

[0079] Optionally, the visual attention model comprises a texture and brightness based visual attention model.

[0080] Optionally, the processor determines the embedding block for performing watermark embedding from the non-salient area during the execution step, including: dividing the non-salient area into a plurality of sub-blocks of a predetermined size; and selecting the embedding block for performing watermark embedding from the plurality of sub-blocks.

[0081] Optionally, the processor filters out the embedding blocks for performing watermark embedding from the multiple sub-blocks during the execution step, including: obtaining, for each sub-block, the number of significant pixel points corresponding to each sub-block; and determining the sub-blocks whose number is less than or equal to a predetermined threshold as the embedding blocks for performing watermark embedding.

[0082] Optionally, the processor performs watermark embedding on the embedded block to generate a watermarked image in the execution step, including: performing discrete cosine transform on each embedded block to obtain a discrete cosine coefficient matrix; selecting discrete cosine coefficient pairs from the discrete cosine coefficient matrix; and using the discrete cosine coefficient pairs to perform watermark embedding on the embedded block to generate a watermarked image.

[0083] Optionally, the processor uses the transform coefficient pair to perform watermark embedding on the embedded block to generate a watermarked image during the execution step, including: using the transform coefficient pair to perform watermark embedding on the embedded block to obtain a frequency domain image embedded with the watermark; and performing frequency domain to time domain conversion processing on the frequency domain image embedded with the watermark to obtain a watermarked image.

[0084] Optionally, the processor may further perform the following steps: determining identification information corresponding to the initial image; converting the identification information into a binary image; performing image scrambling processing on the binary image to generate a scrambled binary image as watermark information.

[0085] In addition, according to an exemplary embodiment of the present application, the processor may also perform the following steps: obtaining a frequency domain image corresponding to the initial image to be processed; determining a non-significant area in the frequency domain image; determining an embedding block for performing watermark embedding from the non-significant area; and extracting watermark information from the embedding block.

[0086] Optionally, the processor extracts the watermark information from the embedded block during the execution step, including: extracting a watermark matrix from the embedded block by means of discrete cosine transform coefficient pairs; and converting the watermark matrix into a watermark image by means of inverse mapping.

[0087] In summary, according to the exemplary embodiment of the present application, the image processing device can determine the non-significant area from the frequency domain image corresponding to the initial image, and determine the embedding block for performing watermark embedding from the non-significant area without affecting the main quality of the image, thereby realizing adaptive selection of the embedding block without knowing the position information of the embedding block, thereby effectively alleviating the problem of alignment of the significant area of ​​the watermark at the embedding end and the detection end.

[0088] It should be noted that the execution subject of each step of the method provided in Example 1 can be the same device, or the method can be executed by different devices. For example, the execution subject of step 21 and step 22 can be device 1, and the execution subject of step 23 can be device 2; for another example, the execution subject of step 21 can be device 1, and the execution subject of step 22 and step 23 can be device 2; and so on.

[0089] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0094] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0095] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0097] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0098] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a frequency domain image corresponding to an initial image to be processed and a saliency mask corresponding to a saliency region of the initial image; Acquire a non-salient area in the frequency domain image by combining the saliency mask with the frequency domain image; Dividing the non-salient area into a plurality of sub-blocks of predetermined sizes; For each sub-block, determining the number of salient pixels of each sub-block in the saliency mask; Determine the sub-blocks whose number is less than or equal to a predetermined threshold as embedding blocks for performing watermark embedding; Watermark embedding is performed on the embedding block to generate a watermarked image.

2. The method according to claim 1, characterized in that Obtaining the frequency domain image corresponding to the initial image to be processed includes: By extracting the brightness information of the initial image, a channel image corresponding to the brightness information is obtained; Performing wavelet transform on the channel image to generate a plurality of sub-images corresponding to the initial image; The frequency domain image is determined from the plurality of sub-images.

3. The method according to claim 2, characterized in that Determining the frequency domain image from the multiple sub-images comprises: A vertical detail sub-image is selected from the multiple sub-images as the frequency domain image.

4. The method according to claim 1, characterized in that Acquiring a saliency mask corresponding to the saliency region of the initial image includes: Using a visual attention model, attention region segmentation is performed on the initial image to obtain a saliency mask corresponding to a saliency region.

5. The method according to claim 1, characterized in that Acquiring the non-salient area by combining the saliency mask with the frequency domain image comprises: Downsampling the saliency mask to obtain a downsampled saliency mask, so that a resolution of the downsampled saliency mask is the same as a resolution of the frequency domain image; The downsampled saliency mask is matched one-to-one with the frequency domain image at each pixel point to obtain the non-saliency area.

6. The method according to claim 4, characterized in that Using the visual attention model to perform attention region segmentation on the initial image to obtain a saliency mask corresponding to the saliency region includes: Using the visual attention model, generating an attention area of ​​the initial image; Based on the maximum inter-class variance method, the attention area is segmented to obtain the saliency mask.

7. The method according to claim 4, characterized in that The visual attention model includes a texture and brightness based visual attention model.

8. The method according to claim 1, characterized in that Performing watermark embedding on the embedding block to generate a watermarked image includes: Performing discrete cosine transform on each embedded block to obtain a discrete cosine coefficient matrix; Selecting discrete cosine coefficient pairs from the discrete cosine coefficient matrix; The discrete cosine coefficient pair is used to perform watermark embedding on the embedding block to generate a watermarked image.

9. The method according to claim 8, characterized in that Using the transform coefficients to embed watermarks on the embedding blocks to generate watermarked images comprises: Using the transform coefficient pair, embedding a watermark on the embedding block to obtain a frequency domain image embedded with the watermark; The frequency domain image embedded with the watermark is converted from the frequency domain to the time domain to obtain the watermarked image.

10. The method according to claim 1, characterized in that Also includes: Determining identification information corresponding to the initial image; Converting the identification information into a binary image; An image scrambling process is performed on the binary image to generate a scrambled binary image as watermark information.

11. An image processing method, characterized in that: include: Acquire a frequency domain image corresponding to an initial image to be processed and a saliency mask corresponding to a saliency region of the initial image; Acquire a non-salient area in the frequency domain image by combining the saliency mask with the frequency domain image; Dividing the non-salient area into a plurality of sub-blocks of predetermined sizes; For each sub-block, determining the number of salient pixels of each sub-block in the saliency mask; Determine the sub-blocks whose number is less than or equal to a predetermined threshold as embedding blocks for performing watermark embedding; The watermark information is extracted from the embedded block.

12. The method according to claim 11, characterized in that Extracting watermark information from the embedded block includes: extracting a watermark matrix from the embedded block by means of discrete cosine transform coefficient pairs; The watermark matrix is ​​converted into a watermark image by inverse mapping.

13. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed, the method of any one of claims 1 to 12 is implemented.

14. An image processing device, characterized in that: include: processor; as well as A memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method as claimed in any one of claims 1 to 12.