Watermark adding method and application thereof in authentic identification of content played on commercial display screen

Abstract

The invention discloses a watermark adding method and application of the watermark adding method in authenticity identification of playing content of a commercial display screen, a watermark image is generated in a time domain dot matrix calculation mode, and the watermark adding method has remarkable advantages in the aspects of processing speed, robustness and application of the commercial display screen. Compared with a frequency domain calculation or deep learning matching method, the scheme has the advantages that in a watermark generation mode, time domain dot matrix calculation is low in calculation complexity; meanwhile, the scheme is excellent in robustness; according to the scheme, various image features including brightness information, color histograms, frame differences and time domain texture differences are combined. The comprehensive utilization of the multi-dimensional features ensures that the watermark can still maintain stability and identifiability after the image is subjected to zooming, brightness adjustment or other processing operations. Compared with a technology which only depends on a single characteristic, the method has stronger adaptability and anti-interference capability, so that the method can technically show unique advantages in commercial display screen application.

Description

Technical Field

[0001] The present invention relates to the fields of image processing technology, data authentication technology and intellectual property protection technology, and in particular to a watermark adding method and its application in authentication of content played on commercial display screens. Background Art

[0002] Commercial display screens are widely used in modern life, especially in corporate meeting rooms, commercial showrooms, hotel lobbies and other places. It not only provides powerful information transmission and publicity capabilities for corporate customers, but also faces the risk of being attacked by hackers and criminals. The content played on commercial display screens is usually planned and produced in advance, and is carefully designed to attract the target audience and convey effective information. The content is usually played in a loop during the playback process. Although this method can ensure the continuous exposure of information, it also means that once the content is attacked, destroyed or maliciously replaced, it will have a very bad impact.

[0003] In order to avoid and prevent the malicious tampering of the image data of commercial display screens, researchers have proposed three methods for marking counterfeit information: screen watermarking, deep learning, and frequency domain invisible watermarking. Their advantages and limitations are compared as follows:

[0004] (1) Screen watermarks are usually embedded in specific areas of the video; this approach makes the location of the watermark easy to know and thus can be tampered with or removed in a targeted manner. During the playback process of commercial display screens, due to differences in the connected hardware devices, the video may undergo operations such as scaling, rotation, and brightness changes. Traditional screen watermarks are easily affected by these changes, resulting in the loss or damage of watermark information.

[0005] (2) Deep learning-based methods analyze video features by extracting feature vectors. Although deep learning technology performs well in some aspects, its training and detection process requires a lot of computing resources and time. Due to the complexity and computing requirements of deep learning models, it is difficult to achieve real-time monitoring and verification in practical applications, especially in the case of real-time playback on commercial display screens, which makes it difficult to meet the requirements of high efficiency.

[0006] (3) The frequency domain invisible watermark method requires frequency domain transformation of the image to embed and detect watermark information. Frequency domain watermarking has high anti-interference ability in theory, but its implementation process is complicated and the detection process often takes a long time. In practical applications, the real-time detection capability of frequency domain invisible watermarking is limited and it is difficult to meet the needs of efficient real-time monitoring of commercial display screens.

[0007] Therefore, how to improve the reliability of watermark information and its ability to prevent illegal copying and theft is a research topic with very positive and practical significance. Summary of the invention

[0008] In view of this, the purpose of the present invention is to propose a watermark adding method with strong adaptability, good anti-interference ability and the ability to provide better anti-counterfeiting, anti-illegal copying and anti-theft, and its application in the authentication of content played on commercial display screens.

[0009] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0010] A method for generating watermark information, comprising:

[0011] S01, acquiring image data according to preset conditions;

[0012] S02, identifying the core content area in the image data, and then using it as the image fingerprint feature area;

[0013] S03, dividing the image fingerprint feature area into N pieces of sub-images, and then extracting multi-dimensional feature information from the N pieces of sub-images according to preset conditions;

[0014] S04, calculating the comprehensive feature value of the sub-image according to the multi-dimensional feature information corresponding to the sub-image and combining with a preset calculation model;

[0015] S05. Generate a watermark sequence according to preset conditions by comparing the comprehensive feature values ​​between the N sub-images.

[0016] As a possible implementation, further, in the solution S01, the source of the image data is one or more of the following:

[0017] (1) Image data obtained by frame extraction from video data;

[0018] (2) Image data pre-stored in the electronic device;

[0019] (3) Image data obtained by electronic devices from the Internet;

[0020] (4) Image data obtained through data communication interaction between electronic devices.

[0021] As a possible implementation, further, in the present solution S02, identifying the core content area in the image data includes:

[0022] The image data is read to obtain edge information in the image data, and the black edges or white edges at the edges of the image are removed according to the edge information, and the remaining image area is used as the core content area.

[0023] As a possible implementation, further, in the present solution S03, the width and height specifications of the image fingerprint feature area are W×H, and it is divided into N sub-images in a horizontal or vertical manner, and the width and height specifications of each sub-image are or

[0024] As a preferred implementation option, preferably, the image data in this solution is obtained by frame extraction from video data;

[0025] In S03, the multi-dimensional feature information extracted from the N sub-images according to the preset conditions includes: average brightness Color histogram H i , frame pixel difference D i , temporal domain frame texture difference TD i .

[0026] As a preferred implementation option, preferably, the average brightness described in this solution is The calculation formula is defined as follows:

[0027]

[0028] The color histogram H i It is used to describe the statistical information of color distribution (R, G, B) in the image as the color feature of the segmented area. Its calculation formula is defined as follows:

[0029]

[0030] The frame pixel difference D i It is used to describe the pixel difference between two frames of images. It describes the image difference characteristics between videos during the video playback process of commercial display screens. The formula is defined as follows:

[0031]

[0032] The time domain frame texture difference TD i It is used to describe the image texture changes of image data between video frames. It captures the dynamic changes of the image on the time axis by comparing the texture features of adjacent frames, including the difference between the center pixel and the surrounding difference pixels. The formula is defined as follows:

[0033] Temporal difference of local binary patterns:

[0034]

[0035] Texture differences between adjacent frames:

[0036]

[0037] In S04, the formula for calculating the comprehensive feature value of the sub-image in combination with the preset calculation model is defined as follows:

[0038]

[0039] As a preferred implementation option, preferably, this solution S05 includes:

[0040] The comprehensive feature values ​​of N sub-images are numbered, and then the comprehensive feature values ​​between the N sub-images are compared, and a watermark sequence is generated according to preset conditions, which includes:

[0041]

[0042] Based on the above, the present invention further provides a watermark adding method, which includes the watermark information generating method described above, and further includes:

[0043] According to the watermark sequence, a dot matrix watermark is generated according to preset conditions, and the dot matrix watermark is attached to the image data to complete the watermark addition.

[0044] As a preferred implementation option, preferably, the solution generates a dot matrix watermark according to a watermark sequence and preset conditions, including:

[0045] Based on the watermark sequence being a 0, 1 sequence, the watermark color corresponding to the watermark sequence is determined according to the following formula:

[0046]

[0047] According to the watermark sequence and its corresponding watermark color, a watermark color block with M×N pixel size is generated, and then multiple watermark color blocks are combined into a dot matrix watermark, that is, a watermark image. The formula is defined as follows:

[0048]

[0049] Based on the above, the present invention also provides a method for authenticating content played on a commercial display screen, which includes the watermark adding method described above. The authentication method includes:

[0050] Obtain the content played on the commercial display screen, extract the image corresponding to the content played, set it as the image data to be identified, and then locate and extract the watermark from the image data to be identified to obtain the watermark to be identified;

[0051] According to the image data to be authenticated, the watermark image corresponding to it is generated in combination with the watermark adding method, and then the watermark image is compared and matched with the watermark to be authenticated, and the authentication result is generated according to the matching result.

[0052] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: the present solution generates a watermark image by time domain dot matrix calculation, which has significant advantages in processing speed, robustness and commercial display screen application. Compared with the prior art, the present invention has the following advantages:

[0053] (1) In terms of watermark generation, the advantage of time-domain lattice calculation in this scheme is its low computational complexity compared with frequency-domain calculation or deep learning matching methods. The frequency-domain calculation of traditional schemes requires Fourier transform, which not only increases the amount of calculation but also may introduce additional complexity. The deep learning matching of traditional schemes involves training and reasoning processes, which require high computing resources and time. Relatively speaking, the time-domain lattice calculation of this scheme directly operates on the pixels of the image, with faster calculation speed and stronger real-time performance, and can complete the embedding and extraction of watermarks more quickly.

[0054] (2) This scheme performs well in terms of robustness; it combines multiple image features, including brightness information, color histogram, frame difference, and temporal texture difference. The comprehensive use of this multi-dimensional feature ensures that the watermark remains stable and recognizable after the image is scaled, brightness adjusted, or otherwise processed. Compared with technologies that rely solely on a single feature, this method has stronger adaptability and anti-interference capabilities.

[0055] (3) In commercial display applications, this technology demonstrates unique advantages. Commercial displays are often used to play important advertisements and commercial content. Ensuring the security and integrity of the content is key. The embedded watermark contains the difference characteristics of consecutive frames. This technology can effectively prevent hackers from tampering with or replacing the displayed content, ensuring the authenticity of the played information. In addition, the dynamic characteristics of the watermark enable it to adapt to the playback of dynamic content, and provide anti-counterfeiting functions to prevent illegal copying and theft, thereby protecting commercial interests. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 It is a brief illustration of the scheme that adds a dot matrix watermark to the image data;

[0058] Figure 2 This is a brief illustration of the scheme generating a dot matrix watermark based on a watermark sequence. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0060] This embodiment provides a method for generating watermark information, which includes:

[0061] S01, acquiring image data according to preset conditions;

[0062] S02, identifying the core content area in the image data, and then using it as the image fingerprint feature area;

[0063] S03, dividing the image fingerprint feature area into N pieces of sub-images, and then extracting multi-dimensional feature information from the N pieces of sub-images according to preset conditions;

[0064] S04, calculating the comprehensive feature value of the sub-image according to the multi-dimensional feature information corresponding to the sub-image and combining with a preset calculation model;

[0065] S05. Generate a watermark sequence according to preset conditions by comparing the comprehensive feature values ​​between the N sub-images.

[0066] In solution S01 of this embodiment, the source of the image data is one or more of the following:

[0067] (1) Image data obtained by frame extraction from video data;

[0068] (2) Image data pre-stored in the electronic device;

[0069] (3) Image data obtained by electronic devices from the Internet;

[0070] (4) Image data obtained through data communication interaction between electronic devices.

[0071] In this embodiment, the image data obtained by extracting frames from video data is taken as an example. As a possible implementation, further, in this solution S02, identifying the core content area in the image data includes:

[0072] The image data is read to obtain edge information in the image data, and the black edges or white edges at the edges of the image are removed according to the edge information, and the remaining image area is used as the core content area.

[0073] In the image segmentation, in the present solution S03, the width and height specifications of the image fingerprint feature area are W×H, and it is divided into N sub-images by horizontal segmentation or vertical segmentation, and the width and height specifications of each sub-image are It can also be

[0074] As a preferred implementation option, preferably, the image data in this solution is obtained by frame extraction from video data;

[0075] In S03, the multi-dimensional feature information extracted from the N sub-images according to the preset conditions includes: average brightness Color histogram H i , frame pixel difference D i , temporal domain frame texture difference TD i .

[0076] In the specific calculation, as a better implementation choice, preferably, the average brightness described in this scheme is The calculation formula is defined as follows:

[0077]

[0078] N i Refers to the number of pixels in the ith block. If the size of the selected sub-image blocks is the same, that is, 3*3, then N i =9, L(x,y) refers to the brightness value of the pixel at coordinate (x,y) in the i-th block.

[0079] The color histogram H i It is used to describe the statistical information of color distribution (R, G, B) in the image as the color feature of the segmented area. Its calculation formula is defined as follows:

[0080]

[0081] N i Refers to the number of pixels in the i-th block, H(x,y) refers to the RGB color value of the pixel. For example, the histogram of the third color block (H(x,y)) is 3*3 in size, so the RGB value of each pixel from 1 to 9 will be obtained. Each RGB is normalized to an 8-bit color depth and counted to obtain the 3*3 histogram data.

[0082] Specifically, it can be expressed as: RGB1(100,200,50)RGB2(200,100,50)RGB3(100,50,20)RGB4(150,100,100)RGB5(200,50,100)RGB6(200,50,150)RGB7(0,50,100)RGB8(100,0,150)RGB9(50,0,150).

[0083] As an example, the red channel histogram is:

[0084] The red value 100 appears 3 times (RGB1, RGB3, RGB8)

[0085] The red value 200 appears 3 times (RGB2, RGB5, RGB6)

[0086] Red value 150 appears once (RGB4)

[0087] Red value 0 appears once (RGB7)

[0088] Red value 50 appears once (RGB9)

[0089] The green channel histogram is:

[0090] Green value 200 appears once (RGB1)

[0091] Green value 100 appears 2 times (RGB2, RGB4)

[0092] Green value 50 appears 4 times (RGB3, RGB5, RGB6, RGB7)

[0093] Green value 0 appears 2 times (RGB8, RGB9)

[0094] The blue channel histogram is:

[0095] Blue value 50 appears 2 times (RGB1, RGB2)

[0096] Blue value 20 appears once (RGB3)

[0097] Blue value 100 appears 2 times (RGB4, RGB7)

[0098] The blue value 150 appears 4 times (RGB6, RGB8, RGB9)

[0099] Then the result is normalized and used as the eigenvalue.

[0100] The frame pixel difference D iIt is used to describe the pixel difference between two frames of images. It describes the image difference characteristics between videos during the video playback process of commercial display screens. The formula is defined as follows:

[0101]

[0102] I t (x, y) refers to the pixel value at position (x, y) in the image frame at time t.

[0103] The time domain frame texture difference TD i It is used to describe the image texture changes of image data between video frames. It captures the dynamic changes of the image on the time axis by comparing the texture features of adjacent frames, including the difference between the center pixel and the surrounding difference pixels. The formula is defined as follows:

[0104] Temporal difference of local binary patterns:

[0105]

[0106] Among them, I t (p) is the gray value of pixel p, I t (c) is the gray value of the pixel at the center point c; for example, a 3*3 color block has 8 points around the center point, p1, p2...P8. If the gray value of the surrounding pixels is greater than that of the center point, it is recorded as 1, otherwise it is recorded as 0 (the function meaning of s), and then 2p performs a numerical weighting to obtain the texture lbp data; TD is the difference between this color block and the previous color block.

[0107] Texture differences between adjacent frames:

[0108]

[0109] In S04, the formula for calculating the comprehensive feature value of the sub-image in combination with the preset calculation model is defined as follows:

[0110]

[0111] α, β, γ, and δ are weight coefficients.

[0112] In this scheme, the comprehensive feature sequence combines the time domain characteristics of the image itself with the continuity characteristics during video playback. The average brightness can capture the brightness changes in the image area and is highly robust to the brightness changes of the entire image; the color histogram can capture the color features in the image and is very sensitive to the changes in color frequency in the image, and can effectively identify the color tampering in the image; the frame difference can capture the movement and changes between frames, and is highly sensitive to the changes in moving objects or video content, and can effectively identify attacks on video insertion; the time domain texture difference can capture the subtle texture changes in the image area, is sensitive to the dynamic details in the image, and increases the difficulty of attack and forgery.

[0113] In terms of watermark sequence generation, as a preferred implementation option, preferably, this solution S05 includes:

[0114] The comprehensive feature values ​​of N sub-images are numbered, and then the comprehensive feature values ​​between the N sub-images are compared, and a watermark sequence is generated according to preset conditions, which includes:

[0115]

[0116] Based on the above, this embodiment further provides a watermark adding method, which includes the watermark information generating method described above, and further includes:

[0117] According to the watermark sequence, a dot matrix watermark is generated according to preset conditions, and the dot matrix watermark is attached to the image data to complete the watermark addition.

[0118] Combination Figure 1 As shown, a dot matrix watermark is provided on one side of the image data to which the watermark is added.

[0119] As a preferred implementation option, preferably, the solution generates a dot matrix watermark according to a watermark sequence and preset conditions, including:

[0120] Based on the watermark sequence being a 0, 1 sequence, the watermark color corresponding to the watermark sequence is determined according to the following formula (e.g. Figure 2 shown):

[0121]

[0122] According to the watermark sequence and its corresponding watermark color, a watermark color block with M×N pixel size is generated, and then multiple watermark color blocks are combined into a dot matrix watermark, that is, a watermark image. The formula is defined as follows:

[0123]

[0124] Based on the above, this embodiment also provides a method for authenticating content played on a commercial display screen, which includes the watermark adding method described above. The authentication method includes:

[0125] Obtain the content played on the commercial display screen, extract the image corresponding to the content played, set it as the image data to be identified, and then locate and extract the watermark from the image data to be identified to obtain the watermark to be identified;

[0126] According to the image data to be authenticated, the watermark image corresponding to it is generated in combination with the watermark adding method, and then the watermark image is compared and matched with the watermark to be authenticated, and the authentication result is generated according to the matching result.

[0127] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for generating watermark information, characterized in that: It includes: S01, acquiring image data according to preset conditions; S02, identifying the core content area in the image data, and then using it as the image fingerprint feature area; S03, dividing the image fingerprint feature area into N pieces of sub-images, and then extracting multi-dimensional feature information from the N pieces of sub-images according to preset conditions; S04, calculating the comprehensive feature value of the sub-image according to the multi-dimensional feature information corresponding to the sub-image and combining with a preset calculation model; S05. Generate a watermark sequence according to preset conditions by comparing the comprehensive feature values ​​between the N sub-images.

2. The watermark information generating method according to claim 1, characterized in that: In S01, the source of the image data is one or more of the following: (1) Image data obtained by frame extraction from video data; (2) image data pre-stored in the electronic device; (3) Image data obtained by electronic devices from the Internet; (4) Image data obtained through data communication interaction between electronic devices.

3. The watermark information generating method according to claim 1, characterized in that: In S02, the core content area in the image data is identified including: The image data is read to obtain edge information in the image data, and the black edges or white edges at the edges of the image are removed according to the edge information, and the remaining image area is used as the core content area.

4. The watermark information generating method according to claim 1, characterized in that: In S03, the width and height specifications of the image fingerprint feature area are W×H, and it is divided into N sub-images in a horizontal or vertical manner, and the width and height specifications of each sub-image are or 5. The watermark information generating method according to any one of claims 2 to 4, characterized in that: The image data is obtained by frame extraction from video data; In S03, the multi-dimensional feature information extracted from the N sub-images according to the preset conditions includes: average brightness Color histogram H i , frame pixel difference D i , temporal domain frame texture difference TD i .

6. The watermark information generating method according to claim 5, characterized in that: The average brightness The calculation formula is defined as follows: Among them, N i is the number of pixels in the i-th sub-image in the image, the color histogram H i It is used to describe the statistical information of color distribution (R, G, B) in the image as the color feature of the segmented area. Its calculation formula is defined as follows: Among them, N i is the number of pixels in the i-th sub-image in the image, and H(x,y) is the color value of the pixel in RGB; The frame pixel difference D i It is used to describe the pixel difference between two frames of images. It describes the image difference characteristics between videos during the video playback process of commercial display screens. The formula is defined as follows: I t (x, y) is the pixel value at position (x, y) in the image frame at time t; The time domain frame texture difference TD i It is used to describe the image texture changes of image data between video frames. It captures the dynamic changes of the image on the time axis by comparing the texture features of adjacent frames, including the difference between the center pixel and the surrounding difference pixels. The formula is defined as follows: Temporal difference of local binary patterns: Among them, i t (p) is the gray value of pixel p, I t (c) is the gray value of the pixel at the center point c; Texture differences between adjacent frames: In S04, the formula for calculating the comprehensive feature value of the sub-image in combination with the preset calculation model is defined as follows: Among them, α, β, γ, and δ are weight coefficients.

7. The watermark information generating method according to claim 6, characterized in that: S05 includes: The comprehensive feature values ​​of N sub-images are numbered, and then the comprehensive feature values ​​between the N sub-images are compared, and a watermark sequence is generated according to preset conditions, which includes:

8. A watermark adding method, characterized in that: It includes the watermark information generation method according to any one of claims 1 to 7, and further includes: According to the watermark sequence, a dot matrix watermark is generated according to preset conditions, and the dot matrix watermark is attached to the image data to complete the watermark addition.

9. The watermark adding method according to claim 8, characterized in that: According to the watermark sequence, generating a dot matrix watermark according to preset conditions includes: Based on the watermark sequence being a 0, 1 sequence, the watermark color corresponding to the watermark sequence is determined according to the following formula: According to the watermark sequence and its corresponding watermark color, a watermark color block with M×N pixel size is generated, and then multiple watermark color blocks are combined into a dot matrix watermark, that is, a watermark image. The formula is defined as follows:

10. A method for identifying the authenticity of content played on a commercial display screen, characterized in that: It includes the watermark adding method according to claim 8 or 9, and the authentication method includes: Obtain the content played on the commercial display screen, extract the image corresponding to the content played, set it as the image data to be identified, and then locate and extract the watermark from the image data to be identified to obtain the watermark to be identified; According to the image data to be authenticated, the watermark image corresponding to it is generated in combination with the watermark adding method, and then the watermark image is compared and matched with the watermark to be authenticated, and the authentication result is generated according to the matching result.

Images