Video playback protection method and device based on digital watermark

Abstract

The present invention provides a video playback protection method and device based on digital watermarking, including: obtaining a video to be played, determining a video frame image based on the video to be played, obtaining a grayscale layer image corresponding to the video frame image; determining multiple video frame sub-images corresponding to each video frame image based on each grayscale layer image, extracting feature vectors of each video frame sub-image, determining a reassembly vector corresponding to each video frame image based on each feature vector, and determining a first version of a watermark image based on the reassembly vector; performing a discrete cosine transform on the grayscale layer image, extracting low-frequency component image features corresponding to the video frame image based on the discrete cosine transform result, determining a feature string based on the low-frequency component image features, obtaining a hash value based on the feature string and a custom string using a hash algorithm, and determining a second version of a watermark image based on the hash value; and embedding the first and second versions of the watermark image into the video frame image to obtain a video frame watermark image. This application improves the interception effect of tampered video images.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video playback protection method and device based on digital watermarking. Background Art

[0002] With the advancement of modern society and the development of electronic information, public electronic screens, as an information display tool, have become an essential component of various public areas, including shopping malls, commercial streets, hospitals, train stations, airports, schools, and scenic spots. Public electronic screens can efficiently and quickly disseminate public information, commercial advertisements, promotional videos, and other content, making them a crucial component of urban information dissemination. However, since public electronic screens primarily disseminate information to the public, the display of illegal content can easily cause adverse social impacts. In recent years, incidents involving mis-broadcasting on public electronic screens have become a common occurrence, attracting public attention and causing adverse effects.

[0003] There are two main reasons why public electronic screens play illegal images. One is that the broadcast control equipment is affected by external forces, such as viruses, remote control, malicious screen projection, etc. The other is that the operator makes mistakes and plays other content. The above two reasons all lead to the broadcast control equipment transmitting unplanned video content to the public electronic screen. In order to avoid this situation, the invention patent with publication number CN202311599031 discloses an online conference video anti-tampering system including a transmitter, an output device, an input device, and a receiver. The transmitter adds a watermark to the video for transmission by pre-agreed element information and combining the selected watermark form, image information, etc. The receiving end parses the watermark to determine whether the video has been tampered with, thereby protecting the video. Although the system reduces the risk of video tampering to a certain extent, the existing technology cannot effectively intercept tampered images during the video playback protection process, resulting in a certain risk of tampered video images being played. Therefore, how to improve the interception effect of tampered video images is a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a video playback protection method and apparatus based on digital watermarking to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present invention provides a video playback protection method based on digital watermarking, the method comprising:

[0006] Acquire a video to be played, determine a video frame image based on the video to be played, and acquire a grayscale layer image corresponding to the video frame image;

[0007] Determining a plurality of video frame sub-images corresponding to each of the video frame images based on each of the grayscale layer images, extracting a feature vector of each of the video frame sub-images, determining a recombined vector corresponding to each of the video frame images based on each of the feature vectors, and determining a first version of the watermark image based on the recombined vectors;

[0008] Performing a discrete cosine transform on the grayscale layer image, extracting low-frequency component image features corresponding to the video frame image based on the discrete cosine transform result, determining a feature string based on the low-frequency component image features, obtaining a hash value based on the feature string and a custom string using a hash algorithm, and determining a second version of the watermark image based on the hash value;

[0009] The first and second watermark images are embedded into the video frame image to obtain a video frame watermark image.

[0010] In some embodiments of the present invention, the method further comprises:

[0011] Storing a plurality of video frame watermark images within a preset time range into an image cache queue;

[0012] Extract all images in the image cache queue, determine whether all images in the image cache queue meet preset playback requirements, and if so, play the watermarked video frame watermarked image with the watermark removed.

[0013] In some embodiments of the present invention, determining whether all images in the image cache queue meet preset playback requirements includes:

[0014] Acquire a first watermark feature and a second watermark feature of an image in the image cache queue;

[0015] determining whether the image is a secure image based on the first watermark feature, the second watermark feature, the recombined vector, and the feature string;

[0016] When the number of safe images in the image cache queue reaches a preset reference value, it is obtained that all images in the image cache queue meet the preset playback requirement.

[0017] In some embodiments of the present invention, determining whether the image is a secure image based on the first watermark feature, the second watermark feature, the recombined vector, and the feature string includes:

[0018] Calculating a first similarity between the first watermark feature and the recombined vector, and a second similarity between the second watermark feature and the feature character string;

[0019] It is determined whether the image is a safe image based on the first similarity and the second similarity.

[0020] In some embodiments of the present invention, determining a video frame image based on the video to be played includes:

[0021] Obtaining image size data of the video to be played and playback size data of the playback device;

[0022] The video frame of the video to be played is deformed based on the image size data and the playback size data to obtain a video frame image.

[0023] In some embodiments of the present invention, determining the reconstructed vector corresponding to each of the video frame images based on each of the feature vectors includes:

[0024] Combining the eigenvectors to obtain a combined matrix;

[0025] Converting the stitching matrix into a stitching vector;

[0026] The concatenated vectors are recombined to obtain the recombined vectors.

[0027] In some embodiments of the present invention, determining a first version of the watermark image based on the recombined vector includes:

[0028] Determine the color corresponding to each element based on the element components in the recombined vector;

[0029] Determine the line width of the watermark image;

[0030] The recombined vector is mapped into a first version of the watermark image based on the line width of the watermark image and the color corresponding to each element.

[0031] In some embodiments of the present invention, the first watermark image and the second watermark image are located at the lower left corner and the lower right corner of the video frame image, respectively.

[0032] According to another aspect of the present invention, a video playback protection device based on digital watermark is also disclosed. The device includes a processor, a memory, and a computer program stored in the memory. The processor is used to execute the computer program. When the computer program is executed, the device implements the steps of the method described in any of the above embodiments.

[0033] According to yet another aspect of the present invention, a computer-readable storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0034] The digital watermark-based video playback protection method and device disclosed in the above-mentioned embodiments of the present invention determine the reconstructed vector corresponding to the video frame image based on the feature vector of the video frame sub-image, and then determine the first version of the watermark image based on the reconstructed vector. In addition, a second version of the watermark image is obtained by combining the discrete cosine transform method and the hash calculation method. The first and second versions of the watermark image are then embedded into the video frame image to obtain the video frame watermark image. The watermark processing method adopted by this method can effectively prevent the reverse imitation of the watermark, thereby improving the security of the watermark. Moreover, both versions of the watermark are calculated based on the image content, effectively preventing the image content from being tampered with, thereby improving the interception effect of tampered video images.

[0035] In addition, the first version of the watermark image and the second version of the watermark image in the embodiment of the present application are located in the lower left corner and the lower right corner of the video frame image respectively, and the size of the watermark in the image screen is relatively small, which improves the video screen quality.

[0036] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.

[0037] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are intended to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. The components in the drawings are not drawn to scale, but are merely for the purpose of illustrating the principles of the present invention. To facilitate the illustration and description of certain portions of the present invention, corresponding portions in the drawings may be exaggerated, that is, may be larger than other components in an exemplary device actually manufactured according to the present invention. In the drawings:

[0039] Figure 1 Schematic diagram of the process of a video playback protection method based on digital watermarking according to an embodiment of the present application.

[0040] Figure 2 A schematic diagram of the process of adding watermarks to videos according to an embodiment of the present application.

[0041] Figure 3a This is a first exemplary diagram of video frame deformation according to an embodiment of the present application.

[0042] Figure 3b This is a second exemplary diagram of video frame deformation according to an embodiment of the present application.

[0043] Figure 4 Schematic diagram of the image before and after adding the watermark.

[0044] Figure 5 This is a schematic diagram of the watermark detection process according to an embodiment of the present application.

[0045] Figure 6 A schematic diagram of the architecture of a video playback protection system based on digital watermarking according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0047] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0048] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0049] It should also be noted here that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection with an intermediary, and not only to a wired connection but also to a wireless connection, and the specific connection can be changed based on the actual application scenario.

[0050] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0051] Figure 1 This is a flow chart of a video playback protection method based on digital watermarking according to an embodiment of the present application. Figure 1 As shown, the method at least includes steps S10 to S40.

[0052] Step S10: obtaining a video to be played, determining a video frame image based on the video to be played, and obtaining a grayscale layer image corresponding to the video frame image.

[0053] This step is used to determine the video frame image to be played. In order to facilitate the generation of the first version of the watermark image and the second version of the watermark image in the subsequent steps, the video frame image is further scaled according to a certain ratio, and the grayscale layer image of the scaled video frame image is extracted to obtain the grayscale layer image corresponding to the video frame image.

[0054] In one embodiment, determining a video frame image based on the video to be played specifically includes: obtaining image size data of the video to be played and playback size data of a playback device; and deforming the video frame of the video to be played based on the image size data and the playback size data to obtain a video frame image. In this embodiment, original information is extracted from the original video to be played, image size data (such as image width and height size data) in the video is extracted, and playback size information is determined. The playback size information is such as the width and height size data of the output screen of the playback device. For example, when the playback device is a laptop computer and full-screen playback is performed, the playback size data is the screen resolution of the laptop computer.

[0055] When the image size data of the video to be played differs from the playback size data of the playback device, the video frame of the video to be played is further deformed to adapt the image size data of the video to the playback size data of the playback device. For example, the aspect ratio of the video image can be compared to the aspect ratio of the output screen of the playback device. If they are inconsistent, black borders of corresponding sizes are added to the upper and lower boundaries or the left and right boundaries of the video image to ensure that the aspect ratio of the video image after the black borders are added is consistent with the aspect ratio of the output screen of the playback device.

[0056] Figure 3a This is a first example diagram of video frame deformation according to an embodiment of the present application. Figure 3a In the illustrated embodiment, the aspect ratio of the video image is greater than that of the output screen. In this case, the actual height of the video image is calculated based on the width of the video image and the aspect ratio of the screen output by the playback device. The calculated actual height makes the aspect ratio of the video image consistent with the aspect ratio of the screen output by the playback device. The difference between the actual height of the video image and the original height is further calculated. Based on the calculated difference, a black rectangular image with a height of half the difference and a width equal to the width of the original image is added to the upper and lower boundaries of the original video image. In this way, the height of the video image is deformed to the actual height. The deformation effect in this scenario is shown in the figure below. Figure 3a As shown. Similarly, Figure 3b This is a second example diagram of video frame deformation according to an embodiment of the present application. Figure 3bIn the illustrated embodiment, the aspect ratio of the video image is smaller than that of the output screen. In this case, the actual width of the video image is calculated based on the height of the video image and the aspect ratio of the screen output by the playback device. The calculated actual width makes the aspect ratio of the video image consistent with the aspect ratio of the screen output by the playback device. The difference between the actual width of the video image and the original width is further calculated. Based on the calculated difference, a black rectangular image with a width of half the difference and a height equal to the height of the original image is added to the left and right borders of the original video image. In this way, the width of the video image is deformed to the actual width. The deformation effect in this scenario is shown in the figure below. Figure 3b It is understood that the video frame deformation method listed in this embodiment is only an example, and in other embodiments, other deformation methods can also be used to complete the deformation of the video frame.

[0057] Step S20: Determine multiple video frame sub-images corresponding to each of the video frame images based on each of the grayscale layer images, extract feature vectors of each of the video frame sub-images, determine a recombinant vector corresponding to each of the video frame images based on each of the feature vectors, and determine a first version of the watermark image based on the recombinant vector.

[0058] This step is to obtain the first version of the watermark image of the video frame image based on the watermark generation algorithm. For the sake of ease of description, the first version of the watermark image is defined as version A watermark. The grayscale layer image used in this step is the grayscale layer image determined in step S10. In this step, the grayscale layer image is divided into N*N video frame sub-images, and structural features are extracted for each video frame sub-image to obtain feature vectors of each of the video frame sub-images. Exemplarily, when extracting the feature vectors of the video frame sub-images, LBP structural features can be extracted for each video frame sub-image, and the extracted structural features are subjected to dimensionality reduction processing to obtain feature vectors of length D for each video frame sub-image. After obtaining the feature vectors of each video frame sub-image, the recombined vectors corresponding to each of the video frame images are further determined based on the feature vectors of each video frame sub-image.

[0059] In one embodiment, determining the recombined vector corresponding to each of the video frame images based on each of the feature vectors may specifically include: stitching the feature vectors together to obtain a stitched matrix; converting the stitched matrix into a stitched vector; and recombining the stitched vector to obtain a recombined vector. Exemplarily, after extracting the feature vectors of each video frame sub-image, the feature vectors of each video frame sub-image are sequentially merged to form an N*N*D matrix. The stitched matrix is further reduced in dimension to obtain a one-dimensional stitched vector, and the one-dimensional stitched vector is recombined according to a preset recombination method to obtain a final recombined vector. The recombination method includes, but is not limited to: segmenting and reordering the vectors, locally inverting the vectors, and permuting the positions of adjacent vectors.

[0060] Furthermore, determining the first version of the watermark image based on the recombined vector may specifically include: determining the color corresponding to each element based on the element components in the recombined vector; determining the watermark image line width; and mapping the recombined vector to the first version of the watermark image based on the watermark image line width and the color corresponding to each element. In this embodiment, the scores in the recombined vector are represented based on different colors, thereby converting the recombined vector into the version A watermark image. Exemplarily, the line width of the watermark image is set according to the size of the video frame image after the border is supplemented, and then the version A watermark image is generated based on the set watermark image line width; for example, for videos of various resolutions from 1K to 8K, the line width setting values can be: 3 for 1K video (1920x1080 pixels), 5 for 2K video (2560x1440 pixels), 5 for 4K video (3840x2160 pixels), and 10 for 8K video (7680x4320 pixels). In addition, the correspondence between components and colors can be constructed, and then the color corresponding to each element is selected based on the actual component value of each element in the recombined vector. Then, based on the line width of the watermark image, continuous pixels of the same color with a length equal to the line width are used to represent a component of the recombined vector. At the same time, pixels of the segmentation color are added to segment the recombined vector. Finally, the recombined vector is mapped to version A of the watermark image. Version A of the watermark image is as follows: Figure 4 shown.

[0061] Step S30: Perform discrete cosine transform on the grayscale layer image, extract low-frequency component image features corresponding to the video frame image based on the discrete cosine transform result, determine a feature string based on the low-frequency component image features, obtain a hash value based on the feature string and a custom string through a hash algorithm, and determine a second version of the watermark image based on the hash value.

[0062] This step is based on the watermark generation algorithm to obtain the second version of the watermark image for the video frame image. For ease of description, the second version of the watermark image is defined as the B version watermark. The grayscale layer image used in this step is the grayscale layer image determined in step S10. In this step, a discrete cosine transform (DCT transform) is performed on the grayscale layer image. The low-frequency component is extracted from the transform result as the low-frequency component image feature of the video frame image. The low-frequency component can specifically be the N*N-dimensional low-frequency component in the upper left corner of the grayscale layer image. The N*N-dimensional low-frequency component is converted into a one-dimensional vector as the low-frequency component image feature vector. The low-frequency component image feature vector is further converted into a feature string. After the feature string is combined with the custom string, a hash algorithm is used to calculate the hash value of the combined string. The hash value is mapped, that is, different colors are used to represent the characters in the hash value, thereby converting the hash value into the B version watermark image.

[0063] Exemplarily, the low-frequency component image feature vector can be graded according to the numerical value of each component, and the component value range can be [0, 255]. In this case, the value range can be divided into 4 levels, and the characters corresponding to each level are A, B, C, and D respectively. Each component is mapped to a corresponding character according to the range of its numerical value. It can be understood that the level and the characters corresponding to each level can be set based on actual needs. For example, in one embodiment, the level is divided into two levels, and the two-level characters are 0 and 1 respectively. Then, each component in the low-frequency component image feature vector is mapped to a 0 character or a 1 character, thereby obtaining a feature string corresponding to the low-frequency component image feature. The feature string is further combined with a custom specific string, and the hash value of the combined string is calculated using the SM3 algorithm (i.e., the National Commercial Cryptography 3 algorithm, hereinafter referred to as SM3). The hash value consists of letters and numbers. In order to convert the hash value into the B version watermark image, we can use a method of mapping multiple characters to one color, or one character to one color. Then, combined with the watermark line width, continuous pixels of the same color with a length equal to the line width are used to represent a character of the hash value. At the same time, pixels of the segmentation color are added to segment the hash value. Finally, the hash value is mapped to the B version watermark image. The B version watermark image is as follows: Figure 4 shown.

[0064] Step S40: embedding the first watermark image and the second watermark image into the video frame image to obtain a video frame watermark image.

[0065] This step is to embed the watermark image version A and the watermark image version B into the video frame image to complete the watermark addition. In the above steps, the watermark image version A and the watermark image version B are calculated using different watermark calculation methods, and both are dot matrices composed of pixels of different colors.

[0066] Furthermore, the first version of the watermark image and the second version of the watermark image are located at the lower left corner and the lower right corner of the video frame image respectively; in this embodiment, the area occupied by the watermark image is relatively small relative to the entire video screen, such as Figure 4 As can be seen in the figure, the watermark can only be clearly identified after the local area where the watermark is located is magnified 50 times. This method makes the watermark highly concealed and does not affect the clarity of the video image. In addition, after all video frames of the video to be played have been watermarked, the watermark images of all video frames are further merged to complete the production of the watermarked video.

[0067] In a specific embodiment, if Figure 2As shown, when adding a video watermark, the video width and height information is first extracted based on the original video. The width and height information of the playback device output screen is also obtained, and the video frames are extracted. The boundaries of each frame are supplemented to generate a new image. The following methods are used to calculate the A-version watermark and the B-version watermark respectively. The A-version watermark calculation method is as follows: 1. Segment the image, 2. Calculate the sub-image structural features, 3. Merge the sub-image features, 4. Reorganize the features, and 5. Convert the features into the A-version watermark. The B-version watermark calculation method is as follows: 1. Perform a DCT transform on the image, 2. Extract the low-frequency components of the image, 3. Combine the features into a string, 4. Use SM3 to calculate the hash value of the feature string, and 4. Convert the hash value into the B-version watermark. The A-version watermark and the B-version watermark are then added to the lower left and lower right corners of the image, respectively, to complete the image watermarking. Finally, the multiple video frames with watermarks are merged to generate the watermarked video.

[0068] In some embodiments of the present invention, the digital watermark-based video playback protection method further includes the following steps: storing multiple video frame watermark images within a preset duration in an image cache queue; extracting all images from the image cache queue, determining whether all images in the image cache queue meet preset playback requirements, and if so, playing the video frame watermark images after the watermarks have been removed. This embodiment detects whether the video to be played stored in the image cache queue meets the playback requirements. If so, the video is played; if not, the video is blocked to prevent playback. The above steps can be implemented based on a display content protection device. Specifically, after the display content protection device is activated, it accesses the video frame watermark image data through a video access port and stores the video frame watermark image in an image cache queue, denoted as img_queue. In one embodiment, the number of images in the image cache queue is a preset fixed value, typically storing images of at least two seconds. Taking a conventional video stream with a 60Hz refresh rate as an example, the video stream inputs images at a rate of 60 frames per second. To store images of at least two seconds, the image cache queue must store at least 120 images.

[0069] Furthermore, determining whether all images in the image cache queue meet preset playback requirements includes: obtaining first and second watermark features of the images in the image cache queue; determining whether the images are safe images based on the first and second watermark features, the reconstructed vector, and the characteristic string; and determining that all images in the image cache queue meet the preset playback requirements when the number of safe images in the image cache queue reaches a preset reference value. This step can be performed by the display content protection device, that is, the display content protection device detects watermarks on the input video stream.

[0070] Specifically, when performing watermark detection, first, the input image (the input image refers to the image in the cache queue, which may have been tampered with) is subjected to cropping of a certain size from the lower left and right corners of the image. The watermarks in the cropped image fragments are the first and second versions of the watermark image added in step S40. Based on the correspondence between the vector components, eigenvalues and colors determined in the watermark adding step, the watermarks in the image fragments are reverse-mapped into recombined vectors and feature strings. Furthermore, the first and second watermark features are recalculated based on the image using a watermark generation algorithm. Finally, based on the calculated first and second watermark features, and the recombined vectors and feature strings of the watermark image corresponding to the image fragment, it is determined whether the image is a secure image. The image fragment that needs to be watermarked contains version A and version B watermarks, which are a set of fixed color combinations. When performing watermark recognition on the image fragment, pixels with similar colors need to be recognized as one color to offset the noise generated during video compression and playback. The first watermark feature and the second watermark feature obtained by recalculating the image based on the watermark calculation method can be denoted as A1 and B1.

[0071] In addition, the reconstructed vector and characteristic string corresponding to the image segment can be denoted as A0 and B0, respectively. A first similarity between the first watermark feature A1 and the reconstructed vector A0, and a second similarity between the second watermark feature B1 and the characteristic string B0 are further calculated. Based on the first and second similarities, whether the image is a secure image is determined. The similarity between the two strings A0 and A1 can be calculated using the Hamming distance algorithm. However, the SM3 algorithm is used in the calculation of version B watermark. Therefore, the similarity between strings B0 and B1 can only be 0 or 1. When string B0 is equal to string B1, the similarity is 1; otherwise, the similarity is 0. Furthermore, a self-check is performed on the current image based on the first similarity value and the second similarity value. When the similarity meets the set similarity reference value, the play mark of the current image is set to 0 (indicating that the play requirement is met), otherwise it is set to 1 (indicating that the play requirement is not met); illustratively, when the first similarity is greater than 95% and the second similarity is 0, the set similarity threshold is met. At this time, the image and the play mark are cached in a mark list, recorded as play_list.

[0072] In addition, when the number of images in the image cache queue img_queue reaches a set threshold, the play marks in play_list are analyzed. If all play marks in play_list are 0, that is, all checked images meet the watermark self-check, then the first image is taken from the image cache queue img_queue and prepared for playback; if the play marks in play_list are not all 0, it means that there are "illegal" images in the cached images that do not meet the watermark self-check. At this time, the image to be played is replaced with a preset default safe image to prevent the image in the video stream from being spread. For example, play_list can be analyzed based on the summation function sum; when sum(play_list) is equal to 0, it means that all play marks in play_list are 0, that is, all checked images meet the watermark self-check and no watermark "illegal" images are found; if sum(play_list) is not equal to 0, it means that there are "illegal" images in the cached images that do not meet the watermark self-check. In some other embodiments, psize can also be used to represent the number of play marks in play_list. In this embodiment, the fault tolerance parameter error_threshold is also set to 0.1. When sum(play_list)<=psize*error_threshold, it means that there are a small number of images in play_list whose watermark detection does not meet the playback requirements, but the video stream is considered not to be "illegal" at this time, and the first image is still taken out from the image cache queue img_queue for playback; and when sum(play_list)>psize*error_threshold, it means that a large number of "illegal" images that do not meet the watermark detection are found, and video interception is started. At this time, the preset default safe image is used for playback, and the first image is taken out from the image cache queue img_queue, and a deletion operation is performed, so as to keep the number of images in the image cache queue no higher than the threshold; finally, the play mark stored earliest in play_list is deleted, so that the number of play marks in play_list remains stable.

[0073] During the specific playback process, the video frame watermark image taken from the image cache queue img_queue can also be subjected to watermark information erasure processing. In this embodiment, regardless of whether the image to be played is a video stream image containing a watermark or a preset security image, a unified watermark erasure operation can be performed. The size of the watermark in this application is relatively small. For example, the height of the watermark added to the image is only 3-5 pixels; Figure 4 As shown, the watermark occupies a very small area in the entire image, so the watermark information can be erased by cropping, covering, local deformation, etc., so that the video after the watermark information is erased can be played through the display device.

[0074] Figure 5 This is a schematic diagram of the watermark detection process of the video playback protection method based on digital watermark according to an embodiment of the present application. Figure 5 In the specific embodiment shown, the video frame watermark image is stored in a cache queue, and then the image watermark detection step is performed, that is, the watermark mapping values A0 and B0 of the watermark in the image segment are extracted, and the watermark feature values A1 and B1 of the image are recalculated based on the watermark generation algorithm; further, based on the similarity of the two sets of feature values A0-A1 and B0-B1, the image play mark is calculated and stored in a play mark list; then, based on the play mark list, it is determined whether playback is allowed. If playback is allowed, the earliest stored image is taken out from the image cache queue and prepared for playback. If playback is not allowed, the default image is used and prepared for playback; finally, the relevant watermark information at the location of the watermark in the output image is erased.

[0075] Correspondingly, the present invention also provides a video playback protection device based on digital watermarks, which includes a processor, a memory, and a computer program stored in the memory. The processor is used to execute the computer program. When the computer program is executed, the device implements the steps of the method described in any of the above embodiments.

[0076] Exemplarily, a system corresponding to a digital watermark-based video playback protection method may include a video processing device, a display content protection device, and a playback device. The video processing device performs edge padding on each frame of the video and then calculates eigenvalues based on the image's structure and color. The SM3 algorithm is used in the eigenvalue calculation process to improve watermark security. The eigenvalues are then encoded as a watermark and embedded into the image, completing the production of the watermarked video. The display content protection device performs watermark recognition on the received image, extracts the eigenvalues of the watermark within the image, and compares the similarity of the eigenvalues to determine whether the video has been tampered with. Furthermore, the display content protection device uses a built-in image cache queue to delay image playback. If tampering is detected, the device intercepts the video in advance, preventing the tampered content from being disseminated to the public via display devices (such as public electronic screens), thereby protecting the display content. Furthermore, the watermark information on untampered videos is removed before being disseminated to the public via public electronic screens, thereby eliminating the watermark's impact on the video. This system offers advantages such as good compatibility, high security, strong protection, and fast interception speed.

[0077] exist Figure 6In the specific embodiment shown, the video playback protection system based on digital watermarking includes a video processing device, a video content broadcast control device, a display content protection device, and a public electronic screen. The video processing device in this embodiment acts as a front-end device on the video content broadcast control device, and is used to perform watermark processing on the commercial advertisements, promotional videos and other contents to be played, and then perform timed playback, loop playback, etc. through the original video content broadcast control device. The display content protection device acts on the public electronic screen and is connected in series between the video content broadcast control device and the public electronic screen, serving as the last barrier for information protection, preventing tampered content from being disseminated to the public via the public electronic screen. Based on the above content, it can be seen that, compared with the existing public information system consisting only of a broadcast control device and a public electronic screen, the video playback protection system of the present application can realize video watermarking based on the video processing device and watermark self-checking based on the display content protection device, so that the system can effectively intercept tampered images.

[0078] exist Figure 6 In the system shown, the video processing device runs a watermark generation method to perform watermark processing on the video content to be played, that is, to complete the addition of a watermark to the video to be played; the display content protection device executes a watermark recognition and detection method to perform watermark detection on the incoming video stream, and can perform pixel-level control on the image output to the public electronic screen. When the watermark detection of the input image is invalid, the display content protection device intercepts the image and outputs a preset image to the public electronic screen; when the watermark detection of the input image is valid, the display content protection device erases the watermark information from the image and sends it to the public electronic screen through the output port for playback. At this time, the public cannot see any watermark information through the public electronic screen, thereby achieving the watermark-free playback of the video. Specifically, the display content protection device can receive video stream data through the video access port and output the video image through the video output port; the display content protection device can store a certain number of video images and can freely control the video image to be played.

[0079] The present invention discloses a video playback protection system based on digital watermarks. The video processing device and the display content protection device operate independently and do not require network communication. After the display content protection device is initialized, all network services are shut down and the device operates offline throughout the process, physically eliminating the network security risks of the display content protection device. Secondly, the watermark produced includes two versions, A and B. The version B watermark is based on the SM3 algorithm and effectively prevents reverse imitation of the watermark, thereby improving security. In addition, both versions A and B are calculated based on the image content. When any additional elements appear in the image, such as the local insertion of slogans, pictures, etc., the watermark detection result will be invalid and thus intercepted by the system, thereby effectively preventing the image content from being tampered with. In addition, the display content protection device caches images in memory to achieve delayed playback of the video stream. The display content protection device checks and inspects the video stream input image in advance. When an "illegal" image that does not meet the watermark self-check is found, it is intercepted in advance and no "illegal" image frame is disseminated to the public through public electronic screens. Therefore, it has the advantage of strong protection and has a good interception effect on tampered images.

[0080] In addition to the above, the display content protection device detects video stream images at an efficient detection speed of 20 times per second, and can quickly intercept "illegal" videos that do not meet the watermark detection requirements within 1 second. In addition, the watermark area added to each frame of video image is extremely small. Taking conventional 2K high-definition video (resolution of 2560x 1440) as an example, the watermark area is only 0.045% of the image area, less than 0.05% of the image area, effectively preventing the watermark from contaminating the original video content, thereby improving the video image quality. At the same time, the display content protection device uses its own pixel-level image control to erase the watermark on the output image before outputting it to the public electronic screen for playback. The public will not see any watermark information, achieving imperceptible playback of watermarked videos.

[0081] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0082] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0083] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0084] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0085] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A video playback protection method based on digital watermark, characterized in that: The method comprises: Acquire a video to be played, determine a video frame image based on the video to be played, and acquire a grayscale layer image corresponding to the video frame image; Determining a plurality of video frame sub-images corresponding to each of the video frame images based on each of the grayscale layer images, extracting a feature vector of each of the video frame sub-images, determining a recombined vector corresponding to each of the video frame images based on each of the feature vectors, and determining a first version of the watermark image based on the recombined vectors; Performing a discrete cosine transform on the grayscale layer image, extracting low-frequency component image features corresponding to the video frame image based on the discrete cosine transform result, determining a feature string based on the low-frequency component image features, obtaining a hash value based on the feature string and a custom string using a hash algorithm, and determining a second version of the watermark image based on the hash value; The first and second watermark images are embedded into the video frame image to obtain a video frame watermark image.

2. The video playback protection method based on digital watermark according to claim 1 is characterized in that: The method further comprises: Storing a plurality of video frame watermark images within a preset time range into an image cache queue; Extract all images in the image cache queue, determine whether all images in the image cache queue meet preset playback requirements, and if so, play the watermarked video frame watermarked image with the watermark removed.

3. The video playback protection method based on digital watermark according to claim 2 is characterized in that: Determine whether all images in the image cache queue meet the preset playback requirements, including: Acquire a first watermark feature and a second watermark feature of an image in the image cache queue; determining whether the image is a secure image based on the first watermark feature, the second watermark feature, the recombined vector, and the feature string; When the number of safe images in the image cache queue reaches a preset reference value, it is obtained that all images in the image cache queue meet the preset playback requirement.

4. The video playback protection method based on digital watermark according to claim 3 is characterized in that: Determining whether the image is a secure image based on the first watermark feature, the second watermark feature, the recombined vector, and the feature string includes: Calculating a first similarity between the first watermark feature and the recombined vector, and a second similarity between the second watermark feature and the feature character string; It is determined whether the image is a safe image based on the first similarity and the second similarity.

5. The video playback protection method based on digital watermark according to claim 1 is characterized in that: Determining a video frame image based on the video to be played includes: Obtaining image size data of the video to be played and playback size data of the playback device; The video frame of the video to be played is deformed based on the image size data and the playback size data to obtain a video frame image.

6. The video playback protection method based on digital watermark according to claim 1 is characterized in that: Determining a reconstructed vector corresponding to each of the video frame images based on each of the feature vectors includes: Combining the eigenvectors to obtain a combined matrix; Converting the stitching matrix into a stitching vector; The concatenated vectors are recombined to obtain the recombined vectors.

7. The video playback protection method based on digital watermark according to claim 6 is characterized in that: Determining a first version of the watermark image based on the recombined vector includes: Determine the color corresponding to each element based on the element components in the recombined vector; Determine the line width of the watermark image; The recombined vector is mapped into a first version of the watermark image based on the line width of the watermark image and the color corresponding to each element.

8. The video playback protection method based on digital watermark according to any one of claims 1 to 7, characterized in that: The first version of the watermark image and the second version of the watermark image are respectively located at the lower left corner and the lower right corner of the video frame image.

9. A video playback protection device based on digital watermark, comprising a processor, a memory, and a computer program stored in the memory, characterized in that: The processor is configured to execute the computer program. When the computer program is executed, the device implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images