Video watermark extraction method and device based on quick response code, equipment and medium

Abstract

The invention discloses a video watermark extraction method and device based on a quick response code, equipment and a medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first scene change frame and a quick response code matrix of a first video, and carrying out the processing of the first scene change frame and the quick response code matrix, and obtaining an encrypted quick response code matrix and a plurality of size image blocks; performing discrete cosine transform on the brightness component of the first scene change frame, and combining the encrypted quick response code matrix to obtain a DC coefficient and an AC coefficient; inverse discrete cosine transform is carried out on the image blocks with the multiple sizes, embedding of watermark information is achieved, and a first video with a watermark is obtained; processing the first video with the watermark to obtain a target estimation value of the encrypted quick response code matrix; and decrypting and decoding the target estimation value in sequence to obtain target watermark information, thereby realizing extraction of the watermark information. According to the method, the size of the embedded watermark can be dynamically adjusted, the extraction efficiency is improved, and the safety of the watermark can also be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular, to a method, device, equipment and medium for extracting video watermark based on Quick Response Code. Background Art

[0002] With the development of the Internet, the dissemination of digital media content has become more and more convenient, and the rights of authors have been severely damaged. However, as an algorithm for embedding specific copyright information into digital media content, the digital watermark algorithm can effectively protect the copyright of digital media content. Therefore, the research on the digital watermark algorithm is of great significance to better protect the rights of authors. In the existing technology, according to the different embedding positions, video watermark algorithms can be divided into compressed-domain video watermark algorithms and original-domain video watermark algorithms. The compressed-domain video watermark algorithm refers to an algorithm for embedding and extracting watermarks in a compressed video stream or a partially decoded compressed video, and the original-domain video watermark algorithm refers to an algorithm for directly embedding and extracting watermarks in uncompressed video data.

[0003] Compared with the compressed-domain video watermark algorithm, the advantage of the original-domain video watermark algorithm is that it is not affected by the video format. Generally, a fixed size is used to embed the watermark. However, in the smooth area of the video frame, since the pixel values change little, embedding the watermark with a fixed size will cause many blocks to contain a lot of similar information, resulting in information redundancy, increasing the computational amount, and leading to a very low extraction efficiency of the watermark. Summary of the Invention

[0004] The present application provides a method, device, equipment and medium for extracting video watermark based on Quick Response Code, which can dynamically adjust the size of the embedded watermark, does not need to process the video frame according to a fixed size, reduces the computational amount, improves the extraction efficiency, and also adds encryption processing to improve the security of the watermark.

[0005] To achieve the above object, the present application adopts the following technical solutions: In a first aspect, the present application provides a method for extracting video watermark based on Quick Response Code, the method comprising: Obtaining a first video and a Quick Response Code matrix, and selecting a first scene change frame in the first video; Processing the Quick Response Code matrix to obtain an encrypted Quick Response Code matrix; Performing size block processing on the first scene change frame in the first video to obtain a plurality of size image blocks; Performing discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain a first DC coefficient and a first low-frequency AC coefficient; Process the first DC coefficient, the first low-frequency AC coefficient, and the encrypted QR code matrix to obtain a second DC coefficient and a second low-frequency AC coefficient; Perform inverse discrete cosine transform on multiple sized image blocks according to the second DC coefficient and the second low-frequency AC coefficient to embed watermark information, and obtain a first video with watermark; Process the first video with watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain an estimated value of the target of the encrypted QR code matrix; Decrypt the estimated value of the target of the encrypted QR code matrix to obtain a target QR code matrix, and decode the target QR code matrix to obtain target watermark information, thereby realizing the extraction of watermark information.

[0006] In some possible implementation manners, the step of performing size block processing on the first scene change frame in the first video to obtain multiple sized image blocks includes: Calculate the visual saliency value of the pixel points of the first scene change frame according to the first scene change frame in the first video, and perform size block processing on the first scene change frame in the first video according to the visual saliency value to obtain multiple sized image blocks.

[0007] In some possible implementation manners, the step of processing the first video with watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain an estimated value of the target of the encrypted QR code matrix includes: Perform reduction processing on the second DC coefficient and the second low-frequency AC coefficient to obtain a first partial estimated value and a second partial estimated value of the encrypted QR code matrix, and perform processing according to the first partial estimated value and the second partial estimated value of the encrypted QR code matrix to obtain an estimated value of the target of the encrypted QR code matrix.

[0008] In some possible implementation manners, the step of processing the QR code matrix to obtain an encrypted QR code matrix includes: Generate a dynamic noise matrix for the QR code matrix by using a Bernoulli distribution, and process the QR code matrix, the dynamic noise matrix, and a time decay factor to obtain an encrypted QR code matrix.

[0009] In some possible implementation manners, the method further includes: Encrypt the dynamic noise matrix according to the dynamic noise matrix and the time decay factor to obtain an encrypted noise parameter; During the encryption process, a random private key is generated, and the security of the random private key is verified. A hash value is obtained based on the random private key, and the hash value is compared with a preset range of secure hash values to obtain a first judgment result. If the first judgment result indicates that the hash value is within the preset range of secure hash values, it is confirmed that the random private key is secure and available, and the encrypted noise parameter is decrypted according to the random private key, and then the watermark information is extracted.

[0010] In some possible implementation manners, the method further includes: If the first judgment result indicates that the hash value is not within the preset range of secure hash values, a new random private key is regenerated and rejudged until the new random private key is secure and available.

[0011] In a second aspect, the present application provides a video watermark extraction device based on a quick response code. The device includes: An acquisition module, configured to acquire a first video and a quick response code matrix, and select a first scene change frame in the first video; process the quick response code matrix to obtain an encrypted quick response code matrix; An embedding module, configured to perform size block processing on the first scene change frame in the first video to obtain a plurality of size image blocks; perform discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain a first direct current coefficient and a first low-frequency alternating current coefficient; process the first direct current coefficient, the first low-frequency alternating current coefficient, and the encrypted quick response code matrix to obtain a second direct current coefficient and a second low-frequency alternating current coefficient; perform inverse discrete cosine transform on the plurality of size image blocks according to the second direct current coefficient and the second low-frequency alternating current coefficient to implement the embedding of watermark information, and obtain a first video with a watermark. An extraction module, configured to process the first video with a watermark according to the second direct current coefficient and the second low-frequency alternating current coefficient to obtain an estimated value of the target encrypted quick response code matrix; decrypt the estimated value of the target encrypted quick response code matrix to obtain a target quick response code matrix, and decode the target quick response code matrix to obtain target watermark information, thereby implementing the extraction of watermark information.

[0012] In a third aspect, the present application provides a computing device, including a memory and a processor; Wherein, one or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device is caused to execute the method according to any one of the first aspect.

[0013] Fourthly, the present application provides a computer-readable storage medium for storing a computer program for executing the method according to any one of the first aspect.

[0014] Fifthly, the present application provides a computer program product including one or more computer instructions, which, when executed by a computer, cause the computer to execute the method according to any one of the first aspect.

[0015] It can be seen from the above technical solutions that the present application has at least the following beneficial effects: In the present application, a first video and a quick response code matrix are obtained, and a first scene change frame in the first video is selected; the quick response code matrix is processed to obtain an encrypted quick response code matrix; the first scene change frame in the first video is subjected to size block processing to obtain a plurality of size image blocks, which can dynamically adjust the size according to the first scene change frame of the first video, and then perform block processing to reduce the calculation amount caused by a fixed size and improve the accuracy during subsequent watermark information extraction; then, discrete cosine transform is performed on the luminance component of the first scene change frame in the first video to obtain a first DC coefficient and a first low-frequency AC coefficient; the first DC coefficient, the first low-frequency AC coefficient, and the encrypted quick response code matrix are processed to obtain a second DC coefficient and a second low-frequency AC coefficient; inverse discrete cosine transform is performed on the plurality of size image blocks according to the second DC coefficient and the second low-frequency AC coefficient to embed watermark information, and a first video with a watermark is obtained; the first video with a watermark is processed according to the second DC coefficient and the second low-frequency AC coefficient to obtain an estimated value of the encrypted quick response code matrix; the estimated value of the encrypted quick response code matrix is decrypted to obtain a target quick response code matrix, and the target quick response code matrix is decoded to obtain target watermark information, thereby realizing the extraction of watermark information.

[0016] In the traditional solution, video watermarking algorithms can be divided into compressed-domain video watermarking algorithms and original-domain video watermarking algorithms. Among them, the compressed-domain video watermarking algorithm has a faster extraction efficiency. However, since the video needs to be compressed, it is affected by the format and can only process the watermark in the video using a fixed format. The original-domain video watermarking algorithm is not affected by the format. The original-domain video watermarking algorithm generally embeds the watermark using a fixed size. However, in the smooth area of the video frame, since the pixel values change little, embedding the watermark using a fixed size will cause many blocks to contain a large amount of similar information, resulting in information redundancy, increasing the calculation amount, and leading to a very low extraction efficiency of the watermark. It can be seen that by providing a method for extracting video watermark based on quick response code, the present application can dynamically adjust the size of the embedded watermark, does not need to process all video frames according to a fixed size, reduces the calculation amount, improves the extraction efficiency, and also adds encryption processing to improve the security of the watermark.

[0017] It should be understood that the description of technical features, technical solutions, beneficial effects or similar languages in the present application does not imply that all features and advantages can be achieved in any single embodiment. On the contrary, it can be understood that the description of features or beneficial effects means that at least one embodiment includes specific technical features, technical solutions or beneficial effects. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in this embodiment can be combined in any appropriate manner. Those skilled in the art will understand that an embodiment can be implemented without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a method for extracting video watermark based on quick response code provided by an embodiment of the present application; Figure 2 It is a schematic diagram of a device for extracting video watermark based on quick response code provided by an embodiment of the present application; Figure 3 It is a schematic diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The terms "first", "second", "third", etc. in the specification and drawings of the present application are used to distinguish different objects, rather than to limit a specific order.

[0020] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0021] For the sake of clear and concise description of the following embodiments, a brief introduction to the related technologies is given first: The Quick Response Code (QRcode) is distributed in a two-dimensional plane direction according to a specific geometric pattern regularly, and consists of black and white patterns composed of dark and light modules. These modules are arranged and combined to record data symbol information, and can store various types of data. The Quick Response Code can be divided into 40 versions according to its size. The size of Version 1 is 21 × 21, and the size of Version 40 is 177 × 177. Each higher version has 4 additional modules on each side. According to the error correction ability, QRCode is divided into Level L, Level M, Level Q, and Level H. The approximate correction amounts for each level are shown in Table 1: Table 1:

[0022] The Discrete Cosine Transform (DCT) is a mathematical transformation method widely used in the fields of digital signal processing and image processing. It is a tool for transforming a set of discrete data points from the spatial domain to the frequency domain. In the field of images, an image can be regarded as the spatial distribution information composed of a large number of pixel points, and each pixel point has its specific brightness or color value. The discrete cosine transform can transform the spatial information represented by these pixel values into the combined information of different frequency components. It can also decompose an image into the superposition of cosine waves of different frequencies. Among them, the low-frequency part represents the basic information such as the main outline, general shape, and brightness of the image, just like the basic framework and main color tone of a painting; the high-frequency part corresponds to the details, textures, edges, etc. of the image, such as the veins of leaves and the fine edges of objects.

[0023] At present, video watermarking algorithms based on the discrete cosine transform can be divided into compressed-domain video watermarking algorithms and original-domain video watermarking algorithms. Among them, the compressed-domain video watermarking algorithm has a faster extraction efficiency. However, since the video needs to be compressed, it is affected by the format and can only process the watermark in the video using a fixed format. The original-domain video watermarking algorithm is not affected by the format. The original-domain video watermarking algorithm generally embeds the watermark using a fixed size. However, in the smooth area of the video frame, since the pixel values change little, embedding the watermark using a fixed size will cause many blocks to contain a lot of similar information, resulting in information redundancy, increasing the computational complexity, and leading to a very low extraction efficiency of the watermark.

[0024] In view of this, the embodiments of the present application provide a method for extracting video watermark based on a quick response code. In this method, a first video and a quick response code matrix are obtained, and a first scene change frame in the first video is selected; the quick response code matrix is processed to obtain an encrypted quick response code matrix; the first scene change frame in the first video is processed by size block division to obtain a plurality of size image blocks. The size can be dynamically adjusted according to the first scene change frame of the first video, and then block division is performed to reduce the computational complexity caused by a fixed size and improve the accuracy during subsequent watermark information extraction; then, discrete cosine transform is performed according to the luminance component of the first scene change frame in the first video to obtain a first direct current coefficient and a first low-frequency alternating current coefficient; the first direct current coefficient, the first low-frequency alternating current coefficient, and the encrypted quick response code matrix are processed to obtain a second direct current coefficient and a second low-frequency alternating current coefficient; inverse discrete cosine transform is performed on the plurality of size image blocks according to the second direct current coefficient and the second low-frequency alternating current coefficient to embed the watermark information, and a first video with a watermark is obtained; the first video with a watermark is processed according to the second direct current coefficient and the second low-frequency alternating current coefficient to obtain an estimated value of the target encrypted quick response code matrix; the estimated value of the target encrypted quick response code matrix is decrypted to obtain a target quick response code matrix, and the target quick response code matrix is decoded to obtain target watermark information, thus realizing the extraction of the watermark information.

[0025] It can be seen that by providing a method for extracting video watermark based on a quick response code, the present application can dynamically adjust the size of the embedded watermark, does not need to process the video frame according to a fixed size, reduces the computational complexity, improves the extraction efficiency, and also adds encryption processing to improve the security of the watermark.

[0026] To make the technical solution of the present application clearer and easier to understand, the following will introduce a method for extracting video watermark based on a quick response code provided by the embodiments of the present application with reference to the accompanying drawings. As Figure 1 shown, this figure is a flowchart of a method for extracting video watermark based on a quick response code provided by the embodiments of the present application. The method for extracting video watermark based on a quick response code includes: S101. Obtain a first video and a quick response code matrix, and select a first scene change frame in the first video.

[0027] In this application, a sample video is selected from a sample library and set as the first video for embedding and extracting watermarks. Before embedding the watermark in the first video, it is necessary to select the first scene change frame of the first video. Since a video is composed of a series of consecutive frames, scene change detection is crucial for accurate watermark embedding. Here, frames at different moments in the first video are selected. By calculating the correlation coefficient between the histogram of the Y component of the frame at the m-th moment and the histogram of the Y component of the frame at the (m - 1)-th moment, it is determined whether the scene has changed. Among them, in the YUV color space, the Y component represents luminance information and has a greater impact on human visual perception. Y component: Represents luminance and describes the brightness of the image. The Y component values in different regions of the image reflect whether the region is bright or dark; U component and V component: Represent chrominance and are used to describe the color information of the image. The U component and the V component together determine the color attributes such as hue and saturation of the image.

[0028] There are many advantages to using the YUV color space. On the one hand, the human eye is much more sensitive to luminance than to chrominance. Separating the luminance alone allows for more refined processing of luminance information during image processing and video coding, while appropriately simplifying the chrominance information helps improve coding efficiency, save storage space, and transmission bandwidth. On the other hand, in many video analysis tasks, such as scene change detection and object recognition, analyzing the Y component first can quickly obtain the basic light and dark features of the image, laying a foundation for subsequent more complex analysis.

[0029] The correlation coefficient between the histogram of the Y component of the frame at the m-th moment and the histogram of the Y component of the frame at the (m - 1)-th moment is calculated from the covariance of the histogram of the Y component of the frame at the m-th moment combined with the histogram of the Y component of the frame at the (m - 1)-th moment and the variance of the histogram of the Y component of the frame at the m-th moment combined with the histogram of the Y component of the frame at the (m - 1)-th moment. The covariance of the histogram of the Y component of the frame at the m-th moment combined with the histogram of the Y component of the frame at the (m - 1)-th moment measures the similarity degree of the Y component histograms of adjacent two frames. The variance of the histogram of the Y component of the frame at the m-th moment combined with the histogram of the Y component of the frame at the (m - 1)-th moment and the correlation coefficient between the histogram of the Y component of the frame at the m-th moment and the histogram of the Y component of the frame at the (m - 1)-th moment respectively represent the dispersion degree of the Y component histograms of the two frames. The value calculated in this way can reflect the difference between adjacent frames.

[0030] To make the content described in this application clearer, the following provides a formula for calculating the correlation coefficient between the histogram of the Y component of the frame at the m-th moment and the histogram of the Y component of the frame at the (m - 1)-th moment, as shown in the following formula:

[0031] Among them, is the histogram of the Y component of the frame at the (m - 1)-th moment, is the histogram of the Y component of the frame at the m-th moment, is and 's covariance, is 's variance, is 's variance, is the correlation coefficient between the histogram of the Y component of the frame at the m-th moment and the histogram of the Y component of the frame at the (m - 1)-th moment.

[0032] Set a threshold T to judge whether it is less than the threshold T. If is less than the threshold T, then is identified as the first scene change frame of the first video. T is a constant. To ensure finding more frames that may contain watermarks during the extraction process, the T value during the extraction process is usually set higher than the T value during the embedding process.

[0033] In the embodiment of this application, set the threshold T = 0.9. If < 0.9, it is determined as a scene change frame. After a large number of experimental verifications, when 's value is less than 0.9, it indicates that the content difference between adjacent frames is large, and a scene change is very likely to occur. For example, in a video, the previous frame is an indoor scene and the next frame switches to an outdoor scene. At this time, the calculated will be significantly less than 0.9 and thus be determined as a scene change frame.

[0034] S102. Process the QR code matrix to obtain an encrypted QR code matrix.

[0035] The size of the QR code matrix is r × r. In this application, set r = 21. This size can ensure containing sufficient watermark information while also being able to better adapt to subsequent processing operations. Here, use the Bernoulli distribution to generate a dynamic noise matrix for the QR code matrix, and process the QR code matrix, the dynamic noise matrix, and the time decay factor to obtain the encrypted QR code matrix. The calculation formula is as follows:

[0036] Among them, is the dynamic noise matrix, is the element located at the i-th row and j-th column in the dynamic noise matrix. i is the row index of the element in the dynamic noise matrix, j is the column index of the element in the dynamic noise matrix, and r represents the number of rows and columns of the dynamic noise matrix. , is a Bernoulli distribution with a probability p = 0.3. In the dynamic noise matrix, each element takes the value of 1 with a probability p = 0.3 and takes the value of 0 with a probability . In this way, noise elements are randomly generated within the range of the r×r QR code matrix to prepare for subsequent noise superposition; is the element located in the i-th row and j-th column of the encrypted QR code matrix, is the time decay factor, t is the timestamp of the frame at the m-th moment, is the element located in the i-th row and j-th column of the QR code matrix, represents the exclusive OR operation.

[0037] Then, according to the dynamic noise matrix and the time decay factor, the dynamic noise matrix is encrypted to obtain the encrypted noise parameter:

[0038] where k is the random private key, G is the elliptic curve base point, is the dynamic noise matrix, is the time decay factor, is the encrypted noise parameter, is the public key, and then the public key is calculated using the base point of the elliptic curve.

[0039] During the encryption process, a random private key is generated, and the security of the random private key is verified. According to the random private key, a hash value is obtained, and the hash value is judged against the preset secure hash value range to obtain a first judgment result. If the first judgment result indicates that the hash value is within the preset secure hash value range, it is confirmed that the random private key is secure and available, and the encrypted noise parameter is decrypted according to the random private key, and then the watermark information is extracted; if the first judgment result indicates that the hash value is not within the preset secure hash value range, a new random private key is regenerated and rejudged until the new random private key is secure and available. Only the legitimate recipient with the corresponding private key k can decrypt the encrypted noise parameter to correctly extract the watermark, greatly improving the security of the watermark.

[0040] If the security of the watermark embedded in the video is insufficient, it may lead to the illegal acquisition of the watermark information. For example, in digital media copyright protection, if the watermark is not secure, pirates can easily extract the watermark information, then remove or tamper with the watermark, and then spread the pirated content. In this way, the copyright owner cannot prove the copyright ownership of the work through the watermark, resulting in rampant copyright infringement.

[0041] In some scenarios that require verifying the authenticity of information, such as watermarks for electronic document signatures or identity authentication, if the watermark security is compromised, attackers can forge false documents or identity information with legitimate watermarks. Take electronic contracts as an example. Attackers may tamper with the contract content and then use the obtained watermark information to make it appear as a legitimate contract, which can pose serious legal and economic risks to activities such as commercial transactions. Watermarks are also commonly used to ensure data integrity. If the watermark is not secure, attackers can modify the data content at will without being detected because they can control the extraction and modification of the watermark. For example, in medical data storage, if the watermark of a medical image is illegally modified and the image content is tampered with, it may lead to serious consequences such as misdiagnosis.

[0042] Therefore, the process of generating a random private key and performing security verification increases the difficulty of obtaining a valid private key. Since not any randomly generated private key can be used, only the private key whose hash value passes the verification and falls within the preset secure hash value range is considered secure and usable. This judgment mechanism makes it difficult for attackers to easily guess or generate a valid private key. For example, assume that the hash value is generated by a complex encryption algorithm such as SHA - 256, and the output hash value has a high degree of discreteness and unpredictability. For an attacker to generate a private key within the preset secure hash value range, a large number of attempts are required, and such attempts are computationally very expensive.

[0043] Only the legitimate recipient with the corresponding private key k can decrypt the encrypted noise parameter. This means that the watermark information is effectively protected, just like a safe with a special keyhole that can only be opened by a key of a specific shape. For illegal users without the private key, even if they obtain the encrypted noise parameter, they cannot decrypt it correctly to extract the watermark.

[0044] The process of regenerating a new private key and re - judging when the generated private key hash value is not within the secure range further increases the time and computational cost for attackers to obtain a valid private key. If an attacker wants to obtain a valid private key through brute - force cracking, they need to restart the verification process every time a non - compliant private key is generated, which greatly reduces their probability of success.

[0045] Automatically adjust the encryption strategy according to the content type of the video. For example, for action movies, since the picture changes rapidly and the amount of information is large, the noise intensity can be appropriately increased or the update frequency of the encryption key can be adjusted to enhance the security of the watermark; for animated cartoons, a finer encryption granularity can be adopted, and encryption parameters can be set separately for different animated characters or scenes. Optimize the encryption based on the quality parameters of the video such as resolution and frame rate. For example, for high-resolution videos, increase the accuracy of encryption calculations to prevent the increased risk of watermark information leakage caused by high resolution; for low-frame-rate videos, adjust the number of iterations of the encryption algorithm to balance the consumption of computing resources and the security of the watermark.

[0046] S103. Perform size block processing on the first scene change frame in the first video to obtain multiple size image blocks.

[0047] Calculate the visual saliency value of the pixel points of the first scene change frame in the first video, and perform size block processing on the first scene change frame in the first video according to the visual saliency value to obtain multiple size image blocks. The calculation formula is as follows:

[0048]

[0049] Among them, is the visual saliency value of the pixel point of the first scene change frame. This visual saliency value reflects the significant degree of this pixel point for human eye vision in the entire image of the first scene change frame. The larger the value, the more the pixel point can attract the attention of the human eye. c is the color channel, including the red channel R, the green channel G, and the blue channel B. is the pixel value of the pixel point with coordinates on the color channel c. x is the coordinate position index of the pixel point in the horizontal direction of the first scene change frame, and y is the coordinate position index of the pixel point in the vertical direction of the first scene change frame. is the mean value of the pixel point on the color channel c. is the Gaussian kernel. is the Gaussian kernel function. The dynamic size of the size image block is determined by n. is the length of the horizontal direction and the vertical direction of the dynamic size of the size image block. is the maximum visual saliency value of the frame. is the full-frame mean value.

[0050] To achieve adaptive block processing, it is first necessary to calculate the visual saliency value of the first scene change frame. For each pixel point in the first scene change frame, calculations are performed separately on the three color channels of red (R), green (G), and blue (B). Indicates the difference between the pixel point and the mean value of that color channel on a certain color channel. The larger the difference, the more significant the pixel point. Then, through a Gaussian kernel weights these differences which can better balance the smoothing effect and edge preservation at this time. Finally, the results of the three color channels are added together to obtain the visual saliency value of the pixel point . The calculated visual saliency value is used to determine the dynamic size of the size image block. Here, based on 8, the block size is adjusted by the ratio of the maximum saliency value of the frame to the mean value of the full frame. For example, if there is a very significant object in a certain frame, such as a bright car headlight, which makes the maximum saliency value of the frame larger while the mean value of the full frame is relatively small, then the calculated n value will increase accordingly, that is, a larger block size is used for the significant region; on the contrary, in the non-significant region, the block size will be relatively small. Thus, the purpose of dynamic blocking is achieved, reducing the computational amount and improving the efficiency of watermark embedding and extraction

[0051] S104. Perform discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain the first DC coefficient and the first low-frequency AC coefficient

[0052] For an image of a given size, the discrete cosine transform calculates for each pixel point of the image. When both corresponding parameters in the calculation take 0, through the calculation process of the discrete cosine transform, the values of all pixel points of the image are comprehensively calculated. Simply put, it is to take into account the values of each pixel point in the image, perform a specific summation operation, and then through some processing related to the image size, the result obtained is the DC coefficient. The DC coefficient reflects the overall average brightness situation of the image, which is equivalent to making a general numerical representation of the image brightness from the overall perspective

[0053] Similarly, based on the calculation process of the discrete cosine transform, when specific parameter values are set, here set to both take 0.1, which can be changed according to needs in actual applications, the values of each pixel point of the image are calculated. This calculation process involves using a specific cosine function to weight the values of each pixel point, and then summing up the values of all pixel points after weighting and other operations. The final result obtained is the AC coefficient. The AC coefficient does not reflect the overall brightness of the image, but the information about the texture, details, etc. in the image. It can help us understand the characteristics of the image from another perspective. Below, the specific formula calculation will be used to elaborate in detail how the first DC coefficient and the first low-frequency AC coefficient are obtained through the discrete cosine transform

[0054] The formulas for calculating the first DC coefficient and the first low-frequency AC coefficient by the discrete cosine transform are as follows

[0055]

[0056] When and at this time, F(0, 0) is called the DC coefficient, that is, the first DC coefficient, and the formula is as follows:

[0057] In this application, it is set that when and at this time, F(0.1, 0.1) is called the AC coefficient, that is, the first low-frequency AC coefficient, and the formula is as follows:

[0058] Among them, M represents the number of pixels of the image in the horizontal direction, N is the number of pixels of the image in the vertical direction, u represents the frequency component in the horizontal direction, v represents the frequency component in the vertical direction, and different ( u , v ) values determine the position of this coefficient in the frequency domain, , are the weighting coefficients related to the frequency coordinates u , v , represents the pixel value of the image at the spatial domain coordinates ( x , y ).

[0059] S105. Process the first DC coefficient, the first low-frequency AC coefficient, and the encrypted quick response code matrix to obtain the second DC coefficient and the second low-frequency AC coefficient.

[0060] The calculation formula is as follows:

[0061]

[0062] Among them, is the second DC coefficient, is the second low-frequency AC coefficient, is the encrypted quick response code matrix, is the first DC coefficient, that is , is the first low-frequency AC coefficient, that is , since there may be multiple low-frequency AC coefficients in the size image block, the k-th low-frequency AC coefficient is selected as the first low-frequency AC coefficient, and k is an integer, is the first correlation coefficient related to the significance value of the size image block, is the second correlation coefficient related to the significance value of the size image block.

[0063] The DC coefficient represents the average luminance information of the size image block. Here, the elements of the encrypted QR code matrix are combined with the DC coefficient, is a coefficient related to the visual significance value of the size image block, . is the i row and the j column block significance metric value in the image, which is used to quantify the degree of attraction of this region to human vision. This means that the higher the visual significance of the block, the greater the modification amplitude of the DC coefficient, so as to more effectively embed the watermark in the visually sensitive area. In addition to the DC coefficient, the low-frequency alternating current (AC) coefficient is also modulated. The low-frequency AC coefficient contains the main texture information of the image. Similarly, is combined with the low-frequency AC coefficient, is a coefficient related to the visual significance value, . In this way, under the premise of ensuring the image quality, the watermark information is more covertly embedded into the video frame. Concentrate the watermark in the visually sensitive area (PSNR > 48dB). The visually sensitive area is often the part of the image with rich features and large amount of information. Embedding the watermark here can utilize these rich information features to better integrate the watermark with the image and enhance the robustness of the watermark. Compared with embedding the watermark in the non-sensitive area with single information and scarce features, the watermark embedded in the visually sensitive area is more difficult to be destroyed or removed by various image processing operations, thus improving the stability and detectability of the watermark in the subsequent use and dissemination process of the image, ensuring that the watermark can continuously play its roles of copyright protection, content authentication, etc. Moreover, while ensuring the existence of the watermark information, the impact on the image quality is minimized, making the PSNR value greater than 48dB, and the human eye can hardly detect the existence of the watermark.

[0064] S106. Perform the inverse discrete cosine transform on multiple size image blocks according to the second DC coefficient and the second low-frequency AC coefficient to implement the embedding of the watermark information, and obtain the first video with the watermark.

[0065] The calculation formula of the inverse discrete cosine transform is as follows:

[0066] The Discrete Cosine Transform (DCT) converts an image from the spatial domain to the frequency domain, while the Inverse Discrete Cosine Transform (IDCT) converts the image back from the frequency domain to the spatial domain. By performing the IDCT operation on the modified blocks of coefficients, the coefficients containing the watermark information in the frequency domain are converted back to the spatial domain, thereby obtaining the Y component containing the watermark. This step converts the watermark information from the form of frequency domain coefficients to the form of the luminance component of the image, enabling the watermark information to be embedded in the luminance part of the first scene change frame, thus achieving the embedding of the watermark information.

[0067] Since the width or height of the Y component was adjusted previously, after selecting the Y component of the scene change frame, it is determined whether its width or height can be r divided evenly. If not, operations such as enlargement or reduction are performed. Here, it is necessary to adjust the size of the Y component back to its original size. This step is to ensure the overall size and structural integrity of the first scene change frame, so that the first scene change frame with the embedded watermark is visually consistent with the original video frame in size and does not show deformation or other situations. For example, if the width or height of the Y component was enlarged to be divisible by r previously, now it needs to be reduced back to the original width and height values according to a certain ratio.

[0068] S107. Process the first video with the watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain the target estimated value of the encrypted quick response code matrix.

[0069] The calculation formula for the target estimated value of the encrypted quick response code matrix is as follows:

[0070]

[0071]

[0072] Among them, is the first part estimated value of the encrypted quick response code matrix, is the second part estimated value of the encrypted quick response code matrix, is the target estimated value of the encrypted quick response code matrix, is the first correlation coefficient related to the significance value of the image block of size, is the second correlation coefficient related to the significance value of the image block of size, is the second DC coefficient, is the second low-frequency AC coefficient, is the first DC coefficient, that is , is the first low-frequency AC coefficient, that is .

[0073] S108. Decrypt the target estimated value of the encrypted quick response code matrix to obtain the target quick response code matrix.

[0074] Use the corresponding decryption algorithm, such as the elliptic curve cryptosystem (ECC) decryption algorithm, to perform decryption. This step is to remove the encryption operation performed on the quick response code matrix during the watermark embedding process and restore its original coding form. At the same time, if interference factors such as noise are introduced during the watermark embedding process, operations such as noise elimination need to be combined with relevant noise parameters. For example, there may be operations such as quantum noise injection in the watermark scheme. At this time, according to the principle of noise generation and superposition, the influence of noise on the quick response code matrix needs to be reversely eliminated. After decryption and noise elimination operations, the target quick response code matrix is obtained.

[0075] S109. Decode the target quick response code matrix to obtain the target watermark information, and realize the extraction of the watermark information.

[0076] Since the method for decoding the target quick response code matrix is a prior art and will not be elaborated here, through decoding, the target watermark information is obtained, and the extraction of the watermark information is realized.

[0077] In order to further reduce the computational complexity of watermark information embedding and extraction, a watermark extraction optimization model is established here; first, calculate the confidence of the target watermark information; perform a reliability assessment according to the confidence of the target watermark information. If the action of terminating watermark extraction is taken and the confidence of the target watermark information is higher than the first confidence threshold, it is considered that the target watermark information is reliable, and a reward value is given to the watermark extraction optimization model; if the confidence of the target watermark information is higher than the first confidence threshold but the action of continuing watermark extraction is taken, it is considered that the extraction action is a waste of resources, and a penalty value is given to the watermark extraction optimization model; output the confidence ∈[0, 1].

[0078] When extracting the watermark, first decode the quick response code matrix for each frame in the video stream. Due to the influence of noise and various factors in the video processing process, the reliability of the decoding result needs to be evaluated through the confidence. The confidence ranges from 0 to 1. 0 means that the decoding result is completely untrustworthy, and 1 means that the decoding result is very reliable. For example, when decoding the quick response code matrix of a certain frame, if the noise interference is small and the decoded information matches the expected quick response code matrix format and error correction information well, the confidence of this frame will be close to 1; conversely, if it is affected by severe noise interference or decoding errors, it will be close to 0. The state in reinforcement learning is determined by the confidence of the current frame and the historical mean of the confidence of previous frames It is composed of. This state information reflects the real-time situation and overall trend of the confidence during the decoding process. For example, if the confidence of the current frame is high, and the historical mean is also high, it indicates that the current decoding process is relatively stable and reliable. The decision to terminate the intelligent extraction is achieved through the action . The action has only two values. 0 means continuing the decoding operation for the next frame, and 1 means terminating the decoding process, believing that reliable watermark information has been successfully extracted.

[0079] The calculation formulas for the reward and penalty functions are as follows:

[0080] The reward function is used to guide the reinforcement learning algorithm to make optimal decisions. When the action = 1 and the decoding is successful, a relatively high reward value of 10 is given, which encourages the algorithm to terminate in time when it can successfully decode and save computing resources. When the action = 0 and the confidence of the current frame > 0.99, a penalty value of -1 is given because continuing to decode when the confidence is already very high is a waste of resources. In other cases, the reward value is 0. By continuously adjusting the action, the algorithm can obtain the maximum cumulative reward in the long term, thus achieving the termination of intelligent extraction.

[0081] Based on the above, obtain the first video and the QR code matrix, and select the first scene change frame in the first video; process the QR code matrix to obtain an encrypted QR code matrix; perform size block processing on the first scene change frame in the first video to obtain multiple size image blocks, which can dynamically adjust the size according to the first scene change frame of the first video, and then perform block processing to reduce the computational complexity caused by a fixed size and improve the accuracy during subsequent watermark information extraction; then perform discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain the first DC coefficient and the first low-frequency AC coefficient; process the first DC coefficient, the first low-frequency AC coefficient, and the encrypted QR code matrix to obtain the second DC coefficient and the second low-frequency AC coefficient; perform inverse discrete cosine transform on the multiple size image blocks according to the second DC coefficient and the second low-frequency AC coefficient to implement the embedding of watermark information and obtain the first video with watermark; process the first video with watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain the target estimated value of the encrypted QR code matrix; decrypt the target estimated value of the encrypted QR code matrix to obtain the target QR code matrix, and decode the target QR code matrix to obtain the target watermark information, thereby implementing the extraction of watermark information. It can be seen that the present application provides a method for video watermark extraction based on QR codes, which can dynamically adjust the size of the embedded watermark, does not need to process all video frames according to a fixed size, reduces the computational complexity, improves the extraction efficiency, and also adds encryption processing to improve the security of the watermark.

[0082] The embodiment of the present application further provides a device for video watermark extraction based on QR codes, as Figure 2 shown. This figure is a schematic diagram of a device for video watermark extraction based on QR codes provided by the embodiment of the present application. The device includes: an acquisition module 201, an embedding module 202, and an extraction module 203; The acquisition module 201 is configured to acquire the first video and the QR code matrix, and select the first scene change frame in the first video; process the QR code matrix to obtain an encrypted QR code matrix.

[0083] The embedding module 202 is configured to perform size block processing on the first scene change frame in the first video to obtain multiple size image blocks; perform discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain the first DC coefficient and the first low-frequency AC coefficient; process the first DC coefficient, the first low-frequency AC coefficient, and the encrypted QR code matrix to obtain the second DC coefficient and the second low-frequency AC coefficient; perform inverse discrete cosine transform on the multiple size image blocks according to the second DC coefficient and the second low-frequency AC coefficient to implement the embedding of watermark information and obtain the first video with watermark.

[0084] An extraction module 203, configured to process the first video with a watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain a target estimated value of the encrypted QR code matrix; decrypt the target estimated value of the encrypted QR code matrix to obtain a target QR code matrix, and decode the target QR code matrix to obtain target watermark information, thereby realizing the extraction of watermark information.

[0085] In some possible implementation manners, the embedding module 202 is specifically configured to calculate the visual saliency value of the pixel points of the first scene change frame according to the first scene change frame in the first video, and perform size block processing on the first scene change frame in the first video according to the visual saliency value to obtain a plurality of sized image blocks.

[0086] In some possible implementation manners, the extraction module 203 is specifically configured to perform reduction processing on the second DC coefficient and the second low-frequency AC coefficient to obtain a first partial estimated value and a second partial estimated value of the encrypted QR code matrix, and process according to the first partial estimated value and the second partial estimated value of the encrypted QR code matrix to obtain a target estimated value of the encrypted QR code matrix.

[0087] In some possible implementation manners, the extraction module 203 is specifically configured to generate a dynamic noise matrix for the QR code matrix by using a Bernoulli distribution, and process the QR code matrix, the dynamic noise matrix, and a time decay factor to obtain an encrypted QR code matrix.

[0088] In some possible implementation manners, the apparatus further includes: A judgment module, configured to encrypt the dynamic noise matrix according to the dynamic noise matrix and the time decay factor to obtain an encrypted noise parameter.

[0089] During the encryption process, a random private key is generated, the security of the random private key is verified, a hash value is obtained according to the random private key, the hash value is judged against a preset secure hash value range to obtain a first judgment result. If the first judgment result indicates that the hash value is within the preset secure hash value range, it is confirmed that the random private key is secure and available, and the encrypted noise parameter is decrypted according to the random private key, and then the watermark information is extracted.

[0090] In some possible implementation manners, the judgment module is further configured to, if the first judgment result indicates that the hash value is not within the preset secure hash value range, regenerate a new random private key and re-judge until the new random private key is secure and available.

[0091] The embodiment of the present application further provides a computing device. AsFigure 3 As shown, this figure is a schematic diagram of a computing device provided by an embodiment of the present application. The computing device 400 includes a bus 401, a processor 402, a communication interface 403, and a memory 404. The processor 402, the memory 404, and the communication interface 403 communicate with each other through the bus 401.

[0092] The bus 401 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 3 it is only represented by a thick line in the figure, but it does not mean that there is only one bus or one type of bus.

[0093] The processor 402 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0094] The communication interface 403 is used for external communication. For example, when the computing device is the first switch, the communication interface 403 can be used for communication between the first switch and the first user terminal, or for communication between the first switch and the second switch.

[0095] The memory 404 can include volatile memory, such as random access memory (RAM). The memory 404 can also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0096] The memory 404 stores executable code, and the processor 402 executes the executable code to perform the aforementioned method for extracting video watermarks based on quick response codes.

[0097] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc. The computer-readable storage medium includes instructions that direct the computing device to execute the above-mentioned method for extracting video watermarks based on quick response codes.

[0098] The embodiments of the present application also provide a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, they generate all or part of the processes or functions described in the embodiments of the present application.

[0099] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, a computer, or a data center to another website, a computer, or a data center in a wired manner (such as coaxial cable, optical fiber) or a wireless manner (such as infrared, wireless, microwave, etc.).

[0100] When the computer program product is executed by a computer, the computer executes any of the methods for extracting video watermarks based on quick response codes described above. The computer program product can be a software installation package. In the case where any of the methods for extracting video watermarks based on quick response codes described above is needed, the computer program product can be downloaded and executed on the computer.

[0101] The descriptions of the processes or structures corresponding to the above-mentioned respective drawings each have their own focuses. For parts not detailed in a certain process or structure, reference can be made to the relevant descriptions of other processes or structures.

[0102] As described above, the above are only the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application.

Claims

1. A method for extracting video watermark based on Quick Response Code, characterized in that, The method includes: Obtaining a first video and a quick response code matrix, and selecting a first scene change frame in the first video; Processing the quick response code matrix to obtain an encrypted quick response code matrix; Performing size block processing on the first scene change frame in the first video to obtain a plurality of size image blocks; Performing discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain a first DC coefficient and a first low-frequency AC coefficient; Processing the first DC coefficient, the first low-frequency AC coefficient, and the encrypted quick response code matrix to obtain a second DC coefficient and a second low-frequency AC coefficient; Performing inverse discrete cosine transform on the plurality of size image blocks according to the second DC coefficient and the second low-frequency AC coefficient to embed watermark information, and obtaining a first video with a watermark; Processing the first video with a watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain a target estimated value of the encrypted quick response code matrix; Decrypting the target estimated value of the encrypted quick response code matrix to obtain a target quick response code matrix, decoding the target quick response code matrix to obtain target watermark information, and extracting the watermark information.

2. The method according to claim 1, wherein The performing size block processing on the first scene change frame in the first video to obtain a plurality of size image blocks includes: Calculating the visual saliency value of the pixel points of the first scene change frame according to the first scene change frame in the first video, and performing size block processing on the first scene change frame in the first video according to the visual saliency value to obtain a plurality of size image blocks.

3. The method according to claim 1, wherein The processing the first video with a watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain a target estimated value of the encrypted quick response code matrix includes: Performing reduction processing on the second DC coefficient and the second low-frequency AC coefficient to obtain a first partial estimated value and a second partial estimated value of the encrypted quick response code matrix, and processing according to the first partial estimated value and the second partial estimated value of the encrypted quick response code matrix to obtain a target estimated value of the encrypted quick response code matrix.

4. The method according to claim 1, characterized in that, The processing the quick response code matrix to obtain an encrypted quick response code matrix includes: Generating a dynamic noise matrix for the quick response code matrix using a Bernoulli distribution, and processing the quick response code matrix, the dynamic noise matrix, and a time decay factor to obtain an encrypted quick response code matrix.

5. The method according to claim 4, wherein The method further includes: Encrypting the dynamic noise matrix according to the dynamic noise matrix and the time decay factor to obtain an encrypted noise parameter; During the encryption process, generating a random private key, performing security verification on the random private key, obtaining a hash value according to the random private key, judging the hash value with a preset secure hash value range to obtain a first judgment result. If the first judgment result indicates that the hash value is within the preset secure hash value range, it is confirmed that the random private key is safe and available, and decrypting the encrypted noise parameter according to the random private key, and further extracting the watermark information.

6. The method according to claim 5, wherein The method further includes: If the first judgment result indicates that the hash value is not within the preset range of secure hash values, a new random private key is regenerated and rejudged until the new random private key is secure and available.

7. A video watermark extraction device based on a quick response code, characterized in that The device includes: An acquisition module, configured to acquire a first video and a quick response code matrix, and select a first scene change frame in the first video; process the quick response code matrix to obtain an encrypted quick response code matrix; An embedding module, configured to perform size block processing on the first scene change frame in the first video to obtain a plurality of size image blocks; perform discrete cosine transform on the luminance component of the first scene change frame in the first video to obtain a first DC coefficient and a first low-frequency AC coefficient; process the first DC coefficient, the first low-frequency AC coefficient, and the encrypted quick response code matrix to obtain a second DC coefficient and a second low-frequency AC coefficient; perform inverse discrete cosine transform on the plurality of size image blocks according to the second DC coefficient and the second low-frequency AC coefficient to implement the embedding of watermark information and obtain a first video with a watermark. An extraction module, configured to process the first video with a watermark according to the second DC coefficient and the second low-frequency AC coefficient to obtain an estimated value of the target encrypted quick response code matrix; decrypt the estimated value of the target encrypted quick response code matrix to obtain a target quick response code matrix, and decode the target quick response code matrix to obtain target watermark information, thereby implementing the extraction of watermark information.

8. The device according to claim 7, characterized in that, The embedding module is specifically configured to calculate the visual saliency value of the pixel points of the first scene change frame according to the first scene change frame in the first video, and perform size block processing on the first scene change frame in the first video according to the visual saliency value to obtain a plurality of size image blocks.

9. A computing device, characterized in that, It includes a memory and a processor; Wherein, one or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device is caused to execute the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 6.

Images