A deep learning-based encrypted video watermark embedding and extraction method

Abstract

The present application relates to the technical field of digital watermarking, and proposes an encryption video watermark embedding and extraction method based on deep learning, wherein after H.264 compressed video code stream is parsed, a selective encryption operation based on code word is performed to generate encrypted compressed domain video; in response to a video download request of a user, a unique watermark sequence is generated, the encryption state of the video is maintained, the watermark is embedded into the encrypted video, and the encrypted video containing the watermark is sent to the user; a to-be-detected video is acquired, and based on multiple video frames in which watermarks have been embedded, multiple multi-dimensional sparse syntax feature tensors are constructed from the compressed domain; based on the tensors, an anti-recompression watermark extraction network is used to decode the multiple video frames in which watermarks have been embedded, so that a prediction result of a watermark bit sequence carried by each video frame in which a watermark has been embedded is obtained; the multiple prediction results are spliced to extract a complete watermark. The present application can realize fast embedding of video watermark in a cloud environment and effectively resist re-compression attacks.

Description

Technical Field

[0001] This invention relates to the technical field of digital watermarking, and more specifically, to a method for embedding and extracting encrypted video watermarks based on deep learning. Background Technology

[0002] With the maturity of video compression technologies (such as H.264 / AVC) and the widespread adoption of cloud computing and network storage technologies, massive amounts of video data are hosted on cloud servers for centralized storage and distribution. This centralized management greatly reduces enterprise operation and maintenance costs and improves the convenience of data access. However, this convenience also introduces privacy and security risks that cannot be ignored.

[0003] To ensure data security, the industry currently primarily uses cryptographic encryption technology to protect video data uploaded to the cloud. Encryption technology can effectively prevent unauthorized users (such as malicious external attackers) from stealing content during data transmission or static storage. However, encryption technology has an inherent logical limitation: it can only prevent "unauthorized access," but cannot defend against "illegal disclosure after authorized access." Therefore, digital watermarking (or digital fingerprinting) technology—which can retain the identity of data after decryption and enable tracing the source of leakage—has become a necessary supplement to improving cloud storage security systems.

[0004] However, deploying video watermarking in cloud storage architectures faces significant challenges in terms of computational efficiency and compatibility. First, cloud servers face massive concurrent video download requests, making computing resources extremely valuable. Traditional video watermarking algorithms typically operate at the pixel level, meaning the server needs to perform "full decoding—pixel-domain embedding—re-encoding" of the compressed video, consuming enormous computing power and significantly increasing download latency for users. Second, to balance security and efficiency, modern video security solutions often employ "selective encryption," encrypting only key syntactic elements (such as intra-frame prediction modes and motion vector symbols) in the video stream. While efficient, this approach leads to difficulties in watermark embedding: existing compressed-domain watermarking schemes require reading or modifying specific syntactic elements, which may have been encrypted, scrambled, or hidden. How to covertly embed the watermark into the partially encrypted compressed domain without decrypting the video stream, and without compromising the format compliance of the encrypted stream, remains a major technological challenge.

[0005] Meanwhile, existing technologies lack robustness against recompression attacks. Leaked videos, when disseminated online, are highly likely to be subjected to various signal processing attacks. The most common and destructive attack is video recompression / transcoding. When insiders upload videos to social media, instant messaging software, or video sharing platforms, the platforms typically re-encode the videos to save bandwidth. This operation causes drastic changes to the syntax elements of the video compression domain: for example, previously non-zero DCT transform coefficients may become zero due to coarsening quantization, and the original motion vector difference (MVD) may be completely altered due to re-motion estimation. Most existing compression domain watermark extraction schemes rely on simple hard logic rules (such as parity checking) or basic statistical voting (such as majority voting). These traditional methods treat every macroblock in a video frame as equally important, ignoring the spatiotemporal differences in the video content itself. When subjected to strong recompression attacks, these methods cannot distinguish between "reliable features" and "noise features," resulting in extremely high error rates and an inability to accurately reconstruct watermark information, leading to failed source tracing. Summary of the Invention

[0006] To address the challenges of embedding video watermarks and the weak resistance to recompression attacks in cloud storage environments, this invention proposes a deep learning-based method for embedding and extracting encrypted video watermarks. This method enables rapid embedding of video watermarks in cloud environments while effectively resisting recompression attacks.

[0007] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows: This invention proposes a method for embedding encrypted video watermarks, comprising the following steps: In response to a user's video download request, a unique watermark sequence is generated, the video remains encrypted, the watermark is embedded in the encrypted video, and the watermarked encrypted video is sent to the user; the encrypted video is an encrypted compressed video generated by parsing the video stream to be encrypted and performing selective encryption based on codewords.

[0008] Preferably, the process of parsing the video stream to be encrypted and compressed, and then performing selective encryption based on codewords to generate the encrypted compressed domain video, is as follows: S11: Decapsulate the video stream to be encrypted and compressed from the extended byte sequence payload into the original data bitstream; S12: Locate the key syntax element at the macroblock level in the original data bitstream and generate a pseudo-random binary sequence with the same length as the key syntax element; S13: Using the pseudo-random binary sequence, perform XOR encryption on the key syntax elements of the original data bitstream; S14: Repackage the encrypted original data bitstream to generate an encrypted compressed domain video.

[0009] Preferably, in response to a user's video download request, a unique watermark sequence is generated, the video remains encrypted, the watermark is embedded into the encrypted video, and the watermarked encrypted video is sent to the user. The process is as follows: S21: In response to the user's video download request, generate a unique identifier based on the user's authorization information, use the identifier as a watermark sequence, and divide the watermark sequence into independent watermark subgroups; S22: Select the last P frame of each image group in the encrypted video as the watermark embedding frame. In the watermark embedding frame, the watermark embedding process is divided into several independent and serially executed embedding operations. Each embedding operation corresponds to a syntax element channel. In each syntax element channel, the watermark subgroup is cyclically embedded into the syntax element of each watermark embedding frame to obtain the watermarked encrypted video.

[0010] Preferably, the syntax element channels include a trailing coefficient sign channel, a non-zero transform coefficient channel, and a motion vector difference channel; In each syntax element channel, the process of cyclically embedding the watermark subgroups into the syntax element of each watermark embedding frame to obtain the watermarked encrypted video is as follows: Group all watermarks and concatenate them cyclically to form a cyclic sequence; In the trailing coefficient symbol channel, the cyclic sequence is overwritten to the trailing coefficient symbol in the watermark embedding frame until all trailing coefficient symbols in the current watermark embedding frame have been overwritten and replaced. Then the next watermark embedding frame is processed until all watermark embedding frames have been processed. In the non-zero transform coefficient channel, the index of the cyclic sequence is reset. Based on the bits of the cyclic sequence, the parity of each codeword of the non-zero transform coefficient is modified according to the parity mapping rule until all non-zero transform coefficients of the current watermark embedding frame are modified. Then, the next watermark embedding frame is processed until all watermark embedding frames are processed. In the motion vector difference channel, the index of the cyclic sequence is reset. Based on the bits of the cyclic sequence, the parity of each codeword of the motion vector difference is modified according to the parity mapping rule until all motion vector differences of the current watermark embedding frame have been modified. Then, the next watermark embedding frame is processed until all watermark embedding frames have been processed.

[0011] This invention proposes a method for extracting watermarks from encrypted videos, comprising the following steps: The video to be detected is obtained, and multiple multidimensional sparse syntax feature tensors are constructed from the compressed domain based on multiple video frames with embedded watermarks. Based on the multiple multidimensional sparse syntax feature tensors, a pre-trained anti-recompression watermark extraction network is used to decode multiple video frames with embedded watermarks to obtain the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks. Multiple prediction results are spliced ​​together to extract the complete watermark.

[0012] Preferably, the video to be detected is acquired, and based on multiple video frames with embedded watermarks, multiple multidimensional sparse syntax feature tensors are constructed from the compressed domain. The process is as follows: Entropy decoding is performed on the video to be tested in the compressed domain; Based on the decoded video to be detected, multiple multidimensional sparse syntax feature tensors are constructed using macroblocks of video frames with embedded watermarks as basic units. These multidimensional sparse syntax feature tensors are three-dimensional tensors of size H×W×C, where H and W are the frame height and frame width per macroblock, respectively, and C is the number of feature channels, including: the original watermark extraction channel and the anti-attack statistical feature channel. The anti-attack statistical feature channel contains a noisy bitstream initially parsed from the trailing coefficient sign channel, the non-zero transform coefficient channel, and the motion vector difference channel. This noisy bitstream contains a large number of errors. The anti-attack statistical feature channel explicitly aggregates statistics used by the auxiliary network to determine the reliability of the macroblock, including: the total number of non-zero coefficients within the macroblock, the motion vector magnitude, the quantization parameter QP, and the number of bits in the macroblock.

[0013] Preferably, the pre-trained anti-compression watermark extraction network includes a group of convolutional layers, a spatial attention module, a global average pooling layer, and a fully connected layer; Based on the aforementioned multidimensional sparse syntax feature tensors, a pre-trained anti-recompression watermark extraction network is used to decode multiple watermarked video frames to obtain the prediction result of the watermark bit sequence carried by each watermarked video frame. The process is as follows: C1: Utilize convolutional layers to extract macroblocks and spatial correlation features of macroblock neighborhoods from multidimensional sparse grammar feature tensors; C2: Based on spatial correlation features, different weight masks are assigned to macroblocks using the attention mechanism of the spatial attention module; C3: Uses a global average pooling layer to compress spatial correlation features into a global feature vector; C4: Based on the weight mask and global feature vector, using a fully connected layer and activation function, obtain the probability vector of the payload carried by the corresponding watermarked video frame; use the probability vector as the prediction result; C5: Repeat steps C1 to C4 until all video frames with embedded watermarks are decoded, resulting in multiple prediction results.

[0014] The present invention also proposes an encrypted video watermark embedding device for implementing the encrypted video watermark embedding method, including a user terminal and a watermark embedding terminal. On the user side, the encrypted compressed video stream is parsed, and a selective encryption operation based on codewords is performed to generate an encrypted compressed domain video. The watermark embedding terminal is deployed on a cloud server. In response to the user's video download request, the watermark embedding terminal generates a unique watermark sequence, keeps the video encrypted and does not decrypt it, embeds the watermark into the encrypted video, and sends the watermarked encrypted video to the user.

[0015] The present invention also proposes an encrypted video watermark extraction device for implementing the encrypted video watermark extraction method, including an extraction end, wherein the extraction end is deployed on a cloud server; At the extraction end, the video to be detected is obtained, and based on multiple video frames with embedded watermarks, multiple multidimensional sparse syntax feature tensors are constructed from the compressed domain respectively. The extraction end is equipped with a pre-trained anti-recompression watermark extraction network. The pre-trained anti-recompression watermark extraction network decodes multiple video frames with embedded watermarks based on the multiple multidimensional sparse syntax feature tensors, and obtains the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks.

[0016] The present invention also proposes an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement steps in the encrypted video watermark embedding method, or to execute steps in the encrypted video watermark extraction method.

[0017] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention proposes a deep learning-based method for embedding and extracting encrypted video watermarks. On the user's end, after parsing the video stream to be encrypted and compressed, selective encryption based on codewords is performed to generate an encrypted compressed video domain and upload it to a cloud server. The cloud server's embedding end responds to the user's video download request, generates a unique watermark sequence, keeps the video encrypted, embeds the watermark into the encrypted video, and sends the watermarked encrypted video to the user. In the event of video leakage or copyright disputes, the cloud server's extraction end acquires the video to be detected, which may be subject to recompression attacks. Based on multiple watermarked video frames, multiple multidimensional sparse syntax feature tensors are constructed from the video compression domain. Based on these multidimensional sparse syntax feature tensors, a deep learning-based anti-recompression watermark extraction network is used to decode the multiple watermarked video frames, obtaining a prediction result for the watermark bit sequence carried by each watermarked video frame. The multiple prediction results are concatenated to extract the complete watermark. This invention enables rapid embedding of video watermarks in a cloud environment and effectively resists recompression attacks. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the encrypted video watermark embedding method proposed in this embodiment of the invention. Figure 2 This is a schematic diagram illustrating the process of encrypting the compressed video stream to be encrypted on the user end, as proposed in this embodiment of the invention. Figure 3 This is a schematic diagram illustrating the process of embedding watermarks into encrypted compressed video streams as proposed in this embodiment of the invention. Figure 4 This is a flowchart illustrating the encrypted video watermark extraction method proposed in this embodiment of the invention. Figure 5 This diagram illustrates the architecture for constructing a multidimensional sparse syntax feature tensor and extracting watermark sequences proposed in this embodiment of the invention. Figure 6 This diagram illustrates the structure of the encrypted video watermark embedding device proposed in this embodiment of the invention. Figure 7 This diagram illustrates the structure of the encrypted video watermark extraction device proposed in this embodiment of the invention. Figure 8 This is a structural diagram of the electronic device proposed in the embodiments of the present invention. Detailed Implementation

[0019] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions; It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.

[0020] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments; The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0021] Example 1 This embodiment provides a method for embedding encrypted video watermarks. The flowchart of this method is shown below. Figure 1 This includes the following steps: S1: In response to the user's video download request, generate a unique watermark sequence, keep the video encrypted and not decrypted, embed the watermark into the encrypted video, and send the watermarked encrypted video to the user; the encrypted video is an encrypted compressed field video generated by parsing the video stream to be encrypted and performing selective encryption operation based on codewords.

[0022] In this embodiment, the user terminal parses the H.264 compressed video stream, performs selective encryption based on codewords, and uploads the generated encrypted compressed video to the cloud server for storage. The cloud server responds to the user's video download request, generates a unique watermark sequence based on the user's authorization information, and embeds the watermark sequence into the video stream by modifying specific syntax elements in the video compression field while keeping the video encrypted. This generates a watermarked encrypted video and sends it to the downloading user.

[0023] Example 2 In this embodiment, the process of parsing the video stream to be encrypted and compressed, and then performing selective encryption based on codewords to generate an encrypted compressed domain video is as follows: S11: Decapsulate the video stream to be encrypted and compressed from the extended byte sequence payload into the original data bitstream; S12: Locate the key syntax element at the macroblock level in the original data bitstream and generate a pseudo-random binary sequence with the same length as the key syntax element; S13: Using the pseudo-random binary sequence, perform XOR encryption on the key syntax elements of the original data bitstream; S14: Repackage the encrypted original data bitstream to generate an encrypted compressed domain video.

[0024] Specifically, such as Figure 2As shown, in step S11, the video stream to be encrypted and compressed is the original H.264 video stream. By extracting the NAL unit from the network layer, the original H.264 video stream is decapsulated from the Extended Byte Sequence Payload (EBSP) into the original data bitstream SODB. In step S12, the user sets a key generator, which can use either the Advanced Encryption Standard (AES) or the RC4 stream cipher algorithm as a pseudo-random number generator to generate a pseudo-random binary sequence with the same total length as the aforementioned key syntax elements. In step S13, the macroblock-level data in the original data bitstream SODB is parsed, and the key syntax elements are XOR-encrypted bit by bit. In step S14, the encrypted original data bitstream SODB is recapsulated into an H.264 compliant stream as the encrypted compressed video and uploaded to the cloud server. Since only the values ​​of the key syntax elements are modified without destroying their semantic structure, the cloud server can identify the stream structure, such as macroblock boundaries and frame types, without decryption, preserving format compatibility for subsequent operations in the compression domain.

[0025] Key syntactic elements include: Intra Prediction Modes: These include 4x4, 16x16, and chroma prediction modes. Encrypting these modes can destroy the spatial texture structure of the image. Signs of MVD: Encrypting the sign bits can cause errors in the motion compensation direction during decoding, resulting in severe visual jitter. The sign bits of the residual coefficients: only the sign bits of the non-zero transform coefficients are encrypted, which can cause noise interference without changing the bitstream length.

[0026] In this embodiment, in response to a user's video download request, a unique watermark sequence is generated, the video remains encrypted, the watermark is embedded into the encrypted video, and the watermarked encrypted video is sent to the user. The process is as follows: S21: In response to the user's video download request, generate a unique identifier based on the user's authorization information, use the identifier as a watermark sequence, and divide the watermark sequence into independent watermark subgroups; S22: Select the last P frame of each image group in the encrypted video as the watermark embedding frame. In the watermark embedding frame, the watermark embedding process is divided into several independent and serially executed embedding operations. Each embedding operation corresponds to a syntax element channel. In each syntax element channel, the watermark subgroup is cyclically embedded into the syntax element of each watermark embedding frame to obtain the watermarked encrypted video.

[0027] Specifically, such as Figure 3As shown, in step S21, a unique identifier of 50 bits is generated for the user who made the download request, serving as a watermark sequence. To accommodate the risk of packet loss or clipping during network transmission, the 50-bit unique identifier is divided into 10 independent sub-groups, serving as watermark sub-groups, each consisting of a 5-bit payload. In step S22, the embedding process employs a three-channel redundant parallel strategy: the last P-frame of each Group of Picture Optimization (GOP) in the encrypted video stream is selected as the watermark embedding frame. Within this frame, the Sign_of_T1s channel, the Level channel (non-zero transform coefficients), and the MVD channel (motion vector difference) are used in parallel to repeatedly embed the aforementioned sub-group data in all macroblocks of the frame according to the raster scan order. After embedding, the 50-bit watermark sequence is distributed across 10 different GOPs, resulting in a watermarked encrypted video. This strategy ensures that each type of syntax element is filled with watermark information, maximizing channel capacity. The video watermark embedding operation directly modifies the compressed field bits and avoids the encrypted bits in S13, ensuring that the video watermark embedding and the video encryption state do not interfere with each other.

[0028] The syntax element channels include a trailing coefficient sign channel, a non-zero transform coefficient channel, and a motion vector difference channel; In each syntax element channel, the process of cyclically embedding the watermark subgroups into the syntax element of each watermark embedding frame to obtain the watermarked encrypted video is as follows: Group all watermarks and concatenate them cyclically to form a cyclic sequence; In the trailing coefficient symbol channel, the cyclic sequence is overwritten to the trailing coefficient symbol in the watermark embedding frame until all trailing coefficient symbols in the current watermark embedding frame have been overwritten and replaced. Then the next watermark embedding frame is processed until all watermark embedding frames have been processed. In the non-zero transform coefficient channel, the index of the cyclic sequence is reset. Based on the bits of the cyclic sequence, the parity of each codeword of the non-zero transform coefficient is modified according to the parity mapping rule until all non-zero transform coefficients of the current watermark embedding frame are modified. Then, the next watermark embedding frame is processed until all watermark embedding frames are processed. In the motion vector difference channel, the index of the cyclic sequence is reset. Based on the bits of the cyclic sequence, the parity of each codeword of the motion vector difference is modified according to the parity mapping rule until all motion vector differences of the current watermark embedding frame have been modified. Then, the next watermark embedding frame is processed until all watermark embedding frames have been processed.

[0029] Specifically, each watermark subgroup consists of a 5-bit payload. All watermark subgroups are cyclically concatenated to form a cyclic sequence: Bit 1, Bit 2... Bit 5, Bit 1, Bit 2... Bit 5, Bit 1...; In the trailing coefficient sign channel, if the frame has N trailing coefficient signs (Sign_of_T1s), the watermark sequence will be repeated N / 5 times. In the non-zero transform coefficient channel and motion vector difference channel, the parity mapping rule is that odd sets represent 0 and even sets represent 1.

[0030] This spatial cyclic embedding mechanism leverages the massive number of macroblocks within a frame to achieve extremely high information redundancy. Through this independent traversal mechanism, a macroblock may contain multiple watermark bits from different locations; for example, its trailing coefficient symbol carries the first watermark bit, while the non-zero transform coefficient carries the fourth. This unaligned redundancy distribution significantly enhances resistance to attacks, because recompression often causes varying degrees of damage to different types of syntax elements.

[0031] Example 3 This embodiment provides a method for extracting watermarks from encrypted videos. The flowchart of this method is shown below. Figure 4 This includes the following steps: S1: Obtain the video to be detected, and construct multiple multidimensional sparse syntax feature tensors from the compressed domain based on multiple video frames with embedded watermarks; S2: Based on the multiple multidimensional sparse syntax feature tensors, a pre-trained anti-recompression watermark extraction network is used to decode multiple video frames with embedded watermarks to obtain the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks. S3: Combine multiple prediction results to extract the complete watermark.

[0032] In this embodiment, when a video leak or copyright dispute occurs, the extraction end acquires the video to be detected. This video may have been subjected to a recompression attack. Based on multiple video frames with embedded watermarks, multiple multidimensional sparse syntax feature tensors are constructed from the compression domain. The constructed multidimensional sparse syntax feature tensors are input into a pre-trained anti-recompression watermark extraction network. The anti-recompression watermark extraction network is a deep neural network model combined with an attention mechanism. This network uses a global feature aggregation mechanism to directly decode the corresponding video frames with embedded watermarks based on the multidimensional sparse syntax feature tensors, and outputs the prediction result of the watermark sub-bit sequence carried by the frame. The prediction results of all embedded frames are then concatenated in time to restore the complete original watermark information.

[0033] Example 4 In this embodiment, the video to be detected is acquired, and based on multiple video frames with embedded watermarks, multiple multidimensional sparse syntax feature tensors are constructed from the compressed domain. The process is as follows: Entropy decoding is performed on the video to be tested in the compressed domain; Based on the decoded video to be detected, multiple multidimensional sparse syntax feature tensors are constructed using macroblocks of video frames with embedded watermarks as basic units. These multidimensional sparse syntax feature tensors are three-dimensional tensors of size H×W×C, where H and W are the frame height and frame width in macroblock units, respectively, and C is the number of feature channels, including: the original watermark extraction channel and the anti-attack statistical feature channel. The anti-attack statistical feature channel contains a noisy bitstream initially parsed from the trailing coefficient sign channel, the non-zero transform coefficient channel, and the motion vector difference channel, respectively. This noisy bitstream contains a large number of errors. The anti-attack statistical feature channel is explicitly aggregated.

[0034] Statistical measures used by auxiliary networks to determine macroblock reliability include: the total number of non-zero coefficients within the macroblock, the motion vector magnitude, the quantization parameter QP, and the number of bits in the macroblock. Specifically, such as... Figure 5 As shown, entropy decoding is performed on the video to be detected in the compressed domain, but pixel reconstruction is not performed. Based on the decoded video to be detected, for each video frame with an embedded watermark, a multidimensional sparse syntax feature tensor with dimensions H×W×C is constructed, where H and W are the frame height and frame width of the macroblock of that frame, and C is the number of feature channels. The feature channels include the original watermark extraction channel and the anti-attack statistical feature channel.

[0035] In the original watermark extraction channel, the independent sequences of specific syntax elements in each macroblock are traversed. The specific syntax elements include the trailing coefficient symbol Sign_of_T1s, the non-zero transform coefficient Level, and the motion vector difference MVD. All potential binary noisy bitstreams are extracted from the bitstream. These data may contain a large number of errors due to recompression attacks. The macroblock coordinates to which each syntax element belongs are recorded and mapped back to the tensor space.

[0036] Specifically, in Channel 1 (Sign_of_T1s), the average or majority vote value of all tail coefficient sign bits extracted from the macroblock is mapped back to the tensor space; in Channel 2 (Level), the non-zero transform coefficient bits resolved from the macroblock are mapped back to the tensor space; and in Channel 3 (MVD), the motion vector difference bits resolved from the macroblock are mapped back to the tensor space. If a certain type of syntax element is missing from the macroblock, the corresponding position in the tensor space is filled with 0.

[0037] In the anti-attack statistical feature channel, the statistical measures used by the explicit aggregation auxiliary network to judge the reliability of macroblocks include the total number of non-zero coefficients in the macroblock, which is used to characterize texture complexity; motion vector magnitude, which is used to characterize the intensity of motion; quantization parameter QP, which is used to characterize the quantization step size; and macroblock bit count, which is used to characterize the information weight of the macroblock in the encoding.

[0038] Specifically, in Channel 4, the total number of non-zero coefficients in the macroblock is counted; in Channel 5, the motion vector difference (MVD) magnitude of the macroblock is counted; and in Channel 6, the actual number of bits occupied after encoding the macroblock is counted.

[0039] The pre-trained anti-compression watermark extraction network includes a group of convolutional layers, a spatial attention module, a global average pooling layer, and a fully connected layer. Based on the aforementioned multidimensional sparse syntax feature tensors, a pre-trained anti-recompression watermark extraction network is used to decode multiple watermarked video frames to obtain the prediction result of the watermark bit sequence carried by each watermarked video frame. The process is as follows: C1: Utilize convolutional layers to extract macroblocks and spatial correlation features of macroblock neighborhoods from multidimensional sparse grammar feature tensors; C2: Based on spatial correlation features, different weight masks are assigned to macroblocks using the attention mechanism of the spatial attention module; C3: Uses a global average pooling layer to compress spatial correlation features into a global feature vector; C4: Based on the weight mask and global feature vector, using a fully connected layer and activation function, obtain the probability vector of the payload carried by the corresponding watermarked video frame; use the probability vector as the prediction result. C5: Repeat steps C1 to C4 until all video frames with embedded watermarks are decoded, resulting in multiple prediction results.

[0040] Since the above watermark extraction process is frame-independent, there is no need for complex cross-frame synchronization calculations. By simply splicing the multiple prediction results together in the order of video frame playback, a complete watermark sequence can be obtained.

[0041] Specifically, such as Figure 5As shown, the convolutional layer group includes 3-4 convolutional layers. In step C2, based on spatial correlation features, the spatial attention module automatically learns using the Spatial Attention mechanism to generate a weight mask. Watermarks extracted from macroblocks with extremely complex textures or excessively high QP values ​​are usually unreliable, therefore these macroblocks are assigned low weights; macroblocks with moderate textures and high encoding bit counts are assigned higher weights. In step C3, the H×W spatial features are compressed into a global feature vector using a global average pooling layer, where H and W are the frame height and frame width of the macroblock in that frame. In step C4, a probability vector of length 5 is output using a fully connected layer and a sigmoid activation function, for example: [0.9, 0.1, 0.8, 0.05, 0.95], corresponding to the confidence level of the 5-bit payload carried in the current frame. This probability vector is used as the prediction result, and the extracted 5-bit value is determined based on the prediction probability (usually with a threshold of 0.5). In step C5, steps C1 to C4 are performed on 10 consecutive video frames with embedded watermarks. Each group of bits is concatenated according to the frame order to obtain the final extracted watermark.

[0042] In this embodiment, to ensure the robustness of the anti-recompression watermark extraction network, an offline training phase is included. A large number of video sequences are selected, and randomly generated 5-bit group labels are embedded according to the watermark embedding method described in this application. Multiple intensities of H.264 recompression and transcoding attacks are then performed to generate a damaged compressed domain grammatical feature tensor. During training, the damaged compressed domain grammatical feature tensor is used as input, and the original embedded 5-bit group labels are used as ground truth labels. Supervised training using the binary cross-entropy loss function (BCE Loss) optimizes the model parameters, enabling it to learn to accurately infer the nonlinear pattern of the watermark sequence bits from sparse and noisy grammatical elements under recompression interference.

[0043] Example 5 This embodiment provides an encrypted video watermark embedding device, see [link to documentation]. Figure 6 The method for implementing the encrypted video watermark embedding method includes a user terminal 1 and a watermark embedding terminal 2. Client 1 parses the encrypted compressed video stream and performs selective encryption based on codewords to generate encrypted compressed domain video; The watermark embedding terminal 2 is deployed on the cloud server 3. In response to the user's video download request, the watermark embedding terminal 2 generates a unique watermark sequence, keeps the video encrypted and does not decrypt it, embeds the watermark into the encrypted video, and sends the encrypted video with the watermark to the user.

[0044] This embodiment also provides an encrypted video watermark extraction device, see [link to documentation]. Figure 7The method for extracting encrypted video watermarks includes an extraction terminal 5, which is deployed on a cloud server 6. Extraction end 5: Obtain the video to be detected, and construct multiple multidimensional sparse syntax feature tensors from the compression domain based on multiple video frames with embedded watermarks. The extraction end 5 is equipped with a pre-trained anti-recompression watermark extraction network 7. The pre-trained anti-recompression watermark extraction network 7 decodes multiple video frames with embedded watermarks based on the multiple multidimensional sparse syntax feature tensors, and obtains the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks.

[0045] This application also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described encrypted video watermark embedding method embodiment, or implements the various processes of the encrypted video watermark extraction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0046] For details, see Figure 8 This application also provides an electronic device, including a bus 401, a transceiver 402, an antenna 403, a bus interface 404, a processor 405, and a memory 406.

[0047] The transceiver 402 is used to acquire at least one of the video stream to be encrypted and compressed and the video to be detected; The processor 405 parses the video stream to be encrypted and compressed, performs selective encryption based on codewords to generate an encrypted compressed video; responds to the user's video download request, generates a unique watermark sequence, keeps the video encrypted without decryption, embeds the watermark into the encrypted video, and sends the watermarked encrypted video to the user; acquires the video to be detected, and constructs multiple multidimensional sparse syntax feature tensors from the compressed domain based on multiple video frames with embedded watermarks; based on the multiple multidimensional sparse syntax feature tensors, uses a pre-trained anti-recompression watermark extraction network 7 to decode the multiple video frames with embedded watermarks, and obtains the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks; concatenates the multiple prediction results to extract the complete watermark.

[0048] exist Figure 8In this context, a bus architecture (represented by bus 401) is used. Bus 401 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 405 and memory represented by memory 406. Bus 401 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 404 provides an interface between bus 401 and transceiver 402. Transceiver 402 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 405 is transmitted over a wireless medium via antenna 403, which further receives data and transmits data to processor 405.

[0049] Processor 405 is responsible for managing bus 401 and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 406 can be used to store data used by processor 405 during operation.

[0050] Optionally, the processor 405 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0051] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for embedding encrypted video watermarks, characterized in that, Includes the following steps: In response to a user's video download request, a unique watermark sequence is generated, the video remains encrypted, the watermark is embedded in the encrypted video, and the watermarked encrypted video is sent to the user; the encrypted video is an encrypted compressed video generated by parsing the video stream to be encrypted and performing selective encryption based on codewords.

2. The encrypted video watermark embedding method according to claim 1, characterized in that, The process of parsing the video stream to be encrypted and compressed, and then performing selective encryption based on codewords to generate the encrypted compressed domain video is as follows: S11: Decapsulate the video stream to be encrypted and compressed from the extended byte sequence payload into the original data bitstream; S12: Locate the key syntax element at the macroblock level in the original data bitstream and generate a pseudo-random binary sequence with the same length as the key syntax element; S13: Using the pseudo-random binary sequence, perform XOR encryption on the key syntax elements of the original data bitstream; S14: Repackage the encrypted original data bitstream to generate an encrypted compressed domain video.

3. The encrypted video watermark embedding method according to claim 1, characterized in that, In response to a user's video download request, a unique watermark sequence is generated, the video remains encrypted, the watermark is embedded into the encrypted video, and the watermarked encrypted video is sent to the user. The process is as follows: S21: In response to the user's video download request, generate a unique identifier based on the user's authorization information, use the identifier as a watermark sequence, and divide the watermark sequence into independent watermark subgroups; S22: Select the last P frame of each image group in the encrypted video as the watermark embedding frame. In the watermark embedding frame, the watermark embedding process is divided into several independent and serially executed embedding operations. Each embedding operation corresponds to a syntax element channel. In each syntax element channel, the watermark subgroup is cyclically embedded into the syntax element of each watermark embedding frame to obtain the watermarked encrypted video.

4. The encrypted video watermark embedding method according to claim 3, characterized in that, The syntax element channels include a trailing coefficient sign channel, a non-zero transform coefficient channel, and a motion vector difference channel; In each syntax element channel, the process of cyclically embedding the watermark subgroups into the syntax element of each watermark embedding frame to obtain the watermarked encrypted video is as follows: Group all watermarks and concatenate them cyclically to form a cyclic sequence; In the trailing coefficient symbol channel, the cyclic sequence is overwritten to the trailing coefficient symbol in the watermark embedding frame until all trailing coefficient symbols in the current watermark embedding frame have been overwritten and replaced. Then the next watermark embedding frame is processed until all watermark embedding frames have been processed. In the non-zero transform coefficient channel, the index of the cyclic sequence is reset. Based on the bits of the cyclic sequence, the parity of each codeword of the non-zero transform coefficient is modified according to the parity mapping rule until all non-zero transform coefficients of the current watermark embedding frame are modified. Then, the next watermark embedding frame is processed until all watermark embedding frames are processed. In the motion vector difference channel, the index of the cyclic sequence is reset. Based on the bits of the cyclic sequence, the parity of each codeword of the motion vector difference is modified according to the parity mapping rule until all motion vector differences of the current watermark embedding frame have been modified. Then, the next watermark embedding frame is processed until all watermark embedding frames have been processed.

5. A method for extracting watermarks from encrypted videos, characterized in that, Includes the following steps: The video to be detected is obtained, and multiple multidimensional sparse syntax feature tensors are constructed from the compressed domain based on multiple video frames with embedded watermarks. Based on the multiple multidimensional sparse syntax feature tensors, a pre-trained anti-recompression watermark extraction network is used to decode multiple video frames with embedded watermarks to obtain the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks. Multiple prediction results are spliced ​​together to extract the complete watermark.

6. The method for extracting encrypted video watermarks according to claim 5, characterized in that, To obtain the video to be detected, based on multiple video frames with embedded watermarks, construct multiple multidimensional sparse syntax feature tensors from the compressed domain. The process is as follows: Entropy decoding is performed on the video to be tested in the compressed domain; Based on the decoded video to be detected, multiple multidimensional sparse syntax feature tensors are constructed using macroblocks of video frames with embedded watermarks as basic units. The multidimensional sparse syntax feature tensor is a three-dimensional tensor of size H×W×C, where H and W are the frame height and frame width in macroblock units, respectively, and C is the number of feature channels, including: the original watermark extraction channel and the anti-attack statistical feature channel; the anti-attack statistical feature channel contains a noisy bitstream initially parsed from the trailing coefficient sign channel, the non-zero transform coefficient channel, and the motion vector difference channel, respectively, and the noisy bitstream contains a large number of bit errors; the anti-attack statistical feature channel explicitly aggregates the statistics used by the auxiliary network to judge the reliability of the macroblock, including: the total number of non-zero coefficients in the macroblock, the motion vector magnitude, the quantization parameter QP, and the number of bits in the macroblock.

7. The method for extracting encrypted video watermarks according to claim 5, characterized in that, The pre-trained anti-compression watermark extraction network includes a group of convolutional layers, a spatial attention module, a global average pooling layer, and a fully connected layer. Based on the aforementioned multidimensional sparse syntax feature tensors, a pre-trained anti-recompression watermark extraction network is used to decode multiple watermarked video frames to obtain the prediction result of the watermark bit sequence carried by each watermarked video frame. The process is as follows: C1: Utilize convolutional layers to extract macroblocks of multidimensional sparse grammar feature tensors and spatial correlation features of macroblock neighborhoods; C2: Based on spatial correlation features, different weight masks are assigned to macroblocks using the attention mechanism of the spatial attention module; C3: Uses a global average pooling layer to compress spatial correlation features into a global feature vector; C4: Based on the weight mask and global feature vector, using a fully connected layer and activation function, obtain the probability vector of the payload carried by the corresponding watermarked video frame; use the probability vector as the prediction result. C5: Repeat steps C1 to C4 until all video frames with embedded watermarks are decoded, resulting in multiple prediction results.

8. An encrypted video watermark embedding device, used to implement the encrypted video watermark embedding method according to any one of claims 1-4, characterized in that, Including the user end and the watermark embedding end; On the user side, the encrypted compressed video stream is parsed, and a selective encryption operation based on codewords is performed to generate an encrypted compressed domain video. The watermark embedding terminal is deployed on a cloud server. In response to the user's video download request, the watermark embedding terminal generates a unique watermark sequence, keeps the video encrypted and does not decrypt it, embeds the watermark into the encrypted video, and sends the watermarked encrypted video to the user.

9. An encrypted video watermark extraction device, used to implement the encrypted video watermark extraction method according to any one of claims 5-7, characterized in that, Includes an extraction terminal, which is deployed on a cloud server; At the extraction end, the video to be detected is obtained, and based on multiple video frames with embedded watermarks, multiple multidimensional sparse syntax feature tensors are constructed from the compressed domain respectively. The extraction end is equipped with a pre-trained anti-recompression watermark extraction network. The pre-trained anti-recompression watermark extraction network decodes multiple video frames with embedded watermarks based on the multiple multidimensional sparse syntax feature tensors, and obtains the prediction result of the watermark bit sequence carried by each video frame with embedded watermarks.

10. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the encrypted video watermark embedding method as described in any one of claims 1 to 4, or to execute the steps in the encrypted video watermark extraction method as described in any one of claims 5 to 7.

Images