A digital watermark embedding method and device, electronic equipment and storage medium

Abstract

This invention discloses a digital watermark embedding method, apparatus, electronic device, and storage medium. The method includes: utilizing the inter-frame reference relationship between video frames during video encoding to select a set of digital watermark embedding frames from the video sequence; then adjusting the digital watermark embedding strength and / or redundancy coding rate of the digital watermark embedding frames by constructing a digital watermark signal response model; and finally embedding the digital watermark signal in the embedding domain of the digital watermark embedding frames. The technical solution provided by this invention embeds the digital watermark only in a portion of the reference frames, enabling the digital watermark information to automatically propagate from the embedded frames to the non-embedded frames. This significantly reduces the computational complexity and time overhead of digital watermark embedding while achieving frame-by-frame detectability of the digital watermark, with overall robustness no less than or even better than traditional frame-by-frame embedding schemes.

Description

Technical Field

[0001] This invention relates to the fields of digital content security and video encoding processing technology, and in particular to a digital watermark embedding method, apparatus, electronic device, and storage medium. Background Technology

[0002] Digital watermarking technology is an important means of protecting multimedia content copyright and achieving content authentication and tracking. In video applications, to resist various attacks such as recompression, transcoding, and noise interference, traditional digital watermarking often repeatedly embeds the digital watermark in every frame or even every image block to improve robustness and detection stability. This approach results in high computational overhead and is easily limited in applications with limited computing resources (such as mobile devices and real-time encoding scenarios) or those sensitive to encoding latency. Although reducing the number of embedded frames can reduce overhead, it directly sacrifices the digital watermark detection capability of non-embedded frames, reducing overall robustness.

[0003] Meanwhile, mainstream video coding standards (such as H.264 / AVC, H.265 / HEVC, AV1) generally adopt a hybrid coding framework of block-based motion-compensated prediction and transform coding. Under this framework, the reconstructed content of the current frame heavily depends on its reference frame. The encoder uses motion estimation to find matching blocks in the reference frame, obtains prediction values ​​through motion compensation, and then encodes the residuals. Furthermore, loop filtering techniques (such as deblocking filtering, adaptive sample offset, and adaptive loop filtering) further smooth intra-frame and inter-frame block boundaries, allowing information from the reference frame (including any embedded watermarks) to propagate more smoothly into subsequent frames.

[0004] Based on this, the present invention designs a low-overhead, highly robust digital watermarking embedding method according to the prediction structure of modern video coding. Summary of the Invention

[0005] In view of the above-mentioned problems in the existing technology, the present invention proposes a digital watermark embedding method, device, electronic device and storage medium.

[0006] Specifically, the embodiments of the present invention provide the following technical solutions:

[0007] In a first aspect, embodiments of the present invention provide a digital watermark embedding method, comprising:

[0008] Based on the inter-frame reference information of the video frame sequence, a directed graph of inter-frame prediction relationships is constructed, and the digital watermark propagation gain is calculated.

[0009] The inter-frame reference information includes at least one of the following: group of pictures (GOP) structure information, reference frame list information, and reference frame set.

[0010] Based on the directed graph, a set of digital watermark embedded frames is selected from the video frame sequence.

[0011] Construct a digital watermark signal response model to predict the predicted digital watermark response of the target frame, and adjust the digital watermark embedding strength and / or redundancy coding rate of the embedded frame accordingly.

[0012] The digital watermark signal is embedded in the embedding domain of the digital watermark embedding frame.

[0013] Secondly, embodiments of the present invention provide a digital watermark embedding device, comprising:

[0014] The directed graph construction and propagation gain calculation module is used to construct a directed graph of inter-frame prediction relationships based on inter-frame reference information of video frame sequences and calculate the digital watermark propagation gain.

[0015] The inter-frame reference information includes at least one of the following: group of pictures (GOP) structure information, reference frame list information, and reference frame set.

[0016] An embedded frame selection module is used to filter out a set of digital watermark embedded frames from the video frame sequence based on the directed graph.

[0017] The propagation prediction and intensity shaping module is used to construct a digital watermark signal response model, predict the predicted digital watermark response of the target frame, and adjust the digital watermark embedding intensity and / or redundancy coding rate of the embedded frame accordingly.

[0018] The digital watermark embedding module is used to embed digital watermark signals in the embedding field of the digital watermark embedding frame.

[0019] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the digital watermark embedding method as described in the first aspect.

[0020] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the digital watermark embedding method as described in the first aspect.

[0021] As can be seen from the above technical solutions, the digital watermarking embedding method provided by the embodiments of the present invention utilizes the inter-frame prediction features of the encoding to embed digital watermarks only in a portion of video frames. By leveraging the inter-frame prediction and filtering mechanisms of video encoding, the digital watermark information is automatically propagated from the embedded frames to the non-embedded frames. This significantly reduces the computational complexity and time overhead of digital watermark embedding while achieving frame-by-frame detectability of the digital watermark, with overall robustness no less than or even better than traditional frame-by-frame embedding schemes. Furthermore, the technical method provided by the embodiments of the present invention is based on a general reference frame structure modeling, does not depend on a specific digital watermarking embedding algorithm, and can be adapted to various video encoding standards and core digital watermarking technologies, exhibiting wide adaptability. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a digital watermark embedding method provided in an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the structure of a digital watermark embedding device provided in an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solutions of the present invention more clearly, and should not be used to limit the scope of protection of the present invention.

[0027] Figure 1 A flowchart of a digital watermark embedding method according to an embodiment of the present invention is shown, as follows: Figure 1 As shown, the digital watermark embedding method provided in this embodiment of the invention specifically includes the following:

[0028] Step 1: Based on the inter-frame reference information of the video frame sequence, construct a directed graph G = (V,E) of the inter-frame prediction relationship and calculate the digital watermark propagation gain.

[0029] In the directed graph G = (V, E), V is a set of video frame sequences, u∈V and v∈V, and the directed edge (u, v) ∈E indicates that video frame v references video frame u during encoding.

[0030] In this step, the inter-frame reference information includes at least one of the following: group of pictures (GOP) structure information, reference frame list information, and reference frame set.

[0031] In one possible implementation, the calculation of the digital watermark propagation gain involves calculating the single-step propagation gain g(u→v) for each directed edge. This g represents the attenuation ratio of the digital watermark as it propagates directly from video frame u to video frame v. The calculation formula is as follows:

[0032] g(u→v) = α · R cov (u,v) · exp(-β · E mv (u,v)) · exp(-γ · |QP v - QP u |) · κ(filter)

[0033] Among them, R cov (u,v) represents the reference coverage, indicating the proportion of coded blocks in video frame v that reference video frame u for motion compensation, 0 ≤ R. cov ≤ 1; E mv (u,v) represents the motion vector energy, indicating the intensity of motion of video frame v relative to video frame u, usually expressed as the sum of squares or average amplitude of the motion vector magnitudes; |QP v - QP u | represents the absolute value of the difference in quantization parameters between video frame u and video frame v, characterizing the difference in quantization levels between the two video frames; κ(filter) is the loop filter retention coefficient, used to estimate the degree of retention of the digital watermark signal by operations such as deblocking filtering and sample adaptive offset, 0 ≤ κ≤ 1; α, β, γ are all model adjustment parameters not less than 0.

[0034] Furthermore, based on the inter-frame prediction characteristics, the propagation of the digital watermark from video frame u to video frame v actually includes both single-step propagation (such as direct propagation from video frame u to video frame v) and multi-step propagation (such as the digital watermark propagating from video frame u to video frame k, and then from video frame k to video frame v). Therefore, in order to more accurately calculate the total attenuation ratio of the digital watermark signal on all possible predicted paths of the digital watermark propagation from video frame u to video frame v, in another possible implementation, based on the single-step propagation gain g(u→v), the multi-step cumulative propagation gain p(u→v) of the digital watermark propagation from video frame u to video frame v is calculated, and the calculation formula is as follows:

[0035]

[0036] Where Ref(v) is the set of direct reference frames for video frame v, and k is the index of the intermediate reference frame when traversing the set.

[0037] For example, based on the reference characteristics between video frames, assume that there are two paths for the digital watermark to propagate from video frame 1 (u) to video frame 3 (v): 1. Video frame 1 (u) propagates to video frame 2 (v1), and then from video frame 2 (v1) to video frame 3 (v); 2. Video frame 1 (u) propagates directly to video frame 3 (v).

[0038] The single-step propagation gain is: g(u→v1)=0.6; g(v1→v)=0.5; g(u→v)=0.2.

[0039] The cumulative propagation of the digital watermark from video frame 1 to video frame 3 is... .

[0040] Step 2: Based on the directed graph, select a set of digital watermark embedded frames from the video frame sequence.

[0041] One possible implementation involves a simple empirical strategy, such as selecting one frame at fixed intervals to add to the digital watermark embedding frame set, or selecting only all I-frames / IDR frames to add to the digital watermark embedding frame set.

[0042] In another possible implementation, a maximum number of embedding frames is preset, and then video frames that meet the required number are randomly selected from the video frame sequence and added to the digital watermark embedding frame set.

[0043] In another possible implementation, for each video frame in the video frame sequence, the number of times it is referenced (including direct reference and / or indirect reference) in the directed graph is calculated, and then the top N video frames with the most references are selected and added to the digital watermark embedding frame set.

[0044] In another possible implementation, the reference frames corresponding to the top N directed edges with the largest propagation gain are selected as the embedded frames and added to the digital watermark embedded frame set.

[0045] In another possible implementation, the set of digital watermark embedding frames is solved using a mathematical optimization model. Specifically, this includes:

[0046] The optimization objective of the mathematical optimization model is defined as follows: under a preset embedding frame count budget B, select an optimal subset S (|S| ≤ B) from the video frame set V as the digital watermark embedding frames, thereby maximizing the digital watermark propagation effect of the video frame sequence. This propagation effect is characterized by the digital watermark propagation coverage, specifically:

[0047] Maximize the cumulative digital watermark coverage = .

[0048]

[0049] Where Agg(v; S) represents the maximum normalized digital watermark coverage that any video frame v can achieve under the embedding set S; s is the embedding frame in the embedding frame set S.

[0050] An approximate solution is obtained using a greedy algorithm:

[0051] (1) Initialize the selected set S to be empty.

[0052] (2) When |S| < B, perform the following iterations:

[0053] (2.1) For each candidate frame c ∈ V \ S, calculate its marginal propagation benefit, which quantifies the increase in the total global digital watermark coverage brought about by adding candidate frame c to the existing set S, specifically as follows.

[0054]

[0055] (2.2) Select the candidate frame with the largest marginal propagation benefit: .

[0056] (2.3) Update the set: .

[0057] (3) Output the final set of embedded frames S.

[0058] Step 3: Construct a digital watermark signal response model, predict the predicted digital watermark signal response of the target frame, and adjust the digital watermark embedding strength and / or redundancy coding rate of the embedded frame accordingly.

[0059] In one possible implementation, the construction of the digital watermark signal response model, the prediction of the predicted digital watermark signal response of the target frame, and the adjustment of the digital watermark embedding strength and / or redundancy coding rate of the embedded frame accordingly, include:

[0060] An attenuation model is constructed to simulate the attenuation of the digital watermark signal, with the digital watermark signal attenuating by a fixed proportion each time it passes through a prediction relationship.

[0061] The predicted digital watermark signal of the target frame is calculated based on the predicted link and attenuation ratio between the embedded frame and the target frame.

[0062] Based on the difference between the predicted digital watermark signal and the preset target digital watermark signal, the digital watermark embedding strength of the embedded frame and / or the redundancy coding rate are adjusted in reverse by an adjustment factor.

[0063] For example, one modeling and solving process is as follows:

[0064] (1) Construct a digital watermark signal attenuation model.

[0065] Set a global fixed decay factor per hop. (0 < < 1). For any embedded frame s ∈ S and target frame i ∈ I, the predicted digital watermark signal strength is calculated using the following formula:

[0066]

[0067] Where a is the digital watermark embedding strength of the embedded frame s; dist(s, i) is the shortest predicted hop count from the embedded frame s to the target frame i in the directed graph G.

[0068] In this step, it should be noted that, due to the exponential decay of the digital watermark signal along the prediction link, the contribution of the embedding frame with a higher prediction hop count to the target frame is much smaller than that of the embedding frame with the shortest prediction link (i.e., the fewest prediction hop count). Therefore, the superposition effect between multiple embedding frames can be ignored, and only the embedding frame with the shortest prediction link is considered as the main source of the digital watermark signal for each target frame.

[0069] (2) Calculate the difference between the preset target digital watermark embedding strength and the predicted digital watermark strength.

[0070] The preset target digital watermark embedding strength T, and / or, the redundancy coding budget ρ.

[0071] For target frame i, its digital watermark signal strength deviation is defined as: .

[0072] in, A positive value indicates insufficient digital watermark strength. A negative value indicates that the digital watermark strength is too strong.

[0073] (3) Adjust the digital watermark embedding strength and / or redundancy coding rate of the embedded frame by adjusting the adjustment factor. The specific calculation formula is as follows:

[0074]

[0075]

[0076] Where a represents the digital watermark embedding strength of the embedded frame s; r represents the redundancy coding rate of the embedded frame; μ a and μ r The default scaling factor is 0 < μ. a μ r <1; Cover(s) is the set of target frames whose main contribution source is the embedded frame s; the superscript (t) indicates the iteration number; a max This represents the preset maximum threshold for the embedding strength of the embedded frame; rmax This represents the preset maximum threshold for the embedded frame redundancy coding rate; The value represents the deviation of the target frame digital watermark signal strength; T represents the preset target digital watermark embedding strength.

[0077] In this step, it should be noted that after adjusting the digital watermark strength and / or redundancy coding rate, the maximum thresholds for embedding strength and redundancy coding rate need to be set to truncate them, ensuring that the embedding strength and redundancy coding rate do not exceed the boundary constraints.

[0078] (4) Repeat steps (1)-(3) above until one or more of the following conditions are met:

[0079] a. The absolute deviation between the predicted digital watermark signal strength and the target digital watermark embedding strength of all target frames is less than the preset threshold.

[0080] b. Reaching the maximum number of iterations t max .

[0081] c. The redundant coding rate increases close to the budget ρ.

[0082] Optionally, as another implementation, the construction of the digital watermark signal response model, predicting the predicted digital watermark signal response of the target frame, and adjusting the digital watermark embedding strength and / or redundancy coding rate of the embedded frame accordingly, includes: using visual distortion and bitrate growth as constraints, and maximizing the total power of the weighted digital watermark signal of the video frame sequence as the objective function, constructing a multi-objective joint optimization model and solving for the embedding strength and / or redundancy coding rate of the embedded frame.

[0083] For example, one modeling and solving process is as follows.

[0084] (1) Set the following multi-objective constraints:

[0085] (i) Visual distortion constraint: For each target frame i, the distortion D caused by the embedded digital watermark is _____. i Not exceeding the preset distortion threshold D max ,Right now:

[0086]

[0087]

[0088] Among them, s m Indicates the m-th embedded frame; a m d(s) represents the embedding strength of the m-th embedding frame; m →i) represents the distortion propagation coefficients from the m-th embedded frame to the target frame; D max p(s) represents the maximum perceptual distortion allowed per frame.m →i) represents the multi-step cumulative propagation gain of the digital watermark from the m-th embedded frame to the target frame; JND(i) is the just-perceptible distortion of the target frame.

[0089] (ii) Bitrate growth constraint: The overall bitrate growth ΔR caused by digital watermark embedding shall not exceed the preset budget ρ, that is:

[0090]

[0091] Where ζ and η represent the coefficients of embedding strength on code rate increase and redundant coding on code rate increase, respectively, and r m Let a represent the redundancy coding rate of the m-th embedded frame. m This represents the embedding strength of the m-th embedded frame; ρ represents the total bitrate growth budget, and S represents the set of embedded frames.

[0092] (iii) Variable boundary constraints: The embedding strength satisfies 0 ≤ a ≤ a max The redundancy rate satisfies 0 ≤ r ≤ r max .

[0093] (2) Construct a multi-objective joint optimization objective function:

[0094]

[0095]

[0096] Among them, w i H represents the detection weight coefficient of target frame i. i Indicates the total predicted digital watermark signal strength of the target frame; s m Indicates the m-th embedded frame; a m p(s) represents the embedding strength of the m-th embedded frame. m →i) represents the multi-step cumulative propagation gain of the digital watermark from the m-th embedded frame to the target frame i; S represents the set of embedded frames.

[0097] (3) Optimization and parameter determination: The multi-objective joint optimization function is solved using mathematical methods. The specific method is not limited, such as semidefinite programming (SDP) relaxation method, successive convex approximation (SCA) method, etc. After solving, the optimal embedding strength and optimal redundancy coding rate of each embedding frame are output.

[0098] Step 4: Embed the digital watermark signal in the embedding field of the digital watermark embedding frame.

[0099] In this step, the present invention does not limit the selection of the embedding domain or the embedding method, and the selection is made according to the actual situation.

[0100] In this step, regarding the selection of the embedding domain, one possible implementation is to convert the pixel data of the video frame to the transform domain and embed the digital watermark signal in the transform domain.

[0101] The transform domain includes, but is not limited to, the DCT (Discrete Cosine Transform) domain, the DST (Discrete Sine Transform) domain, the MDCT (Modified Discrete Cosine Transform) domain, and the WHT (Walsh-Hadamard Transform) domain.

[0102] In another possible implementation, after the video frame is predicted and encoded, a digital watermark signal is embedded in the residual domain between the original pixel value and the predicted pixel value.

[0103] In this step, the choice of embedding method can be implemented in one possible way: spread spectrum modulation embedding; in another possible way: fixed step size quantization index modulation embedding; and in yet another possible way: a preset lattice coding scheme can be used for embedding.

[0104] Figure 2 A digital watermark embedding device according to an embodiment of the present invention is shown, such as Figure 2 As shown, the digital watermark embedding device provided in this embodiment of the invention specifically includes the following components:

[0105] The directed graph construction and propagation gain calculation module is used to construct a directed graph G = (V, E) of inter-frame prediction relationships based on inter-frame reference information of video frame sequences, and calculate the digital watermark propagation gain.

[0106] In the directed graph G = (V, E), V is a set of video frame sequences, u∈V and v∈V, and the directed edge (u, v) ∈E indicates that video frame v references video frame u during encoding.

[0107] In this module, the inter-frame reference information includes at least one of the following: group of pictures (GOP) structure information, reference frame list information, and reference frame set.

[0108] In one possible implementation, the calculation of the digital watermark propagation gain involves calculating the single-step propagation gain g(u→v) for each directed edge. This g represents the attenuation ratio of the digital watermark as it propagates directly from video frame u to video frame v. The calculation formula is as follows:

[0109] g(u→v) = α · R cov(u,v) · exp(-β · E mv (u,v)) · exp(-γ · |QP v - QP u |) · κ(filter)

[0110] Among them, R cov (u,v) represents the reference coverage, indicating the proportion of coded blocks in video frame v that reference video frame u for motion compensation, 0 ≤ R. cov ≤ 1; E mv (u,v) represents the motion vector energy, indicating the intensity of motion of video frame v relative to video frame u, usually expressed as the sum of squares or average amplitude of the motion vector magnitudes; |QP v - QP u | represents the absolute value of the difference in quantization parameters between video frame u and video frame v, characterizing the difference in quantization levels between the two video frames; κ(filter) is the loop filter retention coefficient, used to estimate the degree of retention of the digital watermark signal by operations such as deblocking filtering and sample adaptive offset, 0 ≤ κ≤ 1; α, β, γ are all model adjustment parameters not less than 0.

[0111] Furthermore, based on the inter-frame prediction characteristics, the propagation of the digital watermark from video frame u to video frame v actually includes both single-step propagation (such as direct propagation from video frame u to video frame v) and multi-step propagation (such as the digital watermark propagating from video frame u to video frame k, and then from video frame k to video frame v). Therefore, in order to more accurately calculate the total attenuation ratio of the digital watermark signal on all possible predicted paths of the digital watermark propagation from video frame u to video frame v, in another possible implementation, based on the single-step propagation gain g(u→v), the multi-step cumulative propagation gain p(u→v) of the digital watermark propagation from video frame u to video frame v is calculated, and the calculation formula is as follows:

[0112]

[0113] Where Ref(v) is the set of direct reference frames for video frame v, and k is the index of the intermediate reference frame when traversing the set.

[0114] For example, based on the reference characteristics between video frames, assume that there are two paths for the digital watermark to propagate from video frame 1 (u) to video frame 3 (v): 1. Video frame 1 (u) propagates to video frame 2 (v1), and then from video frame 2 (v1) to video frame 3 (v); 2. Video frame 1 (u) propagates directly to video frame 3 (v).

[0115] The single-step propagation gain is: g(u→v1)=0.6; g(v1→v)=0.5; g(u→v)=0.2.

[0116] The cumulative propagation of the digital watermark from video frame 1 to video frame 3 is... .

[0117] An embedded frame selection module is used to filter out a set of digital watermark embedded frames from the video frame sequence based on the directed graph.

[0118] One possible implementation involves a simple empirical strategy, such as selecting one frame at fixed intervals to add to the digital watermark embedding frame set, or selecting only all I-frames / IDR frames to add to the digital watermark embedding frame set.

[0119] In another possible implementation, a maximum number of embedding frames is preset, and then video frames that meet the required number are randomly selected from the video frame sequence and added to the digital watermark embedding frame set.

[0120] In another possible implementation, for each video frame in the video frame sequence, the number of times it is referenced (including direct reference and / or indirect reference) in the directed graph is calculated, and then the top N video frames with the most references are selected and added to the digital watermark embedding frame set.

[0121] In another possible implementation, the reference frames corresponding to the top N directed edges with the largest propagation gain are selected as the embedded frames and added to the digital watermark embedded frame set.

[0122] In another possible implementation, the set of digital watermark embedding frames is solved using a mathematical optimization model. Specifically, this includes:

[0123] The optimization objective of the mathematical optimization model is defined as follows: under a preset embedding frame count budget B, select an optimal subset S (|S| ≤ B) from the video frame set V as the digital watermark embedding frames, thereby maximizing the digital watermark propagation effect of the video frame sequence. This propagation effect is characterized by the digital watermark propagation coverage, specifically:

[0124] Maximize the cumulative digital watermark coverage = .

[0125]

[0126] Where Agg(v; S) represents the maximum normalized digital watermark coverage that any video frame v can achieve under the embedding set S; s is the embedding frame in the embedding frame set S.

[0127] An approximate solution is obtained using a greedy algorithm:

[0128] (1) Initialize the selected set S to be empty.

[0129] (2) When |S| < B, perform the following iterations:

[0130] (2.1) For each candidate frame c ∈ V \ S, calculate its marginal propagation benefit, which quantifies the increase in the total global digital watermark coverage brought about by adding candidate frame c to the existing set S, specifically as follows.

[0131]

[0132] (2.2) Select the candidate frame with the largest marginal propagation benefit: .

[0133] (2.3) Update the set: .

[0134] (3) Output the final set of embedded frames S.

[0135] The propagation prediction and intensity shaping module is used to construct a digital watermark signal response model, predict the predicted digital watermark signal response of the target frame, and adjust the digital watermark embedding intensity and / or redundancy coding rate of the embedded frame accordingly.

[0136] In one possible implementation, the construction of the digital watermark signal response model, the prediction of the predicted digital watermark signal response of the target frame, and the adjustment of the digital watermark embedding strength and / or redundancy coding rate of the embedded frame accordingly, include:

[0137] An attenuation model is constructed to simulate the attenuation of the digital watermark signal, with the digital watermark signal attenuating by a fixed proportion each time it passes through a prediction relationship.

[0138] The predicted digital watermark signal of the target frame is calculated based on the predicted link and attenuation ratio between the embedded frame and the target frame.

[0139] Based on the difference between the predicted digital watermark signal and the preset target digital watermark signal, the digital watermark embedding strength of the embedded frame and / or the redundancy coding rate are adjusted in reverse by an adjustment factor.

[0140] For example, one modeling and solving process is as follows:

[0141] (1) Construct a digital watermark signal attenuation model.

[0142] Set a global fixed decay factor per hop. (0 < < 1). For any embedded frame s ∈ S and target frame i ∈ I, the predicted digital watermark signal strength is calculated using the following formula:

[0143]

[0144] Where a is the digital watermark embedding strength of the embedded frame s; dist(s, i) is the shortest predicted hop count from the embedded frame s to the target frame i in the directed graph G.

[0145] In this module, it should be noted that, due to the exponential decay of the digital watermark signal along the prediction link, the contribution of the embedding frame with a higher prediction hop count to the target frame is much smaller than that of the embedding frame with the shortest prediction link (i.e., the fewest prediction hop count). Therefore, the superposition effect between multiple embedding frames can be ignored, and only the embedding frame with the shortest prediction link is considered as the main source of the digital watermark signal for each target frame.

[0146] (2) Calculate the difference between the preset target digital watermark embedding strength and the predicted digital watermark strength.

[0147] The preset target digital watermark embedding strength T, and / or, the redundancy coding budget ρ.

[0148] For target frame i, its digital watermark signal strength deviation is defined as: .

[0149] in, A positive value indicates insufficient digital watermark strength. A negative value indicates that the digital watermark strength is too strong.

[0150] (3) Adjust the digital watermark embedding strength and / or redundancy coding rate of the embedded frame by adjusting the adjustment factor. The specific calculation formula is as follows:

[0151]

[0152]

[0153] Where a represents the digital watermark embedding strength of the embedded frame s; r represents the redundancy coding rate of the embedded frame; μ a and μ r The default scaling factor is 0 < μ. a μ r <1; Cover(s) is the set of target frames whose main contribution source is the embedded frame s; the superscript (t) indicates the iteration number; a max This represents the preset maximum threshold for the embedding strength of the embedded frame; r max This represents the preset maximum threshold for the embedded frame redundancy coding rate; The value represents the deviation of the target frame digital watermark signal strength; T represents the preset target digital watermark embedding strength.

[0154] In this module, it should be noted that after adjusting the digital watermark strength and / or redundancy coding rate, the maximum thresholds for embedding strength and redundancy coding rate need to be set to truncate them, ensuring that the embedding strength and redundancy coding rate do not exceed the boundary constraints.

[0155] (4) Repeat steps (1)-(3) above until one or more of the following conditions are met:

[0156] a. The absolute deviation between the predicted digital watermark signal strength and the target digital watermark embedding strength of all target frames is less than the preset threshold.

[0157] b. Reaching the maximum number of iterations t max .

[0158] c. The redundant coding rate increases close to the budget ρ.

[0159] Optionally, as another implementation, the construction of the digital watermark signal response model, predicting the predicted digital watermark signal response of the target frame, and adjusting the digital watermark embedding strength and / or redundancy coding rate of the embedded frame accordingly, includes: using visual distortion and bitrate growth as constraints, and maximizing the total power of the weighted digital watermark signal of the video frame sequence as the objective function, constructing a multi-objective joint optimization model and solving for the embedding strength and / or redundancy coding rate of the embedded frame.

[0160] For example, one modeling and solving process is as follows.

[0161] (1) Set the following multi-objective constraints:

[0162] (i) Visual distortion constraint: For each target frame i, the distortion D caused by the embedded digital watermark is _____. i Not exceeding the preset distortion threshold D max ,Right now:

[0163]

[0164]

[0165] Among them, s m Indicates the m-th embedded frame; a m d(s) represents the embedding strength of the m-th embedding frame; m →i) represents the distortion propagation coefficients from the m-th embedded frame to the target frame; D max p(s) represents the maximum perceptual distortion allowed per frame. m →i) represents the multi-step cumulative propagation gain of the digital watermark from the m-th embedded frame to the target frame; JND(i) is the just-perceptible distortion of the target frame.

[0166] (ii) Bitrate growth constraint: The overall bitrate growth ΔR caused by digital watermark embedding shall not exceed the preset budget ρ, that is:

[0167]

[0168] Where ζ and η represent the coefficients of embedding strength on code rate increase and redundant coding on code rate increase, respectively, and r m Let a represent the redundancy coding rate of the m-th embedded frame. m This represents the embedding strength of the m-th embedded frame; ρ represents the total bitrate growth budget, and S represents the set of embedded frames.

[0169] (iii) Variable boundary constraints: The embedding strength satisfies 0 ≤ a ≤ a max The redundancy rate satisfies 0 ≤ r ≤ r max .

[0170] (2) Construct a multi-objective joint optimization objective function:

[0171]

[0172]

[0173] Among them, w i H represents the detection weight coefficient of target frame i. i Indicates the total predicted digital watermark signal strength of the target frame; s m Indicates the m-th embedded frame; a m p(s) represents the embedding strength of the m-th embedded frame. m →i) represents the multi-step cumulative propagation gain of the digital watermark from the m-th embedded frame to the target frame i; S represents the set of embedded frames.

[0174] (3) Optimization and parameter determination: The multi-objective joint optimization function is solved using mathematical methods. The specific method is not limited, such as semidefinite programming (SDP) relaxation method, successive convex approximation (SCA) method, etc. After solving, the optimal embedding strength and optimal redundancy coding rate of each embedding frame are output.

[0175] The digital watermark embedding module is used to embed digital watermark signals in the embedding field of the digital watermark embedding frame.

[0176] In this module, the embodiments of the present invention do not limit the selection of the embedding field or the embedding method, and the selection is made according to the actual situation.

[0177] In this module, one possible implementation of the choice of embedding domain is to convert the pixel data of the video frame to the transform domain and embed the digital watermark signal in the transform domain.

[0178] The transform domain includes, but is not limited to, the DCT (Discrete Cosine Transform) domain, the DST (Discrete Sine Transform) domain, the MDCT (Modified Discrete Cosine Transform) domain, and the WHT (Walsh-Hadamard Transform) domain.

[0179] In another possible implementation, after the video frame is predicted and encoded, a digital watermark signal is embedded in the residual domain between the original pixel value and the predicted pixel value.

[0180] In this module, the choice of embedding method can be implemented in one possible way: spread spectrum modulation embedding; in another possible way: quantization index modulation embedding with a fixed step size; and in yet another possible way: embedding using a preset lattice coding scheme.

[0181] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, as shown in FIG3. The electronic device specifically includes the following components: a processor 301, a memory 302, a communication interface 303, and a communication bus 304.

[0182] The processor 301, memory 302, and communication interface 303 communicate with each other through the communication bus 304; the communication interface 303 is used to realize information transmission between the devices.

[0183] The processor 301 is used to call the computer program in the memory 302, and when the processor executes the computer program, it implements all the steps of the above-described digital watermark embedding method.

[0184] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements all the steps of the above-described digital watermark embedding method.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A digital watermark embedding method, characterized in that, include: Based on the inter-frame reference information of the video frame sequence, a directed graph of inter-frame prediction relationships is constructed, and the digital watermark propagation gain is calculated. Based on the directed graph, a set of digital watermark embedded frames is selected from the video frame sequence; Construct a digital watermark signal response model to predict the predicted digital watermark response of the target frame, and adjust the digital watermark embedding strength and / or redundancy coding rate of the digital watermark embedding frame accordingly. The digital watermark signal is embedded in the embedding domain of the digital watermark embedding frame.

2. The digital watermark embedding method according to claim 1, characterized in that, The directed graph includes nodes and directed edges; The nodes are used to represent each video frame in the video frame sequence; The directed edges are used to represent the reference relationship between video frames. If the second video frame references the first video frame during encoding, there exists a directed edge from the node representing the first video frame to the node representing the second video frame.

3. The digital watermark embedding method according to claim 1, characterized in that, The calculation of the digital watermark propagation gain can be implemented using any of the following methods: Method 1: Calculate the single-step propagation gain; Method 2: Based on the calculation of the single-step propagation gain, further obtain the multi-step cumulative propagation gain; The calculation of the single-step propagation gain considers at least one of the following factors: reference coverage, motion vector energy, inter-frame quantization parameter difference, and loop filter retention coefficient; wherein, the reference coverage is the proportion of coded blocks in the current frame that reference the reference frame for motion compensation; the motion vector energy is the degree of motion intensity of the current frame relative to the reference frame; the inter-frame quantization parameter difference is the absolute value of the difference between the quantization parameters of the reference frame and the current frame; and the loop filter retention coefficient is the degree to which the filtering operation preserves the digital watermark signal.

4. The digital watermark embedding method according to claim 1, characterized in that, The method of selecting the set of digital watermark embedded frames from the video frame sequence based on the directed graph includes at least one of the following: Embedded frames are selected according to preset empirical rules; The step of selecting embedded frames according to preset empirical rules includes at least one of the following: selecting one frame as an embedded frame every fixed number of frames, or selecting only all I-frames or IDR frames as embedded frames. Randomly select a preset number of embedded frames; Based on the directed graph, the top N reference frames that are referenced the most are selected as the set of embedded frames; Based on propagation gain, the top N video frames with the largest propagation gain are selected as the embedding set; The optimal set of embedded frames is determined by using a mathematical optimization model.

5. The digital watermark embedding method according to claim 4, characterized in that, The step of solving the optimal set of embedded frames through a mathematical optimization model includes: under a preset budget for the number of embedded frames, taking the maximization of the digital watermark propagation effect of the video frame sequence as the optimization objective, iteratively selecting candidate frames with the largest marginal propagation benefit through a greedy algorithm to add to the set of embedded frames until the budget for the number of embedded frames is reached.

6. The digital watermark embedding method according to claim 1, characterized in that, The construction of the digital watermark signal response model, the prediction of the predicted digital watermark response of the target frame, and the adjustment of the digital watermark embedding strength and / or redundancy coding rate of the digital watermark embedding frame accordingly, include: An attenuation model is constructed to simulate the attenuation of the digital watermark signal, with the digital watermark signal attenuating by a fixed proportion each time it passes through a prediction relationship. The predicted digital watermark signal of the target frame is calculated based on the predicted link and attenuation ratio between the embedded frame and the target frame. Based on the difference between the predicted digital watermark signal and the preset target digital watermark signal, the digital watermark embedding strength of the embedded frame and / or the redundancy coding rate are adjusted in reverse by an adjustment factor.

7. The digital watermark embedding method according to claim 1, characterized in that, The step of constructing a digital watermark signal response model, predicting the predicted digital watermark response of the target frame, and adjusting the digital watermark embedding strength and / or redundancy coding rate of the digital watermark embedding frame accordingly, further includes: Under the conditions of visual distortion constraint, bitrate growth constraint and variable boundary constraint, a multi-objective joint optimization model is constructed and the embedding strength and / or redundancy coding rate of the embedded frame are obtained by solving the model. The objective of the multi-objective joint optimization model is to maximize the total power of the weighted digital watermark signal of the video frame sequence.

8. A digital watermark embedding device, characterized in that, include: The directed graph construction and propagation gain calculation module is used to construct a directed graph of inter-frame prediction relationships based on inter-frame reference information of video frame sequences and calculate the digital watermark propagation gain. An embedded frame selection module is used to filter out a set of digital watermark embedded frames from the video frame sequence based on the directed graph. The propagation prediction and intensity shaping module is used to construct a digital watermark signal response model, predict the predicted digital watermark response of the target frame, and adjust the digital watermark embedding intensity and / or redundancy coding rate of the digital watermark embedding frame accordingly. The digital watermark embedding module is used to embed digital watermark signals in the embedding field of the digital watermark embedding frame.

9. The digital watermark embedding device according to claim 8, characterized in that, The directed graph includes nodes and directed edges; The nodes are used to represent each video frame in the video frame sequence; The directed edges are used to represent the reference relationship between video frames. If the second video frame references the first video frame during encoding, there exists a directed edge from the node representing the first video frame to the node representing the second video frame.

10. The digital watermark embedding device according to claim 8, characterized in that, The calculation of the digital watermark propagation gain can be implemented using any of the following methods: Method 1: Calculate the single-step propagation gain; Method 2: Based on the calculation of the single-step propagation gain, further obtain the multi-step cumulative propagation gain; The calculation of the single-step propagation gain considers at least one of the following factors: reference coverage, motion vector energy, inter-frame quantization parameter difference, and loop filter retention coefficient; wherein, the reference coverage is the proportion of coded blocks in the current frame that reference the reference frame for motion compensation; the motion vector energy is the degree of motion intensity of the current frame relative to the reference frame; the inter-frame quantization parameter difference is the absolute value of the difference between the quantization parameters of the reference frame and the current frame; and the loop filter retention coefficient is the degree to which the filtering operation preserves the digital watermark signal.

11. The digital watermark embedding device according to claim 8, characterized in that, The method of selecting the set of digital watermark embedded frames from the video frame sequence based on the directed graph includes at least one of the following: Embedded frames are selected according to preset empirical rules; The step of selecting embedded frames according to preset empirical rules includes at least one of the following: selecting one frame as an embedded frame every fixed number of frames, or selecting only all I-frames or IDR frames as embedded frames. Randomly select a preset number of embedded frames; Based on the directed graph, the top N reference frames that are referenced the most are selected as the set of embedded frames; Based on propagation gain, the top N video frames with the largest propagation gain are selected as the embedding set; The optimal set of embedded frames is determined by using a mathematical optimization model.

12. The digital watermark embedding device according to claim 11, characterized in that, The step of solving the optimal set of embedded frames through a mathematical optimization model includes: under a preset budget for the number of embedded frames, taking the maximization of the digital watermark propagation effect of the video frame sequence as the optimization objective, iteratively selecting candidate frames with the largest marginal propagation benefit through a greedy algorithm to add to the set of embedded frames until the budget for the number of embedded frames is reached.

13. The digital watermark embedding device according to claim 8, characterized in that, The construction of the digital watermark signal response model, the prediction of the predicted digital watermark response of the target frame, and the adjustment of the digital watermark embedding strength and / or redundancy coding rate of the digital watermark embedding frame accordingly, include: An attenuation model is constructed to simulate the attenuation of the digital watermark signal, with the digital watermark signal attenuating by a fixed proportion each time it passes through a prediction relationship. The predicted digital watermark signal of the target frame is calculated based on the predicted link and attenuation ratio between the embedded frame and the target frame. Based on the difference between the predicted digital watermark signal and the preset target digital watermark signal, the digital watermark embedding strength of the embedded frame and / or the redundancy coding rate are adjusted in reverse by an adjustment factor.

14. The digital watermark embedding device according to claim 8, characterized in that, The step of constructing a digital watermark signal response model, predicting the predicted digital watermark response of the target frame, and adjusting the digital watermark embedding strength and / or redundancy coding rate of the digital watermark embedding frame accordingly, further includes: Under the conditions of visual distortion constraint, bitrate growth constraint and variable boundary constraint, a multi-objective joint optimization model is constructed and the embedding strength and / or redundancy coding rate of the embedded frame are obtained by solving the model. The objective of the multi-objective joint optimization model is to maximize the total power of the weighted digital watermark signal of the video frame sequence.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the digital watermark embedding method as described in any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the digital watermark embedding method as described in any one of claims 1 to 7.

Images