Digital watermarking for digital image protection and manifest swapping detection

Abstract

The present disclosure relates generally to image signal processing including digital watermarking. Digital watermarking can be used to detect manifest swapping in, e.g., C2PA protected content. Various combinations and technology are described in this patent document and claims.

Description

[0001] Digital Watermarking for Digital Image Protection and Manifest Swapping Detection

[0002] Related Application Data

[0003] This application claims the benefit of US Provisional Patent Application Nos. 63 / 520,011, filed August 16, 2023, and 63 / 582,478, filed September 13, 2023. This application is also related to assignee’s US Patent Application No. 17 / 992,823, filed November 22, 2022, and PCT Application Nos. PCT / US 24 / 19423, filed March 11, 2024, PCT / US23 / 81649, filed November 29, 2023, and PCT / US22 / 50767, filed November 22, 2022 (published as WO 2023 / 096924). Each of the above patent documents is hereby incorporated herein by reference in its entirety.

[0004] Technical Field

[0005] The disclosed technology relates generally to complex signal processing including digital watermarking, digital image / video / audio authentication and manifest verification.

[0006] Background and Summary

[0007] Disinformation is a rising concern. Just look to the last few United States National elections where fake digital images spread like wildfire on a parched prairie plane. Standards bodies have emerged to combat this problem. For example, the Coalition for Content Provenance and Authenticity (C2PA) is a cross-industry initiative aimed at addressing the pervasive challenges posed by misleading content and misinformation in the digital age. By developing technical standards for content attribution and provenance, C2PA seeks to enable the public to have greater trust in the media they encounter and to provide a clearer understanding of the origin and authenticity of digital content. The coalition works toward creating a unified framework that promotes transparency, aids content creators in getting due credit, and assists users in discerning factual content from fabrications. The framework associates digital content (or “digital assets” such as images, audio, video, PDFs, text, 3D models, artwork, etc.) with so-called manifests. A C2PA manifest is a standardized set of metadata associated with digital content that provides information about its origin, authenticity, and modifications, enabling users to verify the content's provenance and integrity.

[0008] One potential hole in this framework is reliable detection of manifest swapping. That is, swapping an original manifest for a different one; or swapping data within a manifest for different data. Digital watermarking provides solutions for such. Digital watermark can be used to identify and retrieve an original manifest, thus creating a stronger link between the manifest and the digital asset. The original manifest can be consulted to identify whether, e.g., an author has changed since the watermark was embedded, for example (any assertions about edits, rights, and ownership information, or any other metadata can be checked for changes since the watermark was embedded). With this original manifest and the digital asset that we are checking, we can then determine and provide more information about changes in the image since it was watermarked. With a digital watermark, a system can then:

[0009] • Return a correct manifest;

[0010] • Return a hard-binding that existed prior to digital watermarking; and provide a hard-binding between an image and its manifest after digital watermarking applied;

[0011] • Paradoxically, the hard binding of a C2PA spec without a digital watermark does not give you this because of the possibility of manifest swapping; and

[0012] • Create a new hard binding bi-directionality: manifest to image, and image to manifest after digital watermarking.

[0013] For purposes of this disclosure, the terms “digital watermark,” “watermark” and “data hiding” are used interchangeably. (In contrast, the term “visual watermark” means an overt mark or logo superimposed onto an image, video, or other media.). We sometimes use the terms “embedding,” “embed,” “encoding,” “encode” and data hiding” to interchangeably mean modulating or transforming data representing digital asset to include information therein. For example, data hiding may seek to hide or embed an information signal (e.g., a plural bit payload or a modified version of such, e.g., a 2-D error corrected, spread spectrum signal) in a host signal. This can be accomplished, e.g., by modulating a host signal (e.g., representing digital content) in some fashion to carry the information signal. We sometimes use the terms “encoder” and “embedder” to interchangeably means software, circuitry, an apparatus and / or module to modulate or transform data representing digital content to include information therein. Similarly, we sometimes use the terms “decode,” “detect” and “read” (and various forms thereof) to interchangeably mean analyzing content to obtain a payload or signal element embedded or encoded therein. Similarly, we sometimes use the terms “decoder,” “detector” and “reader” to interchangeably means software, circuitry, apparatus and / or module to analyze content to obtain a payload or signal element embedded or encoded therein.

[0014] Digimarc Corporation headquartered in Beaverton, Oregon, USA, is a leader in the field of digital watermarking. Some of Digimarc ’s work in data hiding and digital watermarking is reflected, e.g., in U.S. Patent Nos.: 11,410,262; 11,410,261; 11,233,918; 11,188,996; 11,188,996; 11,062,108; 10,652,422; 10,453,163; 10,282,801; 6,947,571; 6,912,295; 6,891,959. 6,763,123; 6,718,046; 6,614,914; 6,590,996; 6,408,082; 6,122,403 and 5,862,260, and in published US Patent Application Nos. 20210110505, 20220207642 and 20220385783; and in published PCT specifications nos. WO2016153911; WO 2021 / 072346; and WO2020186234. Each of these patent documents is hereby incorporated by reference herein in its entirety. Of course, a great many other approaches are familiar to those skilled in the art. The artisan is presumed to be familiar with a full range of literature concerning steganography, data hiding and digital watermarking.

[0015] Additional aspects, features, combinations, and advantages will be readily apparent with reference to the following figures and the Detailed Description.

[0016] Brief Description of the Drawings

[0017] Fig. 1 is a block diagram of a signal encoder for encoding a data signal into host digital content.

[0018] Fig. 2 is a block diagram of a signal decoder for extracting a data signal from host digital content. Fig. 3 is a flow diagram illustrating operations of a signal generator.

[0019] FIG. 4A-8B are diagrams illustrating functionality and graphical user interfaces associated with an online verification system.

[0020] FIG. 9 is a block diagram of a digital asset validation system.

[0021] FIG. 10 is a flow diagram for a digital asset validation process.

[0022] FIGS. 11 and 12 show various manifest validation processes.

[0023] FIG. 13 is a trusted identity issuance system.

[0024] FIGS. 14-17 illustrate Graphical User Interfaces (GUIs) that allow users to link to a wallet and sign digital assets.

[0025] Detailed Description

[0026] There are two (2) main sections that follow in this Detailed Description (I. Signal Encoding and Decoding, and II. Digital Watermarking for Digital Image, Video and Audio Protection and Manifest Swapping Detection). These sections and their assigned headings are provided merely to help organize the Detailed Description. Of course, description and implementations under one such section are intended to be combined and implemented with description and implementations from the other such section headings. Thus, the section and headings in this document should not be interpreted as limiting the scope of the description.

[0027] I. Signal Encoding and Decoding

[0028] Fig. 1 is a block diagram of a signal encoder for encoding a signal within digital content (e.g., a digital asset such as a digital image, digital artwork, digital 3D models, digital photographs, PDFs, text documents, digital graphics, or designs). We sometimes refer to the signal as an “encoded signal,” “embedded signal” or “digital watermark signal”. We use the term “signal embedder” interchangeably with “signal encoder.” More generally, we use the terms “encoder” and “embedder” interchangeably. One example of a signal encoder or a signal embedder is a “digital watermark embedder” or “digital watermark encoder”. Fig. 2 is a block diagram of a compatible signal decoder for extracting a payload from a signal encoded within the digital content. We use the terms “read,” “detect,” and “decode” interchangeably. Similarly, we use the terms “decoder,” “reader” and “detector” interchangeably.

[0029] Encoding and decoding is typically applied digitally. For example, the encoder generates an output including an embedded signal that can be converted to a rendered form, such as viewable digital content, PDF, displayed image or video, or other viewable digital form. Prior to decoding, and if in an analog form, a decoding device obtains an image or stream of images and converts (if in analog form) it to an electronic signal, which is digitized and processed by signal decoding modules.

[0030] Inputs to the signal encoder include a host signal 150 and auxiliary data 152. The host signal in this context can be the target digital content. The objectives of the encoder include encoding a robust signal with desired capacity per unit of host signal, while maintaining perceptual quality within a human perceptual quality constraint. Human perceptual quality refers to the extent to which a modification of content is perceptible to a human viewer or listener, as determined based on a human perceptual model. In some cases, there may be very little variability or presence of a host signal, in which case, there is little host interference, on the one hand, yet little host content in which to mask the presence of the data channel visually. Some examples include a region of digital content that is devoid of much pixel variability (e.g., a single, uniform color).

[0031] The auxiliary data 152 includes the variable data information (e.g., payload) to be conveyed in the data channel, possibly along with other protocol data used to facilitate the communication.

[0032] The protocol defines the manner in which the signal is structured and encoded for robustness, perceptual quality, or data capacity. For any given application, there may be a single protocol, or more than one protocol. Examples of multiple protocols include cases where there are different versions of the channel, different channel types (e.g., several signal layers within a host signal). Different protocol versions may employ different robustness encoding techniques or different data capacity. Protocol selector module 154 determines the protocol to be used by the encoder for generating a data signal. It may be programmed to employ a particular protocol depending on the input variables, such as user control, application specific parameters, or derivation based on analysis of the host signal.

[0033] Perceptual analyzer module 156 analyzes the input host signal to determine parameters for controlling signal generation and embedding, as appropriate. It is not necessary in certain applications, while in others it may be used to select a protocol and / or modify signal generation and embedding operations. For example, when encoding in a host signal that will be printed or displayed, the perceptual analyzer 156 may be used to ascertain color content and masking capability of the host digital content.

[0034] The embedded signal may be included in one of the layers or channels of the digital content, e.g., corresponding to:

[0035] • one or more color channels of the digital content, e.g., Red, Green, Blue (RGB);

[0036] • Luminance, Chrominance, or in a CIELAB channel (L*, a*, b*);

[0037] • YUV channel;

[0038] • components of a color model (Lab, HSV, HSL, etc.);

[0039] • channels corresponding to Cyan, Magenta, Yellow and / or Black, a spot color layer (e.g., corresponding to a Pantone color), which are specified to be used to print the digital content;

[0040] • audio samples;

[0041] • a coating (e.g., varnish, UV layer, lacquer, sealant, extender, primer, etc.);

[0042] • other material layer (metallic substance, e.g., metallic ink or stamped foil where the embedded signal is formed by stamping holes in the foil or removing foil to leave dots of foil); etc.

[0043] The above are typically specified in a digital content file and are manipulated by an encoder. For example, an encoder is implemented as software modules of a plug-in to Adobe Photoshop or Illustrator processing software. Such software can be specified in terms of image layers or image channels. The encoder may modify existing layers, channels or insert new ones. A plug-in can be utilized with other image or audio processing software, e.g., for Adobe Illustrator.

[0044] The perceptual analysis performed in the encoder depends on a variety of factors, including color or colors of the embedded signal, resolution of the encoded signal, dot structure and screen angle used to print image layer(s) with the encoded signal, content within the layer of the encoded signal, content within layers under and over the encoded signal, etc. The perceptual analysis may lead to the selection of a color or combination of colors in which to encode the signal that minimizes visual differences due to inserting the embedded signal in an ink layer or layers within the digital content. This selection may vary per embedding location of each signal element. Likewise, the amount of signal at each location may also vary to control visual quality. The encoder can, depending on the associated print technology in which it is employed, vary embedded signal by controlling parameters such as:

[0045] • dot shape,

[0046] • signal amplitude at a dot,

[0047] • ink quantity at a dot (e.g., dilute the ink concentration to reduce percentage of ink),

[0048] • structure and arrangement of dot cluster or “bump” shape at a location of a signal element or region of elements. An arrangement of ink applied to x by y two-dimensional array of neighboring locations can be used to form a “bump” of varying shape or signal amplitude, as explained further below.

[0049] The ability to control printed dot size and shape is a particularly challenging issue and varies with print technology. Dot size can vary due to an effect referred to as dot gain. The ability of a printer to reliably reproduce dots below a particular size is also a constraint.

[0050] The encoded signal may also be adapted according to a blend model which indicates the effects of blending the ink of the signal layer with other layers and the substrate. In some cases, a designer may specify that the encoded signal be inserted into a particular layer. In other cases, the encoder may select the layer or layers in which it is encoded to achieve desired robustness and visibility (visual quality of the digital content in which it is inserted).

[0051] The output of this analysis, along with the rendering method (display or printing device) and rendered output form (e.g., ink and substrate) may be used to specify encoding channels (e.g., one or more color channels), perceptual models, and signal protocols to be used with those channels. Please see, e.g., the work on visibility and color models used in perceptual analysis in US Application Nos. 14 / 616,686 (US Patent No. 9,380,186), 14 / 588,636 (US Patent No. 9,401,001) and 13 / 975,919 (US Patent No. 9,449,357), Patent Application Publication 20100150434 (now US Patent No. 9,449,357), and US Patent 7,352,878, which are each hereby incorporated by reference in its entirety.

[0052] The signal generator module 158 operates on the auxiliary data and generates a data signal according to the protocol. It may also employ information derived from the host signal, such as that provided by perceptual analyzer module 156, to generate the signal. For example, the selection of data code signal and pattern, the modulation function, and the amount of signal to apply at a given embedding location may be adapted depending on the perceptual analysis, and in particular on the perceptual model and perceptual mask that it generates. The signal encoder may also comprise one or models, such as encoder, decoder, and generative adversarial network models trained using machine learning. The encoder may employ models, such as neural networks (e.g., convolutional neural networks) trained using adversarial machine learning to optimize perceptual quality and watermark robustness. Please see below and the incorporated patent documents for additional aspects of this process.

[0053] Embedder module 160 takes the data signal and modulates it onto a channel by combining it with the host signal. The host signal may include imagery (e.g., digital image or video) and / or audio. The operation of combining may be an entirely digital signal processing operation, such as where the data signal modulates the host signal digitally, may be a mixed digital and analog process or may be purely an analog process (e.g., where rendered output layers are combined). As noted, an encoded signal may occupy a separate layer or channel of the digital content file. This layer or channel may get combined into an image in the Raster Image Processor (RIP) prior to printing or may be combined as the layer is printed under or over other image layers on a substrate. If video or audio, an encoded layer may be combined with the video or audio during or before rendering of same.

[0054] There are a variety of different functions for combining the data and host in digital operations. One approach is to adjust the host signal value as a function of the corresponding data signal value at an embedding location, which is controlled according to the perceptual model and a robustness model for that embedding location. The adjustment may alter the host channel by adding a scaled data signal or multiplying a host value by a scale factor dictated by the data signal value corresponding to the embedding location, with weights or thresholds set on the amount of the adjustment according to perceptual model, robustness model, available dynamic range, and available adjustments to elemental ink structures (e.g., controlling halftone dot structures generated by the RIP). Weights may be distributed, e.g., unevenly, between different color channels (RGB) of digital content. The adjustment may also be altering by setting or quantizing the value of a pixel to particular signal element value.

[0055] As detailed further below, the signal generator produces a data signal with data elements that are mapped to embedding locations in the data channel. These data elements are modulated onto the channel at the embedding locations. Again, please see the documents incorporated herein for more information on variations.

[0056] The operation of combining a signal with other digital content may include one or more iterations of adjustments to optimize the modulated host for perceptual quality or robustness constraints. One approach, for example, is to modulate the host so that it satisfies a perceptual quality metric as determined by perceptual model (e.g., visibility model) for embedding locations across the signal. Another approach is to modulate the host so that it satisfies a robustness metric across the signal. Yet another is to modulate the host according to both the robustness metric and perceptual quality metric derived for each embedding location. The incorporated documents provide examples of these techniques. Below, we highlight a few examples.

[0057] For digital content including color images or color elements, the perceptual analyzer generates a perceptual model that evaluates visibility of an adjustment to the host by the embedder and sets levels of controls to govern the adjustment (c.g., levels of adjustment per color direction, and per masking region). This may include evaluating the visibility of adjustments of the color at an embedding location (e.g., units of noticeable perceptual difference in color direction in terms of CIE Lab values), Contrast Sensitivity Function (CSF), spatial masking model (e.g., using techniques described by Watson in US Published Patent Application No. US 2006-0165311 Al, which is incorporated by reference herein in its entirety), etc. One way to approach the constraints per embedding location is to combine the data with the host at embedding locations and then analyze the difference between the encoded host with the original. The rendering process may be modeled digitally to produce a modeled version of the embedded signal as it will appear when rendered. The perceptual model then specifies whether an adjustment is noticeable based on the difference between a visibility threshold function computed for an embedding location and the change due to embedding at that location. The embedder then can change or limit the amount of adjustment per embedding location to satisfy the visibility threshold function. Of course, there are various ways to compute adjustments that satisfy a visibility threshold, with different sequences of operations. See, e.g., US Patent Nos. 7,352,878, 9,380,186, 9,401,001, 9,449,357, and US Patent Application Publication 20100150434.

[0058] The embedder also computes a robustness model in some embodiments. The computing a robustness model may include computing a detection metric for an embedding location or region of locations. The approach is to model how well the decoder will be able to recover the data signal at the location or region. This may include applying one or more decode operations and measurements of the decoded signal to determine how strong or reliable the extracted signal. Reliability and strength may be measured by comparing the extracted signal with the known data signal. Below, we detail several decode operations that are candidates for detection metrics within the embedder. One example is an extraction filter which exploits a differential relationship between a signal element and neighboring content to recover the data signal in the presence of noise and host signal interference. At this stage of encoding, the host interference is derivable by applying an extraction filter to the modulated host. The extraction filter models data signal extraction from the modulated host and assesses whether a detection metric is sufficient for reliable decoding. If not, the signal may be re-inserted with different embedding parameters so that the detection metric is satisfied for each region within the host digital content where the signal is applied.

[0059] Detection metrics may be evaluated such as by measuring signal strength as a measure of correlation between the modulated host and variable or fixed data components in regions of the host or measuring strength as a measure of correlation between output of an extraction filter and variable or fixed data components. Depending on the strength measure at a location or region, the embedder changes the amount and location of host signal alteration to improve the correlation measure. These changes may be particularly tailored so as to establish sufficient detection metrics for both the payload and synchronization components of the embedded signal within a particular region of the host digital content.

[0060] The robustness model may also model distortion expected to be incurred by the modulated host, apply the distortion to the modulated host, and repeat the above process of measuring visibility and detection metrics and adjusting the number of alterations so that the data signal will withstand the distortion. See, e.g., US Patent Nos. 9,380,186, 9,401 ,001 and 9,449,357 for image related processing; each of these patent documents is hereby incorporated herein by reference.

[0061] As noted, the signal encoder may comprise one or more trained network models (e.g., deep learning models utilizing convolutional neural networks (CNNs) and / or recurrent neural networks (RNNs)) optimize the embedding of a variable watermark payload in the host signal for robustness to attacks and perceptual quality. These trained network models are employed within the signal encoder to produce the modulated host, carrying the auxiliary data. The digital watermarking may occur as the digital asset is generated. For example, a payload can be inserted into a digital asset during Al asset generation. Machine trained encoders are further discussed, e.g., in assignee’s US Patent Nos. 11,704,765 and 11,625,805, and in assignee’s US Published Application Nos. 20220270199 and 20210357690, each of which is hereby incorporated herein in its entirety.

[0062] This modulated host is then output as an output signal 162, with an embedded data channel. The operation of combining also may occur in the analog realm where the data signal is transformed to a rendered form, such as a layer of ink, including an overprint or under print, or a stamped, etched, or engraved surface marking. In the case of video display, one example is a data signal that is combined as a graphic overlay to other video content on a video display by a display driver. Another example is a data signal that is overprinted as a layer of material, engraved in, or etched onto a substrate, where it may be mixed with other signals applied to the substrate by similar or other marking methods. In these cases, the embedder employs a predictive model of distortion and host signal interference and adjusts the data signal strength so that it will be recovered more reliably. The predictive modeling can be executed by a classifier that classifies types of noise sources or classes of host signals and adapts signal strength and configuration of the data pattern to be more reliable to the classes of noise sources and host signals.

[0063] The output 162 from the embedder signal typically incurs various forms of distortion through its distribution or use. This distortion is what necessitates robust encoding and complementary decoding operations to recover the data reliably.

[0064] Turning to Fig. 2, a signal decoder receives a suspect host signal 200 and operates on it with one or more processing stages to detect a data signal, synchronize it, and extract data. The detector is paired with input device in which a sensor or other form of signal receiver captures an analog form of the signal and an analog to digital converter converts it to a digital form for digital signal processing. Though aspects of the detector may be implemented as analog components, e.g., such as preprocessing filters that seek to isolate or amplify the data channel relative to noise, much of the signal decoder is implemented as digital signal processing modules.

[0065] The detector 202 is a module that detects presence of the embedded signal and other signaling layers. The incoming digital content is referred to as a suspect host because it may not have a data channel or may be so distorted as to render the data channel undetectable. The detector is in communication with a protocol selector 204 to get the protocols it uses to detect the data channel. It may be configured to detect multiple protocols, either by detecting a protocol in the suspect signal and / or inferring the protocol based on attributes of the host signal or other sensed context information. A portion of the data signal may have the purpose of indicating the protocol of another portion of the data signal. As such, the detector is shown as providing a protocol indicator signal back to the protocol selector 204.

[0066] The synchronizer module 206 synchronizes the incoming signal to enable data extraction. Synchronizing includes, for example, determining the distortion to the host signal and compensating for it. This process provides the location and arrangement of encoded data elements of a signal within digital content.

[0067] The data extractor module 208 gets this location and arrangement and the corresponding protocol and demodulates a data signal from the host. The location and arrangement provide the locations of encoded data elements. The extractor obtains estimates of the encoded data elements and performs a series of signal decoding operations.

[0068] As detailed in examples below and in the incorporated documents, the detector, synchronizer, and data extractor may share common operations, and in some cases may be combined. For example, the detector and synchronizer may be combined, as initial detection of a portion of the data signal used for synchronization indicates presence of a candidate data signal, and determination of the synchronization of that candidate data signal provides synchronization parameters that enable the data extractor to apply extraction filters at the correct orientation, scale and start location. Similarly, data extraction filters used within data extractor may also be used to detect portions of the data signal within the detector or synchronizer modules. The decoder architecture may be designed with a data flow in which common operations are re-used iteratively or may be organized in separate stages in pipelined digital logic circuits so that the host data flows efficiently through the pipeline of digital signal operations with minimal need to move partially processed versions of the host data to and from a shared memory, such as a RAM memory.

[0069] The detector module 202 may alternatively comprise one or more trained network models (e.g., deep learning models utilizing convolutional neural networks (CNNs) and / or recurrent neural networks (RNNs)) optimize the detection of a variable watermark payload in a host signal. These trained network models are employed within the signal detector to yield auxiliary data, despite the presence of noise, rotation, scaling, temporal shifts, scaling, etc. Machine trained decoders are further discussed, e.g., in assignee’s US Patent Nos. 11,704,765 and 11,625,805, and in assignee’s US Published Application Nos. 20220270199 and 20210357690, each of which is hereby incorporated herein in its entirety.

[0070] Fig. 3 is a flow diagram illustrating operations of a signal generator. Each of the blocks in the diagram depict processing modules that transform the input auxiliary data (e.g., the payload) into a data signal structure. For a given protocol, each block provides one or more processing stage options selected according to the protocol. In processing module 300, the auxiliary data is processed to compute error detection bits, e.g., such as a Cyclic Redundancy Check, Parity, or like error detection message symbols. Additional fixed and variable messages used in identifying the protocol and facilitating detection, such as synchronization signals may be added at this stage or subsequent stages.

[0071] Error correction encoding module 302 transforms the message symbols into an array of encoded message elements (e.g., binary or M-ary elements) using an error correction method. Examples include block codes, convolutional codes, etc.

[0072] Repetition encoding module 304 repeats the string of symbols from the prior stage to improve robustness. For example, certain message symbols may be repeated at the same or different rates by mapping them to multiple locations within a unit area of the data channel (e.g., one unit area being a tile of bit cells, bumps or “waxels,” as described further below).

[0073] Next, carrier modulation module 306 takes message elements of the previous stage and modulates them onto corresponding carrier signals. For example, a carrier might be an array of pseudorandom signal elements. The data elements of an embedded signal may also be multi-valued. In this case, M-ary or multi-valued encoding is possible at each signal element, through use of different colors, ink quantity, dot patterns or shapes. Signal application is not confined to lightening or darkening an object at a signal element location (e.g., luminance or brightness change). Various adjustments may be made to effect a change in an optical property, like luminance. These include modulating thickness of a layer, surface shape (surface depression or peak), translucency of a layer, etc. Other optical properties may be modified to represent the signal element, such as chromaticity shift, change in reflectance angle, polarization angle, or other forms optical variation. As noted, limiting factors include both the limits of the marking or rendering technology and ability of a capture device to detect changes in optical properties encoded in the signal. We elaborate further on signal configurations below.

[0074] Mapping module 308 maps signal elements of each modulated carrier signal to locations within the channel. In the case where a digital host signal is provided, the locations correspond to embedding locations within the host signal. The embedding locations may be in one or more coordinate system domains in which the host signal is represented within a memory of the signal encoder. The locations may correspond to regions in a spatial domain, temporal domain, frequency domain, or some other transform domain. Stated another way, the locations may correspond to a vector of host signal features at which the signal clement is inserted.

[0075] Various detailed examples of protocols and processing stages of these protocols are provided in, e.g., US Patents 6,614,914, 5,862,260, 6,345,104, 6,993,152 and 7,340,076, which are hereby incorporated by reference in their entirety, and US Patent Publication 20100150434, previously incorporated. More background on signaling protocols, and schemes for managing compatibility among protocols, is provided in US Patent 7,412,072, which is hereby incorporated by reference in its entirety.

[0076] In some case, the output of carrier modulation module 306 and / or mapping module 308 is used to generate a watermark signal that can be concatenated or combined with a host image or audio in a trained convolutional neural network (CNN) or recurrent neural networks (RNN) encoder model. These concatenated or combined host image or audio can be used as training input to such models. Different loss functions and optimization strategies can be employed during the training phase to achieve a desired performance, e.g., desired robustness against attacks (scaling, rotation, translation, cropping, etc.).

[0077] The above description of signal generator module options demonstrates that the form of the signal used to convey the auxiliary data varies with the needs of the application. As introduced at the beginning of this document, signal design involves a balancing of required robustness, data capacity, and perceptual quality. It also involves addressing many other design considerations, including compatibility, print constraints, scanner constraints, robustness to attacks, etc. We now turn to examine signal generation schemes, and in particular, schemes that employ signaling, and schemes for facilitating detection, synchronization, and data extraction of a data signal in a host channel.

[0078] One signaling approach, which is detailed in US Patents 6,614,914, and 5,862,260, is to map signal elements to pseudo-random locations within a channel defined by a domain of a host signal. See, e.g., Fig. 9 of 6,614,914. In particular, elements of a watermark signal arc assigned to pseudo-random embedding locations within an arrangement of sub-blocks within a block (referred to as a “tile”). The elements of this watermark signal correspond to error correction coded bits output from an implementation of stage 304 of Fig. 3. These bits are modulated onto a pseudo-random carrier to produce watermark signal elements (block 306 of Fig. 3), which in turn, are assigned to the pseudorandom embedding locations within the sub-blocks (block 308 of Fig. 3). An embedder module modulates this signal onto a host signal by adjusting host signal values at these locations for each error correction coded bit according to the values of the corresponding elements of the modulated carrier signal for that bit.

[0079] The signal decoder estimates each coded bit by accumulating evidence across the pseudo-random locations obtained after non-linear filtering a suspect host digital content. Estimates of coded bits at the signal clement level arc obtained by applying an extraction filter that estimates the signal element at particular embedding location or region. The estimates are aggregated through de-modulating the carrier signal, performing error correction decoding, and then reconstructing the payload, which is validated with error detection.

[0080] This pseudo-random arrangement spreads the data signal such that it has a uniform spectrum across the tile. However, this uniform spectrum may not be the best choice from a signal communication perspective since energy of a host digital content may concentrated around DC. Similarly, an auxiliary data channel in high frequency components tends to be more disturbed by blur or other low pass filtering type distortion than other frequency components. A variety of signal arrangements are detailed in US Patent No. 9,747,656, which are each hereby incorporated by reference in its entirety. This application details several signaling strategies that may be leveraged in the design of encoded signals, in conjunction with the techniques in this document. Differential encoding applies to signal elements by encoding in the differential relationship between a signal element and other signals, such as a background, host elements, or other signal components (e.g., a sync component).

[0081] US Patent No. 6,345,104, building on the disclosure of US Patent No. 5,862,260, describes that an embedding location may be modulated by inserting ink droplets at the location to decrease luminance at the region, or modulating thickness or presence of line art. Additionally, increases in luminance may be made by removing ink or applying a lighter ink relative to neighboring ink. It also teaches that a synchronization pattern may act as a carrier pattern for variable data elements of a message payload. The synchronization component may be a visible design, within which a sparse data signal (see, e.g., US Patent No. 11,062,108) or dense data signal is merged. Also, the synchronization component may be designed to be imperceptible, using the methodology disclosed in US Patent No. 5,862,260.

[0082] We further discuss the design, encoding and decoding of signals in more detail. As introduced above, one consideration in the design of an encoded signal is the allocation of signal for data carrying and for synchronization. Another consideration is compatibility with other signaling schemes in terms of both encoder and decoder processing flow. With respect to the encoder, the encoder should be compatible with various signaling schemes, including dense and sparse signaling, so that it each signaling scheme may be adaptively applied to different regions of a digital content design, as represented in a digital content, according to the characteristics of those regions. This adaptive approach enables the user of the encoder tool to select different methods for different regions and / or the encoder tool to be programmed to select automatically a signaling strategy that will provide the most robust signal, yet maintain the highest quality image, for the different regions. Additional details regarding sparse digital watermarking are described in Digimarc’s published PCT application no. WO 2020186234, which is hereby incorporated herein by reference in its entirety.

[0083] One example of the advantage of this adaptive approach is in a design that has different regions requiring different encoding strategies. One region may be blank, another blank with text, another with a graphic in solid tones, another with a particular spot color, and another with variable image content.

[0084] With respect to the decoder, this approach simplifies decoder deployment, as a common decoder can be deployed that decodes various types of data signals, including both dense and sparse signals.

[0085] As introduced above with reference to Fig. 3, there arc stages of modulation / dc- modulation in the encoder, so it is instructive to clarify different types of modulation. One stage is where a data symbol is modulated onto an intermediate carrier signal. Another stage is where that modulated carrier is inserted into the host by modulating elements of the host. In the first case, the carrier might be pattern, e.g., a pattern in a spatial domain or a transform domain (e.g., frequency domain). The carrier may be modulated in amplitude, phase, frequency, etc. The carrier may be, as noted, a pseudorandom string of l’s and 0’s or multi-valued elements that is inverted or not (e.g., XOR, or flipped in sign) to carry a payload or sync symbol.

[0086] As noted in US Patent No. 9,747,656, carrier signals may have structures that facilitate both synchronization and variable data carrying capacity. Both functions may be encoded by arranging signal elements in a host channel so that the data is encoded in the relationship among signal elements in the host. US Patent No. 9,747,656 specifically elaborates on a technique for modulating, called differential modulation. In differential modulation, data is modulated into the differential relationship among elements of the signal. In some watermarking implementations, this differential relationship is particularly advantageous because the differential relationship enables the decoder to minimize interference of the host signal by computing differences among differentially encoded elements. In sparse data signaling, there may be little host interference to begin with, as the host signal may lack information at the embedding location.

[0087] Another form of modulating data is through selection of different carrier signals to carry distinct data symbols. One such example is a set of frequency domain peaks (e.g., impulses in the Fourier magnitude domain of the signal) or sine waves. In such an arrangement, each set carries a message symbol. Variable data is encoded by inserting several sets of signal components corresponding to the data symbols to be encoded. The decoder extracts the message by correlating with different carrier signals or filtering the received signal with filter banks corresponding to each message carrier to ascertain which sets of message symbols are encoded at embedding locations.

[0088] Having now illustrated methods to modulate data into the watermark (either dense or sparse), wc now turn to the issue of designing for synchronization. For the sake of explanation, we categorize synchronization as explicit or implicit. An explicit synchronization signal is one where the signal is distinct from a data signal and designed to facilitate synchronization. Signals formed from a pattern of impulse functions, frequency domain peaks or sine waves is one such example. An implicit synchronization signal is one that is inherent in the structure of the data signal. An implicit synchronization signal may be formed by arrangement of a data signal. For example, in one encoding protocol, the signal generator repeats the pattern of bit cells representing a data element. We sometimes refer to repetition of a bit cell pattern as “tiling” as it connotes a contiguous repetition of elemental blocks adjacent to each other along at least one dimension in a coordinate system of an embedding domain. The repetition of a pattern of data tiles or patterns of data across tiles (e.g., the patterning of bit cells in US Patent 5,862,260) create structure in a transform domain that forms a synchronization template. For example, redundant patterns can create peaks in a frequency domain or autocorrelation domain, or some other transform domain, and those peaks constitute a template for registration. See, for example, US Patent No. 7,152,021, which is hereby incorporated by reference in its entirety.

[0089] The concepts of explicit and implicit signaling readily merge as both techniques may be included in a design, and ultimately, both provide an expected signal structure that the signal decoder detects to determine geometric distortion.

[0090] In one arrangement for synchronization, the synchronization signal forms a carrier for variable data. In such arrangement, the synchronization signal is modulated with variable data. Examples include sync patterns modulated with data.

[0091] Conversely, in another arrangement, that modulated data signal is arranged to form a synchronization signal. Examples include repetition of bit cell patterns or tiles.

[0092] The variable data and sync components of the encoded signal may be chosen so as to be conveyed through orthogonal vectors. This approach limits interference between data carrying elements and sync components. In such an arrangement, the decoder correlates the received signal with the orthogonal sync component to detect the signal and determine the geometric distortion. The sync component is then filtered out. Next, the data carrying elements are sampled, e.g., by correlating with the orthogonal data earner or filtering with a filter adapted to extract data elements from the orthogonal data carrier. Signal encoding and decoding, including decoder strategies employing correlation and filtering are described in US Patent No. 9,747,656. Additional examples of explicit and implicit synchronization signals are provided in previously cited patents 6,614,914, and 5,862,260. In particular, one example of an explicit synchronization signal is a signal comprised of a set of sine waves, with pseudorandom phase, which appear as peaks in the Fourier domain of the suspect signal. See, c.g., 6,614,914, and 5,862,260, describing use of a synchronization signal in conjunction with a robust data signal. Also see US Patent No. 7,986,807, which is hereby incorporated by reference in its entirety.

[0093] US Publication No. 20120078989, which is hereby incorporated by reference in its entirety, provides additional methods for detecting an embedded signal with this type of structure and recovering rotation, scale, and translation from these methods.

[0094] Additional examples of implicit synchronization signals, and their use, are provided in US Patent Nos. 9,747,656, 7,072,490, 6,625,297, 6,614,914, and 5,862,260, which are hereby incorporated by reference in their entirety. Signal encoders and decoders may also employ network models trained to embed and extract the auxiliary data signal so as to be robust to geometric and temporal transformations, and thus, provide implicit synchronization. In these machine-learning based approaches, portions of the auxiliary data may function as a synchronization signal. Further, the features or encoding domains in which the models are trained to embed and extract the auxiliary data may be selected to be robust to anticipated forms of geometric or temporal transformation (e.g., spatial or temporal scale, rotation, or shift invariant feature sets).

[0095] Please also see US Patent No. 8,014,557, which is hereby incorporated herein by reference, for a description of watermarking text and text documents.

[0096] II. Digital Watermarking for Digital Image, Video and Audio Protection and Manifest Swapping Detection

[0097] It’s becoming tough to trust digital assets, e.g., digital images, digital audio, digital artwork, PDFs, text documents, 3D models, and / or digital video. Fake digital images and digital video have surfaced, eroding confidence in governments, societal norms, and trusted relationships. Afterall, Generative Artificial Intelligence (Gen Al) and machine-learning algorithms can create convincing fakes at the click of a button.

[0098] Standards bodies have come together to shore up trust between authentic digital assets (e.g., digital images, digital video, digital audio) and their providence. One example is the Coalition for Content Provenance and Authenticity (“C2PA”). Provenance information, in this context, can help establish the truth about the origin, history and authenticity of digital assets, by providing evidence of its creation, discovery, ownership and movement over time. See the C2PA Specification, Version 1.3 at: https: / / c2pa.Org / specifications / specifications / l.3 / specs / C2PA Specification.html . The C2PA Specification, v.1.3, is hereby incorporated herein by reference in its entirety. The C2PA framework associates digital assets, with so-called “manifests”. A manifest is a standardized set of metadata associated with a digital asset that provides information about its origin, authenticity, and modifications, enabling users to verify the content's provenance and integrity.

[0099] Some of the terms used in the C2PA Specification, v.1.3, and found through the above link, include the following:

[0100] 2.3. Core Aspects of C2PA

[0101] 2.3.1. Assertion: A data structure which represents a statement asserted by an actor concerning the asset. This data is a part of the C2PA Manifest.

[0102] 2.3.2. Claim: A digitally signed and tamper-evident data structure that references a set of assertions by one or more actors, concerning an asset, and the information necessary to represent the content binding. If any assertions were redacted, then a declaration to that effect is included. This data is a part of the C2PA Manifest.

[0103] 2.3.3. Claim signature: The digital signature on the claim using the private key of an actor. The claim signature is a part of the C2PA Manifest.

[0104] 2.3.4. C2PA Manifest: The set of information about the provenance of an asset based on the combination of one or more assertions (including content bindings), a single claim, and a claim signature. A C2PA Manifest is part of a C2PA Manifest Store. A C2PA Manifest can reference other C2PA Manifests. 2.3.5. C2PA Manifest Store: A collection of C2PA Manifests that can either be embedded into an asset or be external to its asset.

[0105] 2.3.6. Origin: The C2PA Manifest in the provenance data which represents the software or device that initially created the asset.

[0106] 2.3.7. Active Manifest: The last manifest in the list of C2PA Manifests inside of a C2PA Manifest Store which is the one with the set of content bindings that arc able to be validated.

[0107] 2.3.8. Provenance: The logical concept of understanding the history of an asset and its interaction with actors and other assets, as represented by the provenance data.

[0108] 2.3.9. Provenance data: The set of C2PA Manifests for an asset and, in the case of a composed asset, its ingredients.

[0109] 2.3.10. Authenticity: A property of digital content comprising a set of facts (provenance data and hard bindings) that can be cryptographically verified as not having been tampered with.

[0110] 2.3.11. Content binding: Information that associates digital content to a specific C2PA Manifest associated with a specific asset, either as a hard binding or a soft binding.

[0111] 2.3.12. Hard binding: One or more cryptographic hashes that uniquely identifies either the entire asset or a portion thereof.

[0112] 2.3.13. Soft binding: A content identifier that is either (a) not statistically unique, such as a fingerprint, or (b) embedded as a watermark in the identified digital content.

[0113] 2.3.14. Trust signals: The collection of information that can inform an actor’s judgment of the trustworthiness of an asset. These are in addition to the signer of a claim, upon which the fundamental trust model relies.

[0114] 2.4. Additional Terms

[0115] 2.4.1. Fingerprint: A set of inherent properties computable from digital content that identifies the content or near duplicates of it. EXAMPLE: An asset can become separated from its manifest due to removal or corruption of asset metadata. A fingerprint of the digital content of the asset could be used to search a database to recover the asset with an intact manifest. 2.4.2. Watermark: Information incorporated into the digital content (perceptibly or imperceptibly) of an asset which can be used, for example, to uniquely identify the asset or to store a reference to a C2PA Manifest.

[0116] 2.4.3. Manifest Repository: A repository into which C2PA Manifests and C2PA Manifest Stores can be placed, and which can be searched using a content binding.

[0117] C2PA Version 2.0 can be found at: https: / / c2pa.Org / specifications / specifications / 2.0 / specs / C2PA Specification. html, which is hereby incorporated herein by reference in its entirety.

[0118] Several malicious attacks remain unsolved in the C2PA framework. For example, manifest swapping. Manifest swapping occurs when a malicious attacker strips away an original manifest associated with a digital asset and swaps, in its place, a different, altered manifest, one likely feigning authenticity. This attack can be minimized through use of digital watermarking. Digital watermarks create a strong link between a manifest and the digital asset. Digital watermarking can also be used to reassociate digital assets with their original manifests and allow manifest authentication. And even though the C2PA Specification mentions watermarking to uniquely identify an asset or to store a reference to a C2PA Manifest, it does not use digital watermarking to address and identify manifest swapping.

[0119] Now consider an online verification system with reference to FIG. 4A-8B. The online verification system includes, e.g., software instructions executing on one or more multi-core processors (e.g., two or more multi-core parallel processors), executing on cloud-based servers, and / or running as a Platform as a Service (PaaS). The software instructions provide an online environment. The online environment includes a plurality of graphical user interfaces (GUIs) and / or Application Program Interfaces (“API”). The software instructions may also include call and / or communicate with a variety of other modules, networks and systems, e.g., a digital watermarking embedder, digital watermark decoder, digital asset repository, databases and / or data records, and account management modules, a mobile application, a web browser plugin / library or a bookmarklet. The online environment can be accessed via the internet or other communications network. It should be appreciated that while the following discussion focuses on digital images and artwork, other digital assets (e.g., digital audio, digital video, and / or 3D designs) can be similarly protected and verified.

[0120] FIG. 4A shows a graphical user interface (GUI) provided by the online environment through which a user can upload a digital image to protect and / or verify. In a protect mode, an upload graphic or button can be selected to initiate a protection process. The verification system accesses or opens a file directory, e.g., stored locally on a user’s computer network system (FIG. 4B). A digital asset, here a digital image generated by a Gen Al system is selected (see FIG. 4C), a JPEG file (“astronaut-ai.jpeg”), and uploaded to the verification system. The verification system processes the selected digital asset to extract data associated with the file, e.g., a C2PA manifest formatted as a JSON file (FIG. 4D) and searches the digital asset for a previously embedded digital watermark (FIG. 4E). For example, the verification system calls or operates a digital watermark detector, which searches the digital asset for a digital watermark embedded therein. The digital watermark detector, or stages of the detector, may be executed locally on a user client and / or executed in the online environment (e.g., implemented as a cloud computer service). See, e.g., Section I above, and the documents incorporated above for a discussion of digital watermark detection. The image is displayed in the online environment, here an astronaut floating in space (FIG. 4F) and image manifest information can be accessed, e.g., via selection of the “i” button displayed over the image (FIG. 4G). In this example, the digital watermark detector did not find a digital watermark in the image. Upon selecting 'Next', the verification system initiates digital watermark embedding of the image (FIG. 4H). For example, a digital watermark embedder is called or executed to modify the image to include a plural-bit payload therein; the digital watermark embedder yields a digital watermarked image. As noted, this embedder may include a machine learning trained watermark encoder, executing locally relative to a user, accessed via a network and implemented on a cloud-based computing server, and / or distributed between a client and server. Digital watermark embedding is described in Section I above, and in the above incorporated by reference patent documents. (In an alternative implementation, a digital watermark is found in the image. For example, a capture device (e.g., a smartphone or tablet) is added during or after captured via a device operating system, e.g., a watermark embedder built into a smartphone operating system, or into camera app, or encoded during capture via an image sensor or hardware chip.)

[0121] A hard-binding hash is generated from the digital watermarked image and stored within the manifest (e.g., under an assertions field indicating that the hard-binding hash is created after digital watermarking), yielding an updated manifest. The updated manifest can be further updated to include a hash of the assertions, which enables checking whether assertions have changed. The updated manifest can be cryptographically signed to verify these actions. (The updated manifest can also include digital watermark carried information, e.g., such as a plural-bit payload.) Such a signing helps bind a manifest to a person or entity, or both. After digital watermarking and hash generation, the verification system communicates both the updated manifest and digital watermark plural-bit payload (e.g., e.g., including one or more identifiers, flags, hashes or signatures, and the like) to a digital asset repository. The updated manifest is stored in the digital asset repository and is referred to herein as a “stored manifest”. The digital asset repository can provide a centralized or decentralized repository that stores, manages, and indexes manifests, metadata, and associated configuration details, ensuring consistent and structured data access across various applications and services. This digital asset repository may offer capabilities such as data versioning, discoverability via search and tagging, access control based on user roles and permissions, and integration points via APIs for cloud-native tools. By providing a unified view and control mechanism for digital assets, the digital asset repository ensures that queries into the repository consume consistent and trusted data. Within the digital asset repository, the stored manifest is associated with the digital watermark, e.g., via an identifier, hash, or other metadata included in the payload. The digital asset repository can be co-located with respect of the validation system or located remotely and in communication with the online environment, e.g., via the internet, wireless network, cellular network, or a combination of such networks. This stored manifest and digital watermark payload can be used for future retrieval and verification of the associated digital asset. In fact, the stored manifest, and its association with the digital watermark payload, becomes the bedrock for all future comparisons to help prevent, e.g., manifest swapping.

[0122] The digital watermarking protects the digital image, providing an indication to expect a manifest associated with the image, and a link or index (e.g., via the digital watermark plural-bit identifier) to access the stored manifest. The digital watermarking preferably alters at least some human perceptual elements of the digital asset, meaning elements that are perceptible to a human when the digital asset is rendered and output for perception by a human viewer or listener. For digital imagery, including video, the digital watermarking alters at least some human perceptual elements of the digital imagery, e.g., alters data representing pixel values or color values (e.g., RGB), luminance and / or chrominance that correspond to features perceived by a human viewer when output (e.g., displayed or printed). For digital audio, the digital watermarking alters at least some human perceptual elements of the digital audio, such as digital audio samples or functions of samples, such as magnitude or phase of frequency coefficients, echoes, statistical values, etc. The alterations can be applied in a spatial domain, in a temporal domain, in a transform domain (e.g., a log polar domain) or through a trained machine-learning based system. In one implementation of a machine-learning based system, a digital watermark is added at the time a digital asset is generated. For example, the digital watermarking is pail of or incorporated into the Al digital asset generation process. Similarly, if the digital asset includes audio or video, human perceptual elements of the audio or video are altered by the digital watermarking. Altering human perceptual elements helps the digital watermarking maintain robustness. For example, a digital watermark removal attempt will remove or distort the perceptual elements, diminishing the quality of the original image, audio, or video. In an alternative embedding case, digital watermarking is embedded by altering transform domain coefficients, such as DCT coefficients, wavelets or the like. Example digital image watermarking technology is described above including in the incorporated by reference patents.

[0123] The digital watermarking encoded within the image preferably carries a plural-bit payload, e.g., a plural-bit identifier. The identifier may be used as an index into the digital asset repository that is housing the stored manifest. The plural-bit payload may alternatively include plural portions or fields. In these cases, the payload preferably still carries the identifier in a first field and also carries, e.g., a perceptual hash of some or all of the digital image, in a second field. In related plural-field payload implementations, the payload also (or alternatively) includes a hash (e.g., a reduced-bit representation) or fingerprint of some or all of the manifest data. In still other related implementations, the payload also (or alternatively) includes a cryptographic hash (e.g., using public / private key pair) of some or all of the manifest. In a further implementation, the digital image includes an overt visible watermark, and the digital watermark payload may also (or alternatively) include information regarding the overt visible watermark (e.g., expected spatial location within in image, description, identifier, expected orientation, etc.). In one implementation, the overt visible watermark is a graphical icon in which digital data is embedded, e.g., using above-described methods for encoding, and employing the border or feature points of the icon as an explicit synchronization signal. In another implementation, the payload includes, e.g., a concatenated bit- string representing the identifier and / or address information to access the digital asset repository. In another implementation the payload contains a cryptographically signed version of any of the above implementations. In one example, the digital watermarking includes both a pluralbit payload and a synchronization signal (or synchronization “component”). Example synchronization signals arc discussed above.

[0124] The freshly digital watermarked image (and updated manifest) is presented for download (FIG. 41). Preferably, the original, unwatermarked image is deleted or otherwise secured so that it can’t be inadvertently leaked or accessed. Otherwise, an unscrupulous actor could pass the original off as their own. In another implementation, the original image (or other digital asset) is stored in a secure online digital asset repository or a distributed storage system, e.g., where it can be compared against downstream versions of such to detect alterations. A distributed storage system is described in assignee US Published Patent Application No. 20210233204, which is hereby incorporated herein by reference in its entirety.

[0125] Now let’s consider some authentication aspects of the verification system. We’ll use the same astronaut image as was just protected (e.g., digital watermarked). While not mentioned previously, let’s suppose the digital watermarked image was stored in a file directory as “astronaut-ai-protected.jpg”. A user accesses the online environment through the GUI shown in FIG. 4A and clicks or selects the graphical button “Verify” in the top right comer of the GUI. This launches the upload function, which accesses a file directory (FIG. 5A). The “astronaut-ai-protected.jpg” digital asset is selected, uploaded to the verification system, and displayed (FIG. 5B). As discussed above with reference to FIGS. 4D & 4E, the verification system extracts the digital asset’s manifest and searches the digital asset for a previously encoded digital watermark. Extracting a manifest may include reading the file information associated with the manifest, e.g., reading some or all of the data from a JSON file. The JSON file includes labels to identify information, e.g., a hard-binding perceptual hash of the digital asset, assertions, actions, edits, title, format, etc. Decoding the digital watermark yields at least its plural-bit identifier, which is communicated to the digital asset repository to access the stored manifest that is associated with this particular plural-bit identifier. (If the manifest comprises a field including the plural-bit identifier, the decoded plural-bit identifier and the manifest plural-bit identifier can be compared to see if they match. This can occur locally with respect to the verification system. A mismatch of identifiers indicates a potential manifest swap, or that the image or manifest do not correspond as expected.) Once obtained by the verification system from the digital asset repository, the stored manifest can be compared with the extracted manifest. The term “compared” can range from straight forward to complex. In a first implementation, for example, a character count (e.g., number of characters included in the manifest) of the extracted manifest can be compared to a character count of the stored manifest. If they are the same, then the manifest is considered authentic. In a second implementation, a hash of the stored manifest is compared with a hash of the extracted manifest. Relatedly, the hash may exclude certain fields randomly or based on a predetermined selection. A changed manifest is revealed when the hashes do not match. In a third implementation, the hard-bindings from each manifest version arc compared, perhaps also against a newly generated hash of the digital image (e.g., using the same hashing algorithm that was used to create the hard-bindings). A changed manifest is revealed when the hashes do not match. In a fourth implementation, a character-by-character comparison is carried out between the manifests. A first character in the extracted manifest is co pared with the first character in the stored manifest; a second character in the extracted manifest is compared with the second character in the stored manifest; and so on until the nth character in the extracted manifest is compared with the nth character in the stored manifest (where n is a positive integer). In a fifth implementation, predetermined fields, e.g., hard-binding hash, actions, assertions, and instance_id, are compared character-by-character, or compared via hashes. Mismatches are noted and can be used to determine a match or mismatch.

[0126] Returning to FIGS. 5C and 5D, a pop-up window (e.g., selected via a graphic overlay icon) is configured to show that the digital watermark was found, and the extracted manifest is authentic (via a comparison with the stored manifest). This pop-up can also display a result of watermark matching (e.g., a link or associated between a decoded watermark and its manifest is valid) and that a perceptual hash of the digital asset matches one included in the manifest (e.g., indicating that the image or video has not be significantly modified). This helps prevent manifest swapping as will be discussed further below. Verification may also be integrated into systems for crawling and scraping content (e.g., an internet spider, spiderbot or web crawler) for machine learning model training, as well as into filters for evaluating files in the training data set of a machine learning model prior to training. A web crawler fetches web pages and extract information from them. A crawler may start with a list of seed URLs. This list is often called the URL frontier. The crawler sends an HTTP request to a web server to retrieve the content of a web page associated with a URL. Upon receiving the HTTP response, the crawler downloads the page content, typically in HTML format. The crawler parses the page content and identifies any links (e.g., via tags). These new URLs might be added to the URL frontier, e.g., whether they fulfill certain criteria. Content of the web page, along with relevant metadata, can be stored, often in a database. This database can be used for machine learning model training. Some crawling operations employ multiple crawlers running in parallel, often distributed across different machines or locations. Some websites may use JavaScript to load content dynamically. Thus, a crawler can be configured to execute JavaScript to retrieve such content.

[0127] Now consider a verification action that includes an altered image. Our protected (e.g., digital watermarked) astronaut image is edited to include a black dot in the top helmet area. See FIG. 6A. This edited image still includes its previously embedded digital watermark. This edited image is stored as “astronaut-ai-protected-edited.jpg”. FIG. 6B.

[0128] Now let’s try to verify the edited image via the verification system. A user accesses the verification system (FIG. 4A) and selects the “Verify” option in the upper right comer of the GUI. An image upload button is selected, accessing a file directory, where the astronaut-ai-protected-edited.jpg file is selected (FIG. 6C). After upload, the verification system exacts the C2PA manifest (FIG. 6D) and, utilizing a digital watermark decoder, decodes a digital watermark from the image (FIG. 6E) to obtain a plural-bit identifier carried by a payload. The uploaded image (including the added black dot) can be displayed via the online environment as shown in FIG. 6F.

[0129] The verification system communicates the plural-bit identifier to the digital asset repository to retrieve the stored manifest. A comparison of the image, the extracted manifest and / or stored manifest is executed to determine authenticity. For example, even though the extracted manifest and the stored manifest may match, a newly generated hash of the image itself does not. That is the verification system computes a hash of the edited image using the same hashing algorithm used to create the hard-binding hash carried in the extracted manifest. In some cases, the manifest itself identifies the particular hashing algorithm. This new hash does not match the hard-binding hash stored in the stored manifest. A comparison failure can be shown via a graphical user interface, e.g., as in FIG. 6G (e.g., “The hard bindings are not matching.”).

[0130] In one embodiment when adding the digital watermark to the image, a perceptual hash of the watermarked image is extracted (e.g., pHash). This can be used to further refine if the image was edited significantly. Similarly, storing and comparing a hcatmap showing embedded signal strength and / or predicted robustness of the watermark decoder can be used to assess which parts of a digital asset were modified and how significantly they were modified. For example, a newly embedded digital asset is embedded with a digital watermark signal. The digital watermark signal may include one or more different components, e.g., a synchronization component and a message component. The digital watermark signal is embedded in plural-portions of the digital asset, e.g., in different spatial areas or frequency components of an image or video. (Of course, our inventive systems, methods and technology can be used to secure other digital assets besides images and video, e.g., audio, text, PDFs, graphics, digital art and NFTs, e-books, virtual property, video game assets, artwork and 3D models, etc.) After embedding, a digital watermark detector analyzes the digital image and creates robustness or detectability metrics for each of the one or more different components for each of the plural-portions. See assignee’s US Patent Nos. 10217182 and 10748231, which are each hereby incorporated herein by reference in its entirety for example detectability metrics. The metrics can be used along or in combination to generate a heatmap of the predicted signal robustness, e.g., with different colors representing more or less likely detection areas. A representation of the heatmap, e.g., pixel values, a hash of such values, spatial locations of different values, and / or underlying detection metrics, is stored (e.g., in the digital asset repository) for a digitally watermarked digital asset. The manifest can be optionally updated to reflect the presence of the heatmap representation. A validation confirmation may include generating a heatmap of an encountered digital asset and comparing some or all of the generated heatmap (or underlying values) with the stored heatmap representation (or its underlying values). A change in the generated heatmap relative to the stored heatmap representation indicates that the digital asset has changed. A spatial location of the change can also be determined from the heatmap representation.

[0131] A change could also be detected if the extracted manifest is modified, e.g., updated in the activities list to show the edit. A comparison with the stored manifest would identify that difference. In that case, the comparison of the extracted manifest would not match the stored manifest, e.g., using one of the comparison implementations discussed above.

[0132] In another scenario, an image is uploaded with a digital watermark but without a manifest. It's easy to erase an image's manifest using a tool like Remove EXIF, IPTC, IMGOnline. Moreover, many social media networks remove manifest data to protect user privacy, e.g., so location data is not divulged. Additionally, many tools such as multimedia editors remove manifests. See, e.g., FIG. 7A, where the example file “astronaut- ai-protected.jpeg” (FIGS. 5 A & 5B) is stripped of its manifest. In this case, the verification system is unable to extract a manifest (because it was stripped away prior to upload) but it does decode a digital watermark embedded in the image. The digital watermark decoder returns a plural-bit identifier carried by a decoded payload. The plural-bit identifier is communicated to the digital asset repository to index a corresponding stored manifest. The stored manifest is returned to the verification system for display (see FIGS. 7B & 7C). The stored manifest can be displayed and associated with the image for download. The verification system can also perform a hard-binding check, by computing a hash of the uploaded image and comparing it to the hard-binding hash carried in the stored manifest. This result can also be displayed via a GUI or can be used as a condition precedent prior to allowing access to the stored manifest.

[0133] Now consider a straight up manifest swap. In this attack, a manifest is stripped away from its digital asset such as in FIG. 7A. A different manifest is generated using the digital asset, e.g., using a compliant C2PA manifest generator. The digital asset with its newly minted manifest would appear to be authentic. Afterall, it has a compliant manifest, even though the information contained therein is faked or fraudulent. In this swapping example, let’s assume that the digital asset has been previously digitally watermarked to include a plural-bit identifier, e.g., using the process discussed above with respect to FIGS. 4A-4I. Now consider a verification attempt of the swappedmanifest digital asset. The verification system is accessed to upload an image including an embedded digital watermark but with the wrong manifest. In this example, the file “fakc_astronaut.jpcg” is selected (FIG. 8A). This image has a digital watermark corresponding to the protected version (e.g., FIG. 41). The verification system extracts the manifest, decodes (or calls a decoder to decode) the digital watermark to obtain the plural-bit identifier. A corresponding stored manifest is identified in a digital asset repository via the plural-bit identifier. Once obtained, the stored manifest is compared with the extracted manifest. Here again, the comparison can include any of the comparison implementations discussed above. Here the manifests do not match. In this example, the manifest comparison operates field-by-field. This allows for specific differences to be identified. For example, in FIG. 8B, differences between the manifests include fields: claim_generator, title, format, instance_id, and / or thumbnail, etc.

[0134] One implementation of the verification system is a dedicated web browser extension. Such extensions are typically software programs that can modify and enhance the functionality of a web browser. Extensions can be written using, e.g., HTML, CSS (“Cascading Style Sheets”), and / or JavaScript. The web browser extension can include, deploy, or call a digital watermark decoder and digital watermark encoder, e.g., via decoder and encoder code incorporated via software instructions within the web browser extension, or, alternatively, deployed by being called by the web browser extension. If the web browser extension calls a remotely located digital watermark decoder, the web browser extension can provide imagery to the digital watermark decoder. In another implementation, the web browser extension provides a web address hosting the digital imagery to the digital watermark decoder, which accesses the digital asset by accessing the web address.

[0135] Instead of using a browser extension, the validation system, including a digital watermark decoder and validation / comparison features discussed above, can be incorporated into a standalone application or into a verification website itself or, e.g., via a Javascript SDK, into any website carrying multimedia assets. In an alternative, a plugin is used instead of a web browser extension. In still another alternative, the decoder and verification / comparison features are provided by a webpage or web service. In this case, a digital asset can be verified by sending the web address (URL) or storage location (URI) of the digital asset to the webpage or web service. This allows for instance verification in browsers that do not support extensions (e.g., some mobile browsers) and could use for example a bookmarklet - a bookmark stored in a web browser that contains Javascript commands that add new features to the browser.

[0136] Also, instead of using a web browser extension, a smartphone running a verification app could be used to protect and verify digital assets. In an offline mode, the verification app can yield information based on watermark detection. For example, a watermark is detected but its expected manifest was stripped off. Detection of the watermark signals that the manifest should be expected, so the image is untrustworthy until the original manifest can be obtained. As a further alternative, the verification can be carried out within an operating system (OS), below a level of applications, browsers, etc. For example, before a digital asset is rendered on a system screen or speakers, a digital watermark detector incorporated within or called by an OS can analyze the digital asset and perform a manifest / digital watermark verification. In another alternative, digital watermark detection is performed by a smart contract on a blockchain or distributed ledger technology (DLT) platform.

[0137] Digital Asset Validator and Digital Watermark Resolver

[0138] A digital asset validator system is discussed with reference to FIG. 9. The system includes one or more digital asset validators, one or more digital watermark resolvers, and one or more digital asset repositories, in communication through one or more networks such as the internet, wireless networks, cellular networks, or a combination of such networks. The digital asset validator operates to help users establish trustworthiness of a digital asset and can be implemented using, e.g., software instructions executing on one or more multi-core processors (e.g., two or more multi-core parallel processors), software instructions executing on cloud-based servers, software instructions operating as an application (“app”) on a smartphone, and / or software instructions running as a Platform as a Service (PaaS). The software instructions provide a plurality of graphical user interfaces (GUIs) and / or Application Program Interfaces (“API”). The software instructions may also include call and / or communicate with a variety of other modules, networks and systems, c.g., a digital watermark decoder, digital asset repository, digital watermark resolver, distributed ledgers, databases and / or data records, and account management modules. Of course, the digital asset validator system functionality, operations and algorithms can be realized in hardware, software or a combination of hardware and software. For example, the digital asset validator described above may be implemented as software instructions stored in a memory and executed in one or more processors (including both software and firmware instructions), implemented as digital logic circuitry in a special purpose digital circuit, or combination of instructions executed in one or more multi-core processors, one or more parallel processors and / or one or more digital logic circuit modules. The digital asset validator can also be implemented as a browser plugin, an SDK embedded into a web page (e.g., via Javascript or WebAssembly), an online service, or an application tool plugin (e.g., a Photoshop plugin) or as a mobile application.

[0139] Upon receiving a digital asset, as shown in FIG. 10, the digital asset validator determines whether a digital watermark is embedded within (or should be embedded within) the digital asset. The digital asset itself may include clues indicating an expected presence of a digital watermark. In a first implementation, where the digital asset includes an image or video, the image or video includes a visible logo. (Of course, as mentioned above, our inventive systems, methods and technology can be used to secure other digital assets, c.g., audio, text, PDFs, graphics, digital art and NFTs, e-books, virtual property, video game assets, artwork and 3D models, etc.) Logo (or “icon”) location and / or recognition algorithms analyze the image or video to locate and / or recognize the visible logo. See assignee’s US Patent No. 10,853,903, which is hereby incorporated herein in its entirety, for details regarding icon recognition. Alternatively, a machine-learning system or a deep learning model (such as a convolutional neural network (“CNN”) or recurrent neural networks (RNN)) can be trained to help detect the presence of an icon or logo. The presence of a logo indicates that a digital watermark is expected and triggers digital watermark detection. The logo may also provide indicia or a reference point that can be used to refine or support rotation, scale, and translation. This information can be used to detect an embedded digital watermark. In other cases, the logo carries a plurality of bits, e.g., via digital watermarking, which are geometrically registered via reference to a geometric structure of the logo. The plurality of bits can be decoded and used to identity the logo, a type of expected digital watermarking within the digital asset, or even a digital watermark embedding vendor.

[0140] In a second implementation, upon encountering a digital asset, the digital asset validator calls or communicates with a digital watermark decoder. The digital watermark decoder can reside locally with respect to the digital asset validator or can be remotely called via a communications network (e.g., the internet) as is shown in FIG. 9. For example, the digital watermark decoder can reside locally with respect to either the digital asset validator or the digital watermark resolver. There are advantages to the digital asset validator conducting digital watermark detection locally. For example, a network connection is not required, allowing validation in an “off-line” mode. Local detection also reduces network traffic to a remotely located digital watermark detector. Regardless of its location, the digital watermark detector determines whether a digital watermark is “detected” within the digital asset. The term “detected” at this processing stage can mean several different things. In a first case, the digital watermarking includes a synchronization component (or synchronization “pattern” or “signal”) as discussed above in Section I. Detection can be satisfied if the synchronization component is recognized or detected within the digital asset. Further, detection of a synchronization component could provide a 1 -bit indicator to signal that a manifest is expected. A local or remotely located digital watermark decoder can be invoked for further processing, e.g., payload decoding. In a second case, the digital asset includes multiple layers of digital watermarking. That is, the digital asset includes at least two digital watermarks. The first digital watermark is embedded using an open or publicly known watermarking protocol, and carries a notice (e.g., 1-16 bits of payload data). The notice may indicate the expected presence of a second digital watermark, which is embedded using a secret or private digital watermarking protocol. Or the notice may carry a digital watermark Vendor ID (discussed more below), digital watermark decoder / protocol identifier and / or a notification signalling the expected presence of manifest. Detecting and decoding the notice prompts the digital asset validator to invoke (e.g., execute locally or called remotely) a corresponding second digital watermark detector to decode the expected second digital watermark. In a third case, the detecting includes both detection and decoding of a plural-bit payload carried by a digital watermark. The presence of a decoded payload indicates an association with a manifest. We’ll finish describing the flow in FIG. 10 under the assumption of operating under this third case. (But it should be recognized that the described digital asset validation system can be configured to accommodate the first and second cases as well.)

[0141] If a digital watermark is detected and decoded (“yes” in FIG. 10), the digital asset validator determines whether the digital asset includes an accompanying manifest, e.g., locates and reads a JSON file containing the manifest. If the digital asset includes a manifest, the digital watermark validator can determine whether the manifest and a digital watermark - decoded from the digital asset - match in an expected manner. A user can be notified via a GUI of the match determination. For example, the manifest may include an expected digital watermark identifier and a decoded digital watermark identifier can be compared with the expected digital watermark identifier to determine a match. In another example, the digital watermark includes or indexes one or more hashes. These included or indexed one or more hashes are compared against one or more hashes stored in the manifest to determine a match. In still another example, a decoded digital watermark identifier is used to index into a digital asset repository (e.g., housed within or in communication with the digital watermark resolver in FIG. 9) to locate a corresponding stored manifest. The corresponding stored manifest and the digital asset’s manifest are compared, e.g., as discussed above relative to FIGS. 4A-8B. A “match” for any of the above may include a bit-by-bit match, a match within a predetermined error level or threshold, or a partial match based on a pre-defined set of data fields from the manifest.

[0142] Additional matching algorithms are discussed with reference to FIGS. 11 and 12.

[0143] A manifest will typically include a plurality of manifest hashes, e.g., each created in association with a different assertion or action. An example manifest hash, e.g., a c2pa.hash.data compliant hash, may include, e.g.:

[0144] 3. const manifestHash = {

[0145] 4. "c2pa.hash.data": {

[0146] 5. "alg": "sha256",

[0147] 6. "exclusions": [

[0148] 7. {

[0149] 8. "length": 2059148,

[0150] 9. "start": 20

[0151] 10. }

[0152] 11. ],

[0153] 12. "hash": 99oL8hjSdycWfOSyBoczXT7UahqZYrOcPnNPs92xYIA=",

[0154] 13. "name": "jumbf manifest"

[0155] 14. }

[0156] 15. }

[0157] When creating a digitally watermarked digital asset, the manifest hash, or a permutation of the manifest hash, can be used as a symmetric key to encrypt a digital watermark plural bit payload. That is, prior to embedding, the plural-bit payload (“SDW Payload” in FIG. 11), perhaps after convolutional encoding and error correction, is encrypted using the manifest hash as an encryption key. The term “SDW” implies a secure (or private) digital watermark embedding / decoding algorithm. Various encryption schemes can be utilized here, e.g., AES, RSA, Blowfish, Twofish, ECC, ECDSA, etc. The encrypted plural-bit payload is then embedded within the digital image, yielding a watermarked digital image. A new manifest is created (or the manifest is updated) to include a digital watermarking indicator field. In a C2PA context, the manifest may include this field at an “action” level. An example digital watermark manifest field or assertion may include:

[0158] "c2pa. actions": {

[0159] "actions": [

[0160] "action": "c2pa.soft-binding. watermark",

[0161] "parameters": {

[0162] "icon": "https: / / ...",

[0163] "type": "watermark",

[0164] "version": " 1.0.0"

[0165] }

[0166] }

[0167] ]

[0168] }

[0169] The manifest hash fields can also be updated to include a signature and hash after digital watermark embedding. The following JSON document shows how several manifests can be chained together, which each manifest representing a new set of assertions:

[0170] "active_manifest": "urn:uuid:379f5e41 -9067-4cfe-ae91 -9feecdb433b3",

[0171] "manifests" : {

[0172] "urn:uuid:c07f2471-b931-437e-a2c3-9ca7b5b0eb84": { ...},

[0173] "urn:uuid:379f5e41-9067-4cfe-ae91-9feecdb433b3": { ...}

[0174] }

[0175] } Referring back to FIG. 11 (“detector” stage), the digital watermarked digital asset is analyzed by a digital watermark detector (e.g., local with respect to the digital asset validator). The digital watermark detector (also called a decoder) decodes the digital watermarking to obtain the encrypted payload. The digital watermark detector (or the digital asset validator) reads the manifest hash from the manifest, preferably, the hash of the manifest right before the watermarking hash, since this is the hash that was used to encrypt the payload. The digital watermark detector uses the manifest hash as a decryption key to decode the encrypted SDW Payload. A successful decryption indicates that the manifest is unchanged from the time the digital watermarking payload was encrypted. The encrypted payload may have appended CRC bits, which can be used to check the decryption results, to help determine a successful decryption. An unsuccessful decryption indicates that the manifest has been altered since the digital asset was embedded.

[0176] FIG. 12 illustrates a related validation process utilizing digital watermarking. Initially, a digital watermark embedder receives a digital asset, e.g., a digital image. Of course, these techniques would apply to other digital assets including digital video and digital audio. The digital image is associated with a manifest which includes a manifest hash and an Author Public Key (e.g., assigned by a trusted key authority such as Digimarc Corporation, or an authorized digital watermarking Vendor). The manifest hash, or a permutation of the manifest hash, is used as a symmetric key to encrypt a digital watermark plural bit pay load. That is, the plural-bit pay load, perhaps after convolutional encoding and error correction, is encrypted using the manifest hash as a key. FIG. 12 refers to this as an Encrypted SDW Payload. The term “SDW” implies a secure (or private) digital watermark. The Encrypted SDW Payload is then again encrypted, this time using the Author’s Public Key. The now twice-encrypted SDW payload is embedded within the digital image, yielding a watermarked digital image. (A related challenge is Broadcast Encryption, with a goal of proving that two devices, previously unknown to each other, can agree on a common key for secure communications over a one-way communication path. Broadcast encryption allows for devices that may not have even existed when a group of devices was first grouped together to join into this group and communicate securely. New to our context is that a local detector can sporadically connect to download a new key block. Broadcast Encryption is discussed, e.g., at Broadcast Encryption

[0177] Dave Lassalle, GSEC Practical 1.4c, Option 1, Submitted January 12, 2005, SANS Institute, which is hereby incorporated herein by reference in its entirety.

[0178] The watermarked digital image is analyzed by a digital watermark detector, e.g., incorporated into or otherwise locally available to the digital asset validator. The digital watermark detector (also called a decoder) decodes the digital watermarking to obtain the twice encrypted payload. A Private Key, corresponding to the Author’s Public Key, is looked up via the Author’s public key. A decryption attempt is made at this stage with the Private Key. An unsuccessful decryption attempt indicates that the Author’s Public Key is absent or incorrect, and the manifest cannot be validated. A successful decryption at this point moves the process flow to a second decryption process, this time using the manifest hash. As discussed above, the encrypted payload may have appended CRC bits or like error detection bits computed as a function of other payload bits to help determine a successful decryption. The digital watermark detector (or the digital asset validator) reads the manifest hash from the manifest, preferably, the hash right before the watermarking hash, since this is the one used to encrypt the payload. The digital watermark detector uses the manifest hash as a decryption key to decode the encrypted SDW Payload. A successful decryption indicates that the manifest is unchanged from the time the digital watermarking payload was encrypted. An unsuccessful decryption indicates that the manifest has been altered since the digital asset was embedded.

[0179] A related implementation addresses use of an incorrect decryption key (e.g., derived from manifest hash). An incorrect decryption key results in incorrect output from a decryption process. So how does a digital asset validator (operating in an offline mode) know whether the output is incorrect? One local validation process uses a portion (e.g., 128-512 bits, preferably the first 32-64 bits) of a manifest hash submitted when a digital asset is embedded. Consider the following algorithms: On the embedding side:

[0180] • Obtain a manifest hash and calculate a reduced-bit hash of such, e.g., use a portion of the manifest hash (e.g., 32-512 bits, preferably the first 32-64 bits of the hash), or permute the manifest hash to obtain a reduced-bit hash version of such.

[0181] • Using a random payload allocation methodology (e.g., randomly selecting from a listing of payloads), select a second set of bits, e.g., 24-512 bits, preferably 24-128 bits. Append the reduced-bit hash with the second set of bits to yield a generated digital asset identifier (“DSIN”).

[0182] • Determine whether this generated DSIN collides with an existing DSIN value, e.g., by querying a DSIN registry database. If no collision, this generated DSIN is used as a digital watermark payload for the digital asset. (If it does collide, then a next random second set of bits can be picked and check for collisions. Repeat until no collision is found.)

[0183] • Encrypt the generated DSIN using the manifest hash as a key, yielding an encrypted DSIN. Using a digital watermark embedder, embed the encrypted DSIN into the digital asset.

[0184] • Store the encrypted DSIN along with the generated DSIN in a DSIN registry (or in a digital asset repository, in association with the manifest). This allows on-line detectors to look up the generated DSIN when a manifest hash is not provided.

[0185] Local Validation:

[0186] • Upon encountering a watermarked digital asset, and using a digital watermark decoder, decode the encrypted DSIN carried as a digital watermark pay load. • From the manifest JSON file, obtain the manifest hash (the hash prior to the hash generated after digital watermarking). Decrypt the encrypted DSIN using the obtained manifest hash as a decryption key.

[0187] • Generate a reduced bit hash (e.g., the first 32-64 bits) of the obtained manifest hash.

[0188] • Determine whether the generated reduced-bit hash matches the reduced-bit hash carried in the decoded digital watermark payload. o If Yes - the manifest and the digital watermark match. o If No - the manifest and the digital watermark do not match.

[0189] The digital asset validator can carry out an additional validation process to check for manifest swapping. A decoded digital watermark payload can be communicated to a digital watermark resolver or digital asset repository. The payload is used to index into and find a corresponding stored manifest. A stored manifest hash (e.g., one prior to digital watermark embedding) is located within the stored manifest and returned to the digital asset validator. One benefit of returning a stored manifest hash, instead of returning an entire manifest, is that a stored manifest hash can be efficiently compared against a local manifest hash. This can be accomplished, e.g., by retrieving a manifest hash carried by the local manifest and comparing with the stored manifest hash, or by recomputing a manifest hash from the local manifest using the same algorithm used to create the stored manifest hash. Thus, a decoded digital watermark can be used to verify a local manifest without having to retrieve the whole manifest from a digital asset repository.

[0190] The digital asset resolver (including or communicating with a digital asset repository) can also store a plurality of perceptual hashes related to digital assets. For example, a digital image is embedded to include digital watermarking. The digital watermark can be broken into a plurality of signal tiles that repeat across the image. Each tile of the image can have its own perceptual hash, and the hashes can be combined into a single hash, e.g., via a Merkle Tree structure, and stored in the digital asset repository. The Merkel Tree can also represent the spatial location of the tiles, e.g., if the first tile starts in an expected location like the top left block in the image. This allows a digital asset validator or digital watermarking resolver to recompute the individual tile hashes across an image, and then compare the recomputed hashes against the stored hashes to identify any changes. Pinpoint spatial accuracy can be achieved by finding tiles with mismatching hashes. For example, a faked lunar landing image shows a landing vehicle near a small lunar crater. But the original didn’t include the landing vehicle. Thus, the perceptual hashes of tiles depicting the inserted landing vehicle will differ from the original image. The spatial location of the mismatch can be shown to the user. (For digital assets where a digital watermark cannot be detected, a search within a digital asset repository can be conducted using the Merkel Tree hash structure. If matching candidates are found, a digital watermark can be reconstructed with the know payload, allowing for a forensic analysis of the digital asset.)

[0191] Returning to FIG. 10, if the digital watermark validator does not find a digital watermark, the digital asset can be searched to determine if a digital watermark was stripped out. For example, the presence of a logo / icon as discussed above can trigger a forensic search, e.g., carried out by a digital watermark resolver. Otherwise, the digital asset validator can operate according to standard validation protocols, e.g., as outlined in the C2PA Specification, v.1.3, sections 15 & 16, which is incorporated by reference above. For example, the digital asset validator can check manifest assertions for validity and present the information contained in them, and the signature, to the user in a way that they can then make an informed decision about the trustworthiness of the digital content within the digital asset. Alternatively, the digital asset validator may compute a soft- binding hash of the digital asset (e.g., perceptual hash of an image, video, or audio) for comparison, e.g., to a soft binding hash carried in the manifest.

[0192] If a digital watermark is detected (and decoded) from the digital asset, but the digital asset does not include a manifest, a decoded digital watermark pay load can be used to index into a digital asset repository to locate a corresponding stored manifest. For example, the digital asset validator may communicate the decoded digital watermark payload to a digital watermark resolver (FIGS. 9-10), which indexes a digital asset repository to retrieve the corresponding stored manifest. Alternatively, the digital asset validator can communicate directly with a digital asset repository. The stored manifest can be returned in response. The manifest can be reassociated with the digital asset, including creating a new hard-binding perceptual hash and storing the hash and signature in the returned manifest.

[0193] The digital watermark resolver can be implemented as an online platform, e.g., software instructions operating as a Software as a Service (SAS) and / or software instructions operating as a Platform as a Service (PaaS) in a cloud-computing environment. The software instructions provide a plurality of graphical user interfaces (GUIs) and / or Application Program Interfaces (“API”). The software instructions may also include call and / or communicate with a variety of other modules, networks and systems, e.g., one or more digital watermark decoders, one or more digital asset repository, one or more distributed ledgers, databases and / or data records, and / or account management modules. Of course, the digital watermark resolver functionality, operations and algorithms can be realized in hardware, software or a combination of hardware and software. For example, it may be implemented as software instructions stored in a memory and executed in one or more processors (including both software and firmware instructions), implemented as digital logic circuitry in a special purpose digital circuit, or combination of software instructions executed in one or more multi-core processors, one or more parallel processors and / or one or more digital logic circuit modules.

[0194] The digital watermark resolver may implement a common API through which the digital asset validator can request digital watermark detection and / or stored manifest information. An example digital watermark resolver API includes the following: const resolver = new Watermark.Resolver(URI); const decodeResult = {

[0195] "manifest" : { } ,

[0196] "manifestHash": { }

[0197] } const options = { output: '’manifestHashlmanifest" }; * This function accepts an asset (e.g., image) and returns a decodeResult object.

[0198] * @param {file} asset - The asset that needs to be decoded (e.g., image)

[0199] * @param {options} options - Specifies the format of the result as one of:

[0200] * output : manifest - (default) a C2PA manifest

[0201] * output : manifestHash - a C2PA hash of the corresponding manifest

[0202] * ©returns {decodeResult} A decodeResult object containing a manifest or a hash of the manifest

[0203] * / resolver.getManifest({ asset, / / image options / / specify the desired output

[0204] })

[0205] .then((decodeResult) => { console. logfResult: ${ JSON.stringify(result) }')

[0206] })

[0207] .catch((error) => { console. logC An error occurred: ${err}');

[0208] });

[0209] / **

[0210] * This function accepts a digital asset (e.g., image) and checks if there is a digital watermark present from this vendor.

[0211] * @param {file} asset - The asset that needs checked (e.g., image)

[0212] * ©returns {Boolean} result - true = there is a watermark, false = there is no watermark

[0213] * / resolver. watermarkPresent({ asset / / image

[0214] })

[0215] .then((result) => { console. logfResult: ${ JSON.stringify(result) }')

[0216] })

[0217] .catch((error) => { console. logf An error occurred: ${err}');

[0218] }); const matchingResult = { "result": "01112", "description": "text" };

[0219] / **

[0220] * This function verifies that the manifest of an asset and its digital watermark are matching.

[0221] * @param {file} asset - the asset that needs checked (e.g., image)

[0222] * ©returns {matchingResult} matchingResult - 0 = no match, 1 = match,

[0223] * 2 = partial match, i.e., the asset was altered since the watermark was applied

[0224] * / resolver. watermarkAndManifestMatching({ asset / / image

[0225] })

[0226] .then((matchingResult) => { console. log(' Result: ${ JSON.stringify(result) }')

[0227] })

[0228] .catch((crror) => { console. logf An error occurred: $ { err }' );

[0229] });

[0230] Similarly, a digital watermark embedder (or digital watermarking vendor providing embedding services) can provide a public API through which a digital asset can be embedded with digital watermarking. In many cases, the digital watermark resolver can provide embedding services, or is otherwise associated with an associated digital watermark embedder. An example embedder API may include the following: const encoder = new Watermark.Encoder(URI);

[0231] / **

[0232] * This function accepts an asset and returns a watermarked asset.

[0233] * If the asset is already watermarked the encoder should create a new watermark and attach link it to the new manifest.

[0234] * @ param {file} asset - asset to be watermarked

[0235] * @param {object} manifest - a C2PA manifest for the asset (optional)

[0236] * ©returns {file} asset - the watermarked asset

[0237] * / encoder, encode} { asset, manifest

[0238] })

[0239] ,then((asset) => { console.logCResult: ${JSON.stringify (asset)}')

[0240] })

[0241] .catch((error) => { console.log('An error occurred: ${err}');

[0242] });

[0243] A centralized, decentralized or distributed registry can be maintained to help identify approved or registered digital watermark resolvers (and / or registered digital watermark decoders). An objective of such a registry is to ensure that digital asset validators have a finite and well-known list of digital watermark resolvers and / or watermark decoders that they can identify to facilitate digital watermark detection. A registry of approved digital watermark resolvers may also help ensure that a digital watermark cannot be added by any actor which could lead to security issues (e.g., a validator accepting to swap a manifest). An example registry may include plural fields, e.g., including:

[0244] This registry can reside on a decentralized network, or alternatively, can be a living doc, e.g., maintained on GitHub. If a digital asset includes a digital watermark, a corresponding Vendor ID can be stored in a manifest action or assertion field. Storing such identifying information allows a digital asset validator to identify which digital watermark resolver (or which digital watermark detector) should be called or invoked to help facilitate digital watermark decoding. If a digital asset does not include a Vendor ID, and there are multiple, approved digital watermarking embedders / detectors in the ecosystem, a digital asset validator can cycle through digital watermark resolvers (or, if locally stored, different digital watermark detectors) until a successful digital watermark decode is carried out. For example, each of the different approved digital watermark resolvers (or detector) may include different private watermarking algorithms and protocols, where a first digital watermark resolver (or detector) may not necessarily be able to read or decode a digital watermark embedded by a second digital watermark resolver (or corresponding embedder), and so on. An example manifest action is shown below, including a Vendor ID. const action = {

[0245] "c2pa. actions": {

[0246] "actions": [ {

[0247] "action": "c2pa.soft-binding. watermark",

[0248] "parameters": {

[0249] "icon": "https: / / ...",

[0250] "type": "watermark",

[0251] "vendorld": "VENDOR_ID",

[0252] "version": " 1.0.0"

[0253] }

[0254] }

[0255] ] }

[0256] Trustable Identities for Manifest Signing

[0257] One of the limitations of current assets provenance systems is the lack of trustable identities to be connected with manifests. Certificates can be used to sign claims but understanding to whom these certificates belong to and that they can be trusted remains unaddressed. Consider a related system shown in FIG. 13 that helps ensure trustable identities for digital assets and verification of such assets. Digital asset creators establish an identity with an online registry, e.g., incorporated into a digital watermark resolver such as discussed above with respect to FIGS. 9 and 10 and / or an online verification system as discussed above with respect to FIGS. 4A-8B. The registry provides initial Know Your Customer (KYC) procedures with different levels of trust such as identity verification, e.g., using social media verification (oAuth), driver’s license or passport identification, in-person calls, etc., either directly or via integrated third-party services. Successful verification results in the creator credentials. The creator credentials can be used to sign digital assets and maintain a repository of assets / manifest per creator. As part of the creator credentials, a decentralized identifier (“DID”) is created using, e.g., a blockchain integration hub. The blockchain integration hub may include, e.g., a plurality of APIs to communicate with different blockchains. Alternatively, the creator can connect a wallet (e.g., via Metamask) linked to a DID, see, e.g., FIGS. 14-17 in the online verification system environment described above in FIGS 4A-8B. FIGS. 14-17 illustrate GUIs that allow a user to link a wallet, and sign digital assets (e.g., via a manifest) with a DID. Below is an example DID as a JSON document:

[0258] {

[0259] '©context': [

[0260] 'https: / / www.w3.org / ns / did / vl',

[0261] 'https: / / w3id.org / security / suites / secp256klrecovery-2020 / v2'

[0262] ], id: 'did:ethr:0xb9c5724089438a327f09197987fl6f9e5d976e8a', verificationMethod: [ id: 'did:ethr: 0xb9c5724089438a327f09197987fl6f9e5d976e8a#controller', type: 'EcdsaSecp256klRecoveryMethod2020', controller: ’did:ethr:0xb9c5724089438a327f09197987fl6f9e5d976e8a', blockchainAccountld: 'eipl55:l:0xb9c5714089478a327f09197987fl6f9e5d936e8a'

[0263] I

[0264] ], assertionMethod: [

[0265] 'did:ethr:0xb9c5714089478a327f091979871’16f9e5d936e8a#controller'

[0266] ], authentication: [ 'did:ethr:0xb9c5714089478a327f09197987fl6f9e5d936e8a#controller'

[0267] ]

[0268] }

[0269] In one embodiment, the DID is not created but derived from a wallet key, e.g., via a smart contract. The DID optionally can be anchored to a distributed ledger either directly or by signing a VC acting as a claim that the KYC was executed successfully. As an example, the following VC (represented as a JSON document) binds an identity verified via a KYC to a DID:

[0270] "©context": [

[0271] "https: / / www.w3.org / 2018 / credentials / vl"

[0272] ],

[0273] "credentialSchema": { "id": "https: / / beta.api.schemas.serto.id / vl / public / program-completion- certificate / 1.0 / j son- schema.j son" ,

[0274] "type": "JsonSchemaValidator2018"

[0275] },

[0276] "credentialSubject": {

[0277] "id": "did:ethr:0xl:0x7dla4449bl30b6bl56fe8e93fd70fca25526cd26",

[0278] "firstNamc": "John",

[0279] "Surname": "Doe",

[0280] "thirdPartyJobNumber": "123e4567-e89b-12d3-a456-426614174001"

[0281] },

[0282] "issuanceDate": "2023-09-04T10:29:41.905Z",

[0283] "issuer": "did:ethr:0xl:0xf9879db6113565E591D41E6cl0845C72cEf35A82",

[0284] "proof": {

[0285] "created": "2023-09-04T10:29:41.905Z",

[0286] "eip712": {

[0287] "domain": {

[0288] "chainld": 1,

[0289] "name": "VerifiableCredential",

[0290] "version": "1"

[0291] "primaryType": "VerifiableCredential",

[0292] "types": {

[0293] "CredentialSchema": [

[0294] 1

[0295] I

[0296] },

[0297] "proofPurpose" : "assertionMethod",

[0298] "proofValue":

[0299] "0x827a351030049a8b9bc5224fl05e2ea6d9139fb7c620de8669c7c5103eb29939308fdbf

[0300] 3832bbf9f58542f8c58575696b800e361f23202fdl32311105417e4e31b",

[0301] "type" : " EthereumEip712Signature2021 " ,

[0302] "verificationMethod" :

[0303] "did:ethr:Oxl :0x7dl a4449b 130b6bl 56fe8e93fd70fca25526cd26#controller"

[0304] },

[0305] "type": [

[0306] "VerifiableCredential" ,

[0307] "ThirdParty UserCredential "

[0308] 1

[0309] }•

[0310] The resulting verifiable creator credential can be used to sign digital assets. For example, the C2PA specification supports the inclusion of W3C Verifiable Credentials (VC) into a C2PA Manifest to represent a human or organization that may be directly associated with an asset in some way, such as the creator or publisher. The VC can be signed using the private key connected to the DID / wallet and included in a manifest. Similarly, Verifiable Credentials could also be used to record modifications applied to an asset or any other claim related to an asset. As an example, the example JSON document below represents a claim attesting that the owner of the DID signed a manifest:

[0311] "credential_store": [

[0312] "@ context": [

[0313] "https: / / www.w3.org / 2018 / credentials / vl"

[0314] ],

[0315] "credentialSchema" : {

[0316] "id": "https: / / beta.api.schemas.serto.id / vl / public / program-completion- certificate / 1.0 / j son- schema.) son" ,

[0317] "type": "JsonSchemaValidator2018"

[0318] },

[0319] "credentialSubject": {

[0320] "id": "did:ethr:0xl:0x7dla4449bl30b6bl56fe8e93fd70fca25526cd26",

[0321] "type": "Regular User"

[0322] },

[0323] "issuanceDate": "2023-09-04T10:29:41.905Z",

[0324] "issuer": "did:cthr:0xl:0x7dla4449bl30b6bl56fc8c93fd70fca25526cd26",

[0325] "proof": {

[0326] "created": "2023-09-04T10:29:41.905Z",

[0327] "eip712": {

[0328] "domain": {

[0329] "chainld": 1,

[0330] "name" : " VerifiableCredential" ,

[0331] "version": "1"

[0332] },

[0333] "primaryType": "VerifiableCredential",

[0334] "types": {

[0335] "CredentialSchema" : [

[0336] J,

[0337] "CredentialSubject": [

[0338] ],

[0339] "EIP712Domain": [ ],

[0340] "Proof": [

[0341] ],

[0342] "VerifiableCredential": [

[0343] ],

[0344] "credentialSchema": [

[0345] ],

[0346] "credentialSubject": [

[0347] ],

[0348] "proof": [

[0349] ]

[0350] }

[0351] },

[0352] "proofPurpose": "assertionMethod",

[0353] "proofValue":

[0354] "0x827a351030049a8b9bc5224fl05e2ea6d9139fb7c620de8669c7c5103eb29939308fdbf

[0355] 3832bbf9f58542f8c58575696b800e361f23202fdl32311105417e4e31b",

[0356] "type" : "EthcrcumEip712Signaturc2021 ",

[0357] "verificationMethod" :

[0358] "did:ethr:0xl:0x7dla4449bl30b6bl56fe8e93fd70fca25526cd26#controller"

[0359] I,

[0360] "type": [

[0361] "VerifiableCredential" ,

[0362] "ThirdPartyUserCredential"

[0363] J

[0364] } The process of validation proceeds as discussed above with respect to FIG. 9-10 (or with respect to the online verification system described above with reference to FIGS. 4A-8B). An additional check can be made by a digital asset validator to verify the creator credentials, e.g., by following the verification method specified in a given VC or by consulting a distributed ledger and / or the online registry.

[0365] Concluding Remarks

[0366] The technology, modules, functionality, methods, processes, and systems described above may be implemented in hardware, software or a combination of hardware and software. For example, the verification system and / or the digital asset validation system described above may be implemented as instructions stored in a memory and executed in one or more processors (including both software and firmware instructions), implemented as digital logic circuitry in a special purpose digital circuit, or combination of instructions executed in one or more multi-core processors, one or more parallel processors and / or one or more digital logic circuit modules. For example, the verification system and / or the digital asset validation system described above may be implemented as instructions stored in a memory and executed in one or more multi-core processors (including both software and firmware instructions), implemented as digital logic circuitry in a special purpose digital circuit, or combination of instructions executed in one or more multi-core processors, one or more parallel processors and / or one or more digital logic circuit modules. The technology, modules, methods, services, functionality, and processes described above may be implemented in software programs executed from a system’s memory (a non-transitory computer readable medium such as an electronic, solid-state, optical and / or magnetic storage memory). When the software is executed, its software instructions cause one or more processors, one or more multi-core processors, one or more parallel processors to execute or carry out the various acts or functionality scripted therein. The methods, instructions and circuitry operate on electronic signals, or signals in other electromagnetic forms. These signals further represent physical signals like image signals captured in image sensors, audio captured in audio sensors, as well as other physical signal types captured in sensors for that type. These electromagnetic signal representations are transformed to different states as detailed above to detect signal attributes, perform pattern recognition and matching, determine relative attributes of Scans, etc.

[0367] Example hardware and communication flow between electronic devices, networks and third-party services (e.g., provided by cloud-based computers as cloud-based services) is further detailed in our PCT Application No. PCT / US22 / 50767 (published as WO 2023 / 096924), which are each hereby incorporated here by reference including all drawings, particularly relative to FIGS. 14, 15 and 16 of that PCT application, and we expressly intend to use those described computing environments with the technology described in the present patent document as if reproduced word for word herein. For example, the digital watermark embedder and detector may be hosted on a cloud resource depicted in one or more those figures and accessed via one or more APIs or graphical interfaces.

[0368] Having described and illustrated the principles of the technology with reference to specific implementations, it will be recognized that the technology can be implemented in many other, different, forms. To provide a comprehensive disclosure without unduly lengthening the specification, applicants incorporate by reference - in their entirety - the patents and patent applications referenced above, including all drawings, and any appendices.

[0369] The particular combinations of elements and features in the above-detailed embodiments are exemplary; the interchanging and substitution of these teachings with other teachings in this and the incorporated-by-reference patents / applications are also contemplated. Any headings used in this document arc for the reader’s convenience and are not intended to limit the disclosure. We expressly contemplate combining the subject matter under the various headings.

Claims

What is claimed is:

1. A signal processing method to detect manifest swapping, said signal processing method comprising: accessing a digital asset comprising human perceptual elements, the digital asset comprising digital watermarking embedded therein through modifications of the human perceptual elements, the digital watermarking carrying a plural-bit payload comprising an identifier, the digital asset being associated with a manifest; analyzing, using a digital watermark decoder, the digital asset to decode the identifier, said analyzing yielding a decoded identifier; extracting the manifest to obtain an extracted manifest; accessing a digital asset repository to obtain a stored manifest associated with the decoded identifier; generating a comparison between the stored manifest and the extracted manifest; based on the comparison, detecting whether the extracted manifest has been swapped with an alternative manifest.

2. The signal processing method of claim 1 in which the extracted manifest comprises a first perceptual hash of the digital asset, and in which the comparison indicates that the first perceptual hash differs from a second perceptual hash carried by the stored manifest3. The signal processing method of claim 2 further comprising providing the stored perceptual hash for association with the digital asset.

4. The signal processing method of claim 1 in which the digital watermarking payload further comprises a plain text address, or a Uniform Resource Identifier (URI), or a reduced-bit version of either.

5. The signal processing method of claim 1 in which the digital asset comprises imagery, and wherein digital watermarking embedded therein through modifications to color values representing a portion of the imagery.

6. The signal processing method of claim 1 in which said generating a comparison between the stored manifest and the extracted manifest compares a character count of each manifest.

7. The signal processing method of claim 1 in which the digital watermarking comprises a synchronization component which facilitates synchronization of the digital watermark relative to a base state.

8. The signal processing method of claim 1 in which said each of the stored manifest and the extracted manifest comprises a plurality of data fields, and wherein said generating a comparison between the stored manifest and the extracted manifest compares each of the plurality of data fields from the extracted manifest against each of the plurality of data fields from the stored manifest.

9. The signal processing method of any of the preceding claims in which the stored manifest comprises a C2PA manifest, and the extracted manifest comprises an extracted C2PA manifest.

10. A signal processing apparatus configured to detect manifest swapping, said signal processing apparatus comprising: means for accessing a digital asset comprising human perceptual elements, the digital asset comprising digital watermarking embedded therein through modifications of the human perceptual elements, the digital watermarking carrying a plural-bit payload comprising an identifier, the digital asset being associated with a manifest;a digital watermark decoder configured for analyzing the digital asset to decode the identifier, said digital watermark decoder yielding a decoded identifier; means for extracting the manifest to obtain an extracted manifest; means for accessing a digital asset repository to obtain a stored manifest associated with the decoded identifier; means for generating a comparison between the stored manifest and the extracted manifest; means for detecting whether the extracted manifest has been swapped with an alternative manifest.

11. The signal processing apparatus of claim 10 in which said each of the stored manifest and the extracted manifest comprises a plurality of data fields, and wherein said means for generating a comparison between the stored manifest and the extracted manifest compares each of the plurality of data fields from the extracted manifest against each of the plurality of data fields from the stored manifest.

12. The signal processing apparatus of claim 10 in which the extracted manifest comprises a first perceptual hash of the digital asset, and in which the comparison indicates that the first perceptual hash differs from a second perceptual hash carried by the stored manifest.

13. The signal processing apparatus of claim 12 further comprising means for providing the stored perceptual hash for association with the digital asset.

14. The signal processing apparatus of claim 10 in which the digital watermarking payload further comprises a plain text address, or a Uniform Resource Identifier (URI), or a reduced-bit version of either.

15. The signal processing apparatus of claim 10 in which the digital asset comprises imagery, and wherein the digital watermarking is embedded therein through modifications to color values representing at least a portion of the imagery.

16. The signal processing apparatus of claim 10 in which said means for generating a comparison between the stored manifest and the extracted manifest compares a character count of each manifest.

17. The signal processing apparatus of claim 10 in which said means for generating a comparison between the stored manifest and the extracted manifest generates a hash of the stored manifest with a hash of the extracted manifest, and then compares each of the hashes to one another.

18. The signal processing apparatus of any of claims 10-17 in which the stored manifest comprises a C2PA manifest, and the extracted manifest comprises an extracted C2PA manifest.

19. A signal processing method comprising: accessing a digital asset comprising human perceptual elements, the digital asset comprising a digital watermark embedded therein during digital asset generation by a deep learning or machine-learning process, the digital watermark including a plural-bit payload comprising an identifier, the digital asset being associated with a manifest but not including the manifest during said accessing; analyzing, using a digital watermark decoder, the digital asset to decode the identifier, said analyzing yielding a decoded identifier; accessing a digital asset repository to obtain a stored manifest associated with the decoded identifier; obtaining the stored manifest from the digital asset repository and associating the stored manifest with the digital asset.

20. A signal processing method comprising: obtaining a manifest hash that is associated with a digital asset, the manifest hash contained within a manifest that is associated with the digital asset, and generating a reduced-bit hash of the manifest hash; selecting a set of bits from a plurality of sets of bits as a portion of an identifier; appending the reduced-bit hash with the set of bits to yield a generated digital asset identifier (“DSIN”); encrypting the generated DSIN with the reduced-bit hash of manifest hash as a key, said encrypting yielding an encrypted DSIN. using a digital watermark embedder, embedding the encrypted DSIN into the digital asset.

21. The signal processing method of claim 20 further comprising: determining whether the generated DSIN collides with an existing DSIN value by querying a DSIN registry database.

22. The signal processing method of claim 21 further comprising, upon encountering a collision randomly or pseudo-randomly selecting another set of bits from the plurality of sets of bits as the portion of an identifier.

23. The signal processing method of claim 20 further comprising: storing the encrypted DSIN along with the generated DSIN in a DSIN registry, in which the encrypted DSIN and the generated DSIN arc associated with the manifest,24. The signal processing method of claim 20 in which said generating a reduced-bit hash of the manifest hash comprises of selecting a portion of the manifest hash.

25. The signal processing method of claim 24 in which the portion of the manifest hash comprises a first portion of the manifest hash or comprises a permuted version of the portion of the manifest hash.

26. The signal processing method of claim 25 in which the first portion of the manifest hash comprises 32-512 bits.

27. The signal processing method of claim 20 in which said selecting a set of bits from a plurality of sets of bits comprises randomly or pseudo-randomly selecting a set of bits from the plurality of sets of bits.

28. A signal processing method comprising: obtaining a digitally watermarked digital asset, in which the digitally watermarked digital asset comprises a manifest associated therewith, the manifest comprising a manifest hash, the digital watermark comprising an encrypted digital asset identifier (“DSIN”); using a digital watermark decoder, decoding the encrypted DSIN; obtaining the manifest hash from the manifest, and decrypting the encrypted DSIN using the manifest, or a portion of the manifest hash, and a decryption key, said decrypting yielding a decrypted DSIN; generating a reduced-bit hash from the obtained manifest hash; determining whether the generated reduced-bit hash matches a portion of the decrypted DSIN.

29. The signal processing method of claim 28 further comprising outputting a successful validation indication when the generated reduced-bit hash matches the portion of the decrypted DSIN.

30. The signal processing method of claim 28 further comprising outputting an unsuccessful validation indication when the generated reduced-bit hash does not match the portion of the decrypted DSIN.

31. The method of claim 28 in which the digital watermark decoder utilizes a trained deep learning model or a trained machine-learning model.

32. The method of claim 31 in which the deep learning model or machinelearning model comprises a convolutional neural network (CNN) or recurrent neural networks (RNN) model.

33. An image processing method comprising: obtaining a digital asset comprising imagery in which the imagery comprises digital watermarking embedded therein; generating a heatmap of digital watermark detectability associated with the digital watermarking, said generating a heatmap yielding a generated heatmap; accessing a data repository to obtain a stored heatmap corresponding to digital watermark detectability corresponding to the digital asset; comparing the stored heatmap, or values underlying the stored heatmap, with the generated heatmap, or with values underlying the generated heatmap, to determine whether the imagery has changed since the stored heatmap was stored in the data repository.

34. The image processing method of claim 33 in which the comparing yields a spatial location of imagery changes.

35. The image processing method of claim 33 in which the digital watermarking comprising a plurality of signal components, and the heatmap is associated with detectability metrics associated with the plurality of signal components.

36. The image processing method of claim 36 in which the plurality of signal components comprises a message component and a synchronization component.

37. The image processing method of claim 35 in which the digital watermarking is embedded within plural portions of the imagery.

38. A signal processing method comprising: accessing a digital asset comprising human perceptual elements, the digital asset comprising a first digital watermark embedded therein, the first digital watermark including a notice that indicates an expected presence of a second digital watermark embedded within the digital asset, the second digital watermark signal comprising a plural-bit payload, in which the digital asset is associated with a manifest but does not include the manifest during said accessing; analyzing, using a first digital watermark decoder, the digital asset to obtain the notice; based on obtaining the notice, analyzing, using a second digital watermark decoder, the digital asset to obtain the plural-bit payload; accessing a digital asset repository to obtain a stored manifest associated with the plural-bit payload; obtaining the stored manifest from the digital asset repository and associating the stored manifest with the digital asset.

39. The method of claim 38 in which the notice comprises a single bit of data.

40. The method of claim 38 in which the notice comprises 1-16 bits of data.

41. The method of claim 38 in which the notice comprises a vendor identifier (ID), and wherein said method further comprises, prior to said analyzing, using a seconddigital watermark decoder, determining that the second digital watermark decoder is associated with the vendor ID.

42. The method of claim 1 further comprising communicating with a remotely located resource to access the second digital watermark decoder.

Images