Dynamic visual watermarking for streaming video

Abstract

Systems and methods provide functionality in streaming media and video distribution by acquiring streaming video including video on demand and live streams and applying dynamic visual watermarks as needed by content owners or broadcasters. The dynamic visual watermark may be a visible watermark generated for each stream, viewer, or user. The technology can maintain the quality of the stream, makes anti-piracy implementation feasible (generally about 1% of the cost of the traditional technology), is suitable for live stream, cooperates with the selection of any cloud platform provider, DRM provider or other watermark provider, and can work without the need for large-scale settings and integration. The video streaming solution provides a tool for the broadcaster, and each stream can be marked as needed. Video streaming solutions may be burnt on the server side and may be platform independent of cloud platforms, software, and networks.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to systems and methods for applying dynamic visual watermarks to streaming media videos. Such dynamic visual watermarks may be useful, for example, in anti-piracy applications, as well as in branding, marketing, advertising, and the like.

[0002] Additionally, embodiments of the present invention are able to apply dynamic (also known as session-based) processing, such as forensic watermarking, metadata, and other transcoding of streaming media (audio and video), for both pre-recorded and live applications. Background Art

[0003] The following background information may provide examples of specific aspects of the prior art (such as but not limited to methods, facts or common knowledge), and while these examples are expected to help further educate the reader about other aspects of the prior art, they should not be construed as limiting the present invention or any embodiment thereof to anything expressly or implied therein or inferred therefrom.

[0004] Pirated content is content that is broadcast or sold without the permission of the content producer, content distributor or authorized broadcaster. According to research, the global streaming or video content market will grow from $93 billion in 2018 to more than $135 billion in 2022. According to data, about 39% of this is pirated content.

[0005] Virtually all long-form video content, premium content, and live streams from streaming services large and small are targeted for piracy, and pre-release content is also prematurely “leaked” to the public. This has a huge impact on the profitability of content owners and broadcasters.

[0006] While forensic watermarking techniques exist to track the original leaker or pirate, visible watermarking is often used first to deter piracy. However, typical visible watermarks do little to deter piracy. Traditional solutions that apply visible watermarks directly to video (“burn-in”) are cumbersome and costly, typically costing up to $1 per minute of content watermarked. Additionally, traditional implementations are cumbersome, inflexible, and unable to provide additional anti-piracy protection (e.g., selection of forensic watermarking partners, digital rights management (DRM), lack of compatibility with existing workflows and video platform integrations). Existing solutions also degrade video stream quality and require additional complex components to make them work. Additionally, there is currently no solution that can dynamically embed visible watermarks for scalable live streaming.

[0007] Furthermore, typical traditional solutions that perform some level of "dynamic" or "session-based" or "user" visible watermarking do so on a "client" level. In other words, either on the player or the application. Since they are an overlay on the video stream, they can be hacked and filtered. The overlay can be corrupted, or if the video stream is intercepted, the overlay will not be displayed.

[0008] In view of the foregoing, there is a need for anti-piracy methods that can be used for streaming video content, including live streams, in a computationally resource efficient manner. Summary of the invention

[0009] Embodiments of the present invention provide systems and methods for applying dynamic visual watermarks to streaming video (eg, live streaming, video on demand, etc.).

[0010] According to an embodiment of the present invention, a system for providing dynamic watermarks to streaming media is provided, comprising: a streaming proxy server and a visual watermark encoding service, which receive a streaming media bitstream from a stream packager and receive watermark details from a content provider, wherein the streaming proxy server and the visual watermark encoding service provide a unique bitstream with the dynamic watermark to a user; and before transmitting the dynamic watermark to the user's device, the dynamic watermark is applied to the bitstream server side of the streaming media bitstream.

[0011] According to a further embodiment of the present invention, there is provided a method for providing dynamic processing to streaming data, comprising: receiving the streaming data from a content provider; receiving dynamically processed data from the content provider; preparing a bitstream of the streaming data to which the dynamic processing is applied; and transmitting the streaming data to which the dynamic processing is applied to a user.

[0012] According to a further embodiment of the present invention, a method for providing a dynamic watermark to a video bitstream is provided, comprising: parsing a manifest of the video bitstream; downloading video segments to a streaming proxy server and a visible watermark encoding service; adding the dynamic watermark to the video segments; reformatting the video segments to which the dynamic watermark is applied; constructing a new manifest to play a unique bitstream to which the dynamic watermark is applied; and transmitting the unique bitstream to a user's video viewing device.

[0013] These and other features, aspects and advantages of the present invention will become better understood with reference to the following drawings, description and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Some embodiments of the present invention are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like reference numerals may indicate similar elements.

[0015] FIG1 shows a flow chart of a typical blocker-level protection workflow according to the prior art for a live stream with DRM and forensic A / B dynamic manifests;

[0016] Figure 2 A flow chart showing a video publishing / streaming solution for live streaming with DRM, forensic A / B dynamic manifests and visible watermarks according to an exemplary embodiment of the present invention is shown;

[0017] Figure 3 A flow chart showing a video publishing / streaming solution according to an exemplary embodiment of the present invention for a live stream with only a visible watermark;

[0018] FIG4 shows a flow chart of a typical masker-level protection workflow according to the prior art for video-on-demand streaming with DRM and forensic A / B dynamic manifests;

[0019] Figure 5 A flow chart showing a video publishing / streaming solution for video on demand streaming with DRM, forensic A / B dynamic manifests and visible watermarks according to an exemplary embodiment of the present invention is shown;

[0020] Figure 6 A flow chart illustrating a video publishing / streaming solution for video-on-demand streaming with forensic A / B dynamic manifests and visible watermarks according to an exemplary embodiment of the present invention is shown;

[0021] Figure 7 A flow chart showing a video publishing / streaming solution according to an exemplary embodiment of the present invention for video-on-demand streaming with DRM and visible watermarking;

[0022] Figure 8 a flow chart showing details of an exemplary process for adding a visual watermark to a video stream according to an exemplary embodiment of the present invention; and

[0023] Fig. 9 A flow chart showing details of another exemplary process for adding a visual watermark to a video stream according to an exemplary embodiment of the present invention.

[0024] Unless otherwise indicated, the illustrations in the drawings are not necessarily drawn to scale. The present invention and its various embodiments can now be better understood by the following detailed description, in which the illustrated embodiments are described. It should be clearly understood that the illustrated embodiments are presented as examples and are not limitations of the present invention as ultimately defined in the claims. DETAILED DESCRIPTION

[0025] The terms used herein are only used to describe specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. The singular forms "one", "an" and "the" used herein are intended to include plural forms as well as singular forms, unless the context clearly indicates otherwise. It should be further understood that when the terms "include" and / or "comprise" are used in this specification, the presence of the features, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, steps, operations, elements, components and / or groups thereof is not excluded.

[0026] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. It should also be understood that terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with the meaning in the context of the relevant technology and the present disclosure, and should not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0027] In describing the present invention, it should be understood that many techniques and steps are disclosed. Each of these has its own advantages, and each can also be used in combination with one or more, or in some cases all, of the other disclosed techniques. Therefore, for the sake of clarity, this description will avoid unnecessary repetition of every possible combination of the various steps. However, the specification and claims should be understood as such that these combinations are fully within the scope of the present invention and the claims.

[0028] In the following description, for the purpose of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without these specific details.

[0029] This disclosure should be considered as illustrative of the invention and is not intended to limit the invention to the specific embodiments shown in the following figures or described.

[0030] It is well known to those skilled in the art that many careful considerations and compromises must generally be made when designing the optimal configuration of a commercial implementation of any system, and particularly embodiments of the present invention. Commercial implementations according to the spirit and teachings of the present invention may be configured as needed for a particular application, whereby those skilled in the art may utilize their average skill and known techniques to appropriately omit, include, adjust, mix and match, or improve and / or optimize any aspect(s), feature(s), function(s), result(s), component(s), method(s), or step(s) of the teachings associated with any described embodiments of the present invention to achieve a desired implementation that meets the needs of a particular application.

[0031] Devices or system modules that are at least generally in communication with each other need not be in continuous communication with each other unless explicitly stated otherwise. In addition, devices or system modules that are at least generally in communication with each other may communicate directly or indirectly through one or more intermediaries.

[0032] The description of an embodiment with multiple components in communication with each other does not imply that all such components are required. Instead, various optional components are described to illustrate various possible embodiments of the present invention.

[0033] A "computer" or "computing device" may refer to one or more devices and / or one or more systems that can accept structured input, process the structured input according to specified rules, and produce processed results as output. Examples of computers or computing devices may include: computers; fixed and / or portable computers; computers with a single processor, multiple processors, or multi-core processors, which may run in parallel and / or non-parallel; supercomputers; mainframes; supermicrocomputers; minicomputers; workstations; microcomputers; servers; clients; interactive televisions; network equipment; telecommunications equipment with access to the Internet; hybrid combinations of computers and interactive televisions; portable computers; tablet personal computers (PCs); personal digital assistants (PDAs); portable phones; dedicated hardware for emulating computers and / or software, such as, for example, digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), one chip, multiple chips, systems on a chip, or chipsets; data acquisition devices; optical computers; quantum computers; biological computers; and, in general, devices that can accept data, process data according to one or more stored software programs, generate results, and typically include input units, output units, storage units, arithmetic units, logic units, and control units. "Computers" may include dedicated hardware implementations, including, but not limited to, application specific integrated circuits, programmable logic arrays, and other hardware devices, which may also be configured to implement the methods described herein. Furthermore, the "computer" may include alternative software implementations, including but not limited to distributed processing or component / object distributed processing, parallel processing or virtual machine processing, which may also be constructed to implement the methods described herein.

[0034] "Software" or "application" may refer to the specified rules for operating a computer. Examples of software or applications may include code segments in one or more computer-readable languages; graphical and / or textual instructions; applets; precompiled code; interpreted code; compiled code; and computer programs.

[0035] The exemplary embodiments described herein can be implemented in an operating environment including computer-executable instructions (e.g., software) installed on a computer, in hardware, or in a combination of software and hardware. Computer-executable instructions can be written in a computer programming language or embodied in firmware logic. If written in a programming language that conforms to a recognized standard, these instructions can be executed on a variety of hardware platforms and used to interface with a variety of operating systems. Although not limited thereto, computer software program code for performing operations of various aspects of the present invention may be written in any combination of one or more suitable programming languages, including object-oriented programming languages ​​and / or traditional procedural programming languages, and / or such programming languages, such as, for example, Hypertext Markup Language (HTML), Dynamic HTML, Extensible Markup Language (XML), Extensible Stylesheet Language (XSL), Document Style Semantics and Specification Language (DSSSL), Cascading Style Sheets (CSS), Synchronized Multimedia Integration Language (SMIL), Wireless Markup Language (WML), Java.TM., Jini.TM., C, C++, C#, Objective C, Smalltalk, Python, Perl, UNIX Shell, Visual Basic or Visual Basic Script, Virtual Reality Markup Language (VRML), ColdFusion.TM. or other compilers, assemblers, interpreters or other computer languages ​​or platforms.

[0036] The computer program code for performing the operation of various aspects of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, C#, etc., and traditional process programming languages, such as "C" programming language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using the Internet of an Internet service provider). The program code can also be distributed in multiple computing units, wherein each unit processes a part of the total calculation.

[0037] Aspects of the present invention will be described below with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to embodiments of the present invention. It should be understood that each block in the flow chart and / or block diagram and the combination of the blocks in the flow chart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a computer, a special-purpose computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of a computer or other programmable data processing device create a device for implementing the function / action specified in one or more blocks of a flow chart and / or block diagram.

[0038] These computer program instructions may also be stored in a computer-readable medium, which may instruct a computer, other programmable data processing apparatus, or other device to operate in a specific manner so that the instructions stored in the computer-readable medium produce a manufactured product including instructions for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0039] In addition, although process steps, method steps, algorithms, etc. may be described in order, these processes, methods, and algorithms may be configured to work in an alternating order. In other words, any order or sequence of steps that may be described does not necessarily represent a requirement to perform the steps in that order. The process steps described herein may be performed in any practical order. In addition, some steps may be performed simultaneously.

[0040] It is obvious that the various methods and algorithms described herein can be implemented by, for example, appropriately programmed computers and computing devices. Typically, a processor (e.g., a microprocessor) will receive instructions from a memory or similar device and execute these instructions to perform the processes defined by these instructions. In addition, various known media can be used to store and transmit programs that implement such methods and algorithms.

[0041] When a single device or article is described herein, it is obvious that multiple devices / articles (whether or not they cooperate) can be used to replace a single device / article. Similarly, when more than one device or article is described herein (whether or not they cooperate), it is obvious that a single device / article can be used to replace more than one device or article.

[0042] The term "computer-readable medium" as used herein refers to any medium that participates in providing data (e.g., instructions) that can be read by a computer, processor, or similar device. Such media can take a variety of forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks and other persistent memories. Volatile media include dynamic random access memory (DRAM), which typically constitutes the main memory. Transmission media include coaxial cables, copper wires, and optical fibers, including wires that constitute a system bus coupled to a processor. Transmission media may include or transmit sound waves, light waves, and electromagnetic radiation, such as sound waves, light waves, and electromagnetic radiation generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAMs, PROMs, EPROMs, FLASH EEPROMs, any other memory chips or cassettes, carrier waves described below, or any other computer-readable media.

[0043] Various forms of computer-readable media may be involved in transmitting the sequence of instructions to the processor. For example, the sequence of instructions (i) may be transmitted from a RAM to the processor, (ii) may be carried via a wireless transmission medium, and / or (iii) may be formatted according to a variety of formats, standards or protocols, such as TDMA, CDMA, 3G, 4G, 5G, etc.

[0044] Unless explicitly stated otherwise, and as will be apparent from the following description and claims, it should be understood that terms such as “processing,” “computing,” “calculating,” “determining,” etc., used throughout the description of the specification refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulates and / or transforms data represented as physical quantities (e.g., electronic quantities) in the registers and / or memories of the computing system into other data, which are also represented as physical quantities in the memories, registers, or other such information storage, transmission, or display devices of the computing system.

[0045] Similarly, the term "processor" may refer to any device or portion of a device that processes electronic data from registers and / or memory to transform the electronic data into other electronic data, which may be stored in registers and / or memory or may be transmitted to an external device, thereby causing a physical change or actuation of the external device.

[0046] Broadly speaking, embodiments of the present invention provide systems and methods for providing dynamic visual watermarking features in streaming media and video distribution. The systems and methods can take streaming video, including video on demand and live streams, and apply dynamic visual watermarks as desired by the content owner or broadcaster. Dynamic visual watermarks are visible watermarks that can be uniquely generated for each stream, viewer, or user. These techniques can maintain the quality of the stream, make the implementation of dynamic visual watermarks feasible (e.g., when used for anti-piracy, typically about 1% of the cost of traditional "burn-in" techniques), apply to live streams, work with any chosen cloud platform vendor, DRM provider, or other forensic watermarking vendor, and do not require extensive setup and integration to make them work (if DRM or A / B forensics is not involved). Implementation can take as little time as filling out an online form.

[0047] Video publishing / streaming solutions according to aspects of the present invention can provide broadcasters with a media stream manipulation system (if they choose) to mark each stream with a dynamic visual watermark. This can solve the root problem (people creating pirated streams for others). Unlike traditional solutions that provide client-side solutions on the user's application or display device, which can be hacked and filtered because they are an overlay on the video stream, video publishing / streaming solutions according to aspects of the present invention are "burned" on the "server side" or more accurately the "bitstream server side at the edge", as described in more detail below. Video publishing / streaming solutions according to aspects of the present invention can be platform-independent with respect to cloud platforms, software, and networks.

[0048] In some embodiments, the dynamic visual watermark can be, for example, text, an image, a QR code (or the like), and pixel manipulations such as shading, outlining, etc. In some embodiments, aspects of the invention can provide a system that uses a code on a video stream, and if something captures the screen, the system can still use some of the manipulated pixels to narrow down who might be the "leaker" of the content. Thus, aspects of the invention can provide not only anti-piracy, but also leak prevention, for example, preserving unexpected plots and spoilers for unreleased movies or series or shows.

[0049] In some embodiments, the present invention can utilize commercial off-the-shelf (COTS) and open source cloud platforms, video encoding software, and streaming media packaging software to implement a scalable solution so that all viewers can have a unique watermark.

[0050] In some embodiments, a system implementing a video publishing / streaming solution according to aspects of the present invention includes a VM (virtual machine) or an extensible VM platform (many types, such as Docker, Kubernetes, AWS, Azure, VMware, etc.), which takes a streaming URL as input. This URL usually references a "streaming manifest". A streaming manifest is a URL list of segmented video "chunks". The streaming method of today's video is to process the video from a single file and "chunk" it into smaller segments, such as two-second fragments. The manifest is a playlist that indicates the order in which the streaming player plays these "chunks" or fragments. The resulting playback experience is seamless video.

[0051] According to the method of the present invention, the software may include program code on a virtual machine (VM) platform that can intercept a manifest (aka playlist), parse the URLs within the manifest, "GET" the segments referenced in the manifest, cache them, decode them, watermark them, re-encode them to a uniform format, and then host the new written manifest with the watermarked segments to be sent to the end user / viewer's player or application. This is done for each end user / viewer. The content creator / broadcaster / network may send watermark information to the VM via an API or GUI, which may include the watermark text or image, where the watermark is placed, the frequency (e.g., display 10 seconds of text every 5 minutes), and any desired display effects (e.g., fade in and out, opacity, size).

[0052] If the stream has DRM, the method can be integrated with the original broadcaster or streaming company to sync with their DRM license server so that the VM operating the software program code can decrypt and re-encrypt the video "chunks" during the decoding / encoding steps.

[0053] If the stream has forensic watermarking via an A / B dynamic manifest, aspects of the present invention can simply integrate with the original broadcaster to become a proxy for the player / application to obtain the unique A / B forensic mode, caching the fragments / chunks and watermarks before serving it to the client / application.

[0054] Some of the benefits provided by the anti-piracy solutions of the present invention include (1) they are able to perform the function at approximately 1 / 100 the cost of traditional products; (2) they are able to have mobile watermarks (requested by potential customers but not provided by traditional server-side products); (3) they are able to work while live streaming is in progress, which is not possible with current conventional products; (4) they are able to watermark and preserve the original video quality, whereas conventional products must reduce the quality to make it work without long delays; (5) they utilize a simple VM infrastructure, allowing the solutions of the present invention to work on any cloud platform, such as Google, AWS, Azure, dedicated VM networks, private networks, and even content delivery network (CDN) edge computing, whereas conventional products are typically only used on Amazon Web Services (AWS) and have not been migrated anywhere else; (6) they can be integrated with any forensic watermarking company; and (7) they can be integrated with any DRM provider.

[0055] While the above describes systems and methods that can be used for anti-piracy solutions for streaming video content, the system can be applied to other areas, such as using watermarks for advertising or targeted marketing. For example, dynamic advertising can be provided based on known users, demographics, or other variables based on VAST and similar advertising networks and libraries. In some embodiments, dynamic but regional options can be used for watermarks, where group-based watermarks can be provided instead of individual watermarks. In addition, watermarks can be based on geography, demographics, networks, etc. For example, during a broadcast, the system can watermark network associations based on ISP network connections, for example, when watching a live football game-Cox Communications may appear in front of one viewer, but another viewer in town may have Charter Communications. In some embodiments, unique codes can be streamed so that viewers can identify themselves in interactive television or games (such as promotional sweepstakes).

[0056] In some embodiments, the system of the present invention can be used to watermark digital signs or videos in public spaces, where the watermark can dynamically change with location, so that the image or media that captures the watermark can determine where the advertisement or display is displayed. If the watermark is a QR code, a viewer who captures the QR code can be directed to a different landing site unique to a specific video stream.

[0057] In some embodiments, in addition to or in lieu of any other visual or forensic watermark, aspects of the invention may provide baseband forensic watermarking, where actual transcoding modifications of images or frames of a single video stream may be provided. This differs from traditional watermarking on the edge, where a pattern is provided to infer forensic uniqueness; e.g., "A / B forensic watermarking," which creates a single watermark stream from two watermarked streams.

[0058] Furthermore, intelligent watermark placement may be implemented within the scope of the present invention, where artificial intelligence (AI) or other algorithms may be used to determine which portion of the screen is best suited for placement of a visual watermark in order to: a. increase the impact of the visual watermark (dark text on a light background, or light text on a dark background), b. reduce the impact of the visual watermark (dark text on a dark background), c. determine when to watermark, or d. determine if text or subtitles on the screen should be placed elsewhere - avoiding space - defining no-go zones. Such a system may further provide the ability to define watermark no-go zones and / or define where a watermark should be placed. Such a system may define random placement, for example, based on a drawn or predetermined area. Predefined templates for where a watermark may appear include, for example, a corner, center, random location, cycling between locations such as the four corners, etc. Such a system may further define effects for the watermark, such as fading, font size (if text), opacity, etc.

[0059] As described above, systems and methods according to aspects of the present invention can be used with or without DRM, with or without forensic watermarks (A / B) or with or without forensic watermarks (A / B) or with or without a full "bitrate ladder" watermark. A "bitrate ladder" describes groups of video streams of the same content encoded at different quality bitrates and / or resolutions. As the basis for adaptive bitrate streaming, the different quality video streams in a bitrate ladder allow the player to select which quality to reduce or eliminate buffering during playback. The systems and methods used in the present invention include scalable parallel processing to watermark all bitrates simultaneously, and also support dynamic visual watermarks on adaptive bitrate streaming.

[0060] In summary, according to aspects of the present invention, a video publishing / streaming solution (which may include, for example, an anti-piracy solution) may provide several features that are different from conventional systems and methods. These features include (as non-limiting examples) (1) a dynamic watermark visually burned-in video watermark (session-based) that is unique to each user / viewer / streamer; (2) the ability to handle live streams, not just VOD (video on demand); (3) the ability to fade in and out the watermark; (4) the ability to move the watermark around the screen; (5) the ability to be deployed on any cloud platform and / or CDN; and (6) the ability to maintain the quality of the source video. In addition, aspects of the present invention may provide dynamic visual watermarks on video streams without the use of databases or other similar computing resources.

[0061] Anti-piracy solutions according to aspects of the present invention can be implemented in various ways, such as a pay-as-you-go model, where streaming video providers can charge per minute, per second, per title, etc. The solution can also provide software to track usage based on account variables, such as but not limited to bandwidth, data transferred, storage and other cloud usage or reservations (configured resources). The solution can be licensed and deployed on customer infrastructure or cloud computing accounts and platforms, can be implemented as an API service, or can be used for smaller customers or customers who do not need integration through a GUI-based video platform SaaS.

[0062] Reference now Figure 2 and Figure 3 , shows an implementation scheme of the system and method according to aspects of the present invention, wherein a live stream 10 is used as a streaming video source. FIG. 1 shows how a live stream 10 with DRM and a forensic A / B dynamic manifest is conventionally provided to an end viewer. The live streaming signal 12 can use, for example, a real-time messaging protocol (RTMP), an Mpeg transport stream (Mpeg-TS), or other similar signaling protocols. An ingest server 14 within a cloud computing platform 22 can receive the stream and pass the stream to an encoder server 16, where three instances of an encoder can be used, one for audio, and one each for A video and B video if a forensic A / B watermark stream is required. Otherwise, a single video stream is encoded. The stream packager 18 can provide various protocols (such as HTTP Live Streaming 26 (HLS) and Dynamic Adaptive Streaming (DASH) over HTTP 28) to transmultiplex and repackage the live stream. Based on the security already in place (DRM, forensic A / B dynamic manifests, visible watermarks), the system can deliver the content (including added dynamic visual watermarks) to users 30 via a streaming proxy server and an optional content delivery network (CDN) 24 or distributed cloud platform.

[0063] As shown in FIG. 1 , in some embodiments, the DRM license server 20 may exchange a token 32 and a license key 34 with a video viewing device (eg, browser / application / set-top box (STB) / smart TV) of a user 30 .

[0064] like Figure 2 and Figure 3 As shown below, Figure 8 and Fig. 9 As discussed in more detail, the streaming proxy server and visible watermark encoding service 42 may be provided from the CDN 24 ( Figure 2 ) receives the video stream 40, or may receive the generic video bitstream 58 ( Figure 3 ). In addition, the streaming proxy server and visible watermark encoding service 42 may receive watermark details 46 from the streaming service provider 44 and may provide the user 30 with a unique bitstream having a dynamic visible watermark 50. The streaming service provider 42 may also provide a unique streaming URL 38 to the user's streaming proxy.

[0065] like Figure 2 As shown, when there is a DRM license server 20, the streaming proxy server and visible watermark encoding service 42 can provide DRM integration 52 for real-time decryption and re-encryption. In addition, the streaming proxy server and visible watermark encoding service 42 can provide DRM token / license key exchange 48 with the device of the user 30.

[0066] Refer to Figure 4 to Figure 7 , shows an implementation of a system and method according to aspects of the present invention, wherein a video on demand 60 is used as a streaming source 62. As shown in the figure, a similar configuration to reference Figure 2 and Figure 3 Figure 4 shows how a video on demand stream with DRM and forensic A / B dynamic manifest is conventionally provided to an end viewer. Of course, other implementations may also be included within the scope of the present invention.

[0067] Forensic watermarking 64 (FIG. 4 to Figure 6 ) or may not perform forensic watermarking 64 ( Figure 7). The encoder server 66 can receive the stream, where three encoder instances can be used, one for audio, one each for A video and B video (if forensic A / B watermark streaming is required). Otherwise, a single video stream is encoded. The stream packager 68 can provide various protocols (such as HTTP Live Streaming 72 (HLS) and Dynamic Adaptive Streaming over HTTP 74 (DASH)) for transmultiplexing and repackaging the live stream. When a DRM license server 70 is present, the stream packager 68 can be a transmultiplexing DRM packaged stream packager. Based on the existing security (DRM, forensic A / B dynamic manifests, visible watermarks), the system can deliver the content (including the added dynamic visual watermark) to the user 30 through a streaming proxy server and an optional content delivery network (CDN) 24 or distributed cloud platform.

[0068] As shown in FIG. 4 , in some embodiments, the DRM license server 20 may exchange a token 32 and a license key 34 with a user's 30 browser / application / set-top box (STB) / smart TV.

[0069] like Figures 5 to 7 As shown below, Figure 8 and Fig. 9 As discussed in more detail, the streaming proxy server and visible watermark encoding service 42 may receive the video stream 40 ( Figure 5 and Figure 6 ) or can receive a generic video bitstream 80 ( Figure 7 ). In addition, the streaming proxy server and visible watermark encoding service 42 may receive watermark details 46 from the streaming service provider 44 and may provide the user 30 with a unique bitstream having a dynamic visible watermark 50. The streaming service provider 42 may also provide a unique streaming URL 38 to the user's streaming proxy.

[0070] like Figure 5 and Figure 7 As shown, when there is a DRM license server 20, the streaming proxy server and visible watermark encoding service 42 can provide DRM integration 52 for real-time decryption and re-encryption. In addition, the streaming proxy server and visible watermark encoding service 42 can provide DRM token / license key exchange 48 with the device of the user 30.

[0071] Figure 8 and Fig. 9 The specific implementation of the exemplary embodiment of the present invention is shown, showing how the streaming proxy server and the visible watermark encoding service perform the task of providing dynamic visual watermarks in video streams (whether live streams or video on demand streams) according to the embodiments of the present invention. It should be understood that Figure 8 and Fig. 9The specific steps shown in are merely exemplary embodiments, and according to aspects of the present invention, other implementations may also implement dynamic visual watermarking.

[0072] Figure 8 The solution described in is a straightforward process to achieve dynamic visual watermarking. Fig. 9 A slightly more complex model is shown that optimizes the processing to reduce latency and delay in processing the video stream. In addition, Fig. 9 The solution has additional steps to determine which segments need to be cached and manipulated based on aspects of the watermark effect that are desired. For example, the straightforward approach of watermarking each segment individually will produce a slight "flicker" in the watermark itself. This is primarily because there are typically many frames in a second of video, and when the video segments are manipulated, visual effects may appear at the boundaries or edges of each segment. To reduce this effect, and allow certain effects (such as fading opacity) to be displayed more smoothly, aspects of the present invention may combine multiple segments together before watermarking, and then intelligently split the segments based on calculated timing. This produces a smooth burn-in display. To further reduce computational overhead and processing time, the same computational steps determine which segments are not watermarked and therefore are not processed.

[0073] Unless expressly stated otherwise, all features disclosed in this specification (including any accompanying abstract and drawings) may be replaced by alternative features having the same, equivalent or similar purpose. Therefore, unless expressly stated otherwise, each feature disclosed is only one embodiment in a series of equivalent or similar features.

[0074] The claim elements and steps herein may be numbered and / or lettered only to aid reading and understanding. Any such numbers and letters are not intended and should not be construed as indicating the order of the elements and / or steps in the claims.

[0075] Many changes and modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention. Therefore, it must be understood that the illustrated embodiments are set forth for illustrative purposes only and should not be construed as limiting the present invention as defined by the following claims. For example, although the elements of the claims are listed below in a certain combination, it must be clearly understood that the present invention includes other combinations of fewer, more, or different disclosed elements.

[0076] The words used in this specification to describe the present invention and its various embodiments should be understood not only to have their generally defined meanings but also to include the general structures, materials or acts indicated by the specific definitions in this specification, which represent single species thereof.

[0077] Therefore, the words or elements in the following claims defined in this specification include not only the literally listed element combinations. Therefore, in this sense, any one element in the following claims can be replaced by two or more elements, or a single element can replace two or more elements in the claims. Although the above elements may be described as working in certain combinations and even initially claimed in this way, it should be clearly understood that in some cases, one or more elements in the claim combination can be deleted from the combination, and the claim combination may point to sub-combinations or variations of sub-combinations.

[0078] Changes that are now known or later devised by one of ordinary skill in the art and that are not related to the subject matter of the claims are expressly deemed to be equivalently within the scope of the claims. Therefore, obvious substitutions now or later known to one of ordinary skill in the art are defined to be within the scope of the defined elements.

[0079] Therefore, the claims should be understood to include the contents specifically illustrated and described above, contents that are conceptually equivalent, contents that can be obviously substituted, and contents that contain the basic idea of ​​the invention.

Claims

1. A system for providing a dynamic watermark to a streaming media, comprising: The streaming proxy server and visual watermark encoding service receives the streaming bitstream from the stream packager and receives the watermark details from the content provider, wherein The streaming proxy server and visual watermark encoding service provide a unique bitstream with the dynamic watermark to the user; and The dynamic watermark is applied to the bitstream server side of the streaming media bitstream before transmitting the dynamic watermark to the user's device.

2. The system of claim 1, wherein the dynamic watermark is a forensic watermark.

3. The system of claim 2, wherein the forensic watermark is selected from one or more of a baseband forensic watermark and a compressed domain forensic watermark.

4. The system of claim 1, wherein: The dynamic watermark is a dynamic visual watermark; The streaming media bitstream is a video bitstream; and The user's device is a video viewing device.

5. The system according to claim 4, wherein: When a digital rights management (DRM) license server is present, the streaming proxy server and visual watermark encoding service decrypts the video segment into a playable segment, adds the dynamic visual watermark, and re-encrypts the video segment according to the requirements provided by the DRM license server.

6. The system of claim 5, wherein the streaming proxy server and visual watermark encoding service exchange tokens / license keys with the video viewing device of the user.

7. The system of claim 4, wherein the video stream is received from a content delivery network.

8. The system of claim 4, wherein the streaming proxy server and visual watermark encoding service add the dynamic visual watermark to only a portion of the video stream to reduce flicker in the dynamic visual watermark application video bitstream provided to the user.

9. The system of claim 8, wherein the streaming proxy server and the visual watermark encoding service first combine the multiple segments together before applying the dynamic visual watermark, and then split the combined multiple segments according to a predetermined timing.

10. The system of claim 4, wherein the video bitstream is from a live stream source or a pre-recorded video stream source.

11. A method for providing dynamic processing to streaming data, comprising: receiving the streaming data from a content provider; receiving dynamic processing data from the content provider; preparing a bitstream of streaming data with dynamic processing applied; as well as The streaming data to which the dynamic processing is applied is transmitted to a user.

12. The method according to claim 11, wherein: The dynamic processing includes at least one of a forensic watermark, a dynamic visual watermark, or metadata; and The streaming data is a video bitstream from a live video source or a pre-recorded video source.

13. The method according to claim 12, further comprising: When a digital rights management (DRM) license server is present during pre-processing of the video bitstream, the video segments are decrypted into playable segments, the dynamic visual watermark is added, and the video segments are re-encrypted according to the requirements provided by the DRM license server.

14. The method according to claim 13, further comprising: Exchange a token / license key with the user's video viewing device.

15. The method according to claim 12, further comprising: The video bitstream is received from a content delivery network.

16. The method according to claim 12, further comprising: The dynamic visual watermark is added to only a portion of the video stream to reduce flicker in the dynamic visual watermark application video bitstream provided to the user.

17. The method according to claim 16, further comprising: Before applying the dynamic visual watermark, multiple segments are combined together, and then the combined multiple segments are divided according to a predetermined timing.

18. A method for providing a dynamic watermark to a video bitstream, comprising: parsing a manifest of the video bitstream; Downloading video clips to a streaming proxy server and visible watermark encoding service; adding the dynamic watermark to the video clip; reformatting the video clip to which the dynamic watermark is applied; constructing a new manifest to play a unique bitstream to which the dynamic watermark is applied; as well as The unique bitstream is delivered to the user's video viewing device.

19. The method according to claim 18, further comprising: decrypting the video bitstream into playable segments, the playable segments being downloaded to the streaming proxy server and visible watermark encoding service; as well as The unique bitstream is re-encrypted before being transmitted to the video viewing device of the user.

20. The method of claim 18, further comprising: determining a selected portion of the video clip to download; adding the dynamic watermark only to the selected portion of the video clip; reformatting the selected portion of the video clip to which the dynamic watermark is applied; splitting the reformatted video segments and preparing a new manifest based on the splitting of the reformatted video segments; as well as The new manifest is merged with the manifest with the multi-period manifest to provide playback without flickering due to the dynamic watermark.

Images