Interactive Broadcast Overlays with Dynamic QR Code Embedding

Abstract

A method is disclosed for creating an interactive user interface associated with an audiovisual broadcast. Instructions are provided for generating, along with the audiovisual broadcast, a first user interface that is overlaid on the audiovisual broadcast by a device displaying the audiovisual broadcast. Upon capturing an identifier in the UI at a second device, a second user interface is generated on the second device, and the first user interface is updated to display information in conjunction with the second user interface. A method of truly dynamic quick response (QR) codes is also disclosed. After the QR code encodes data representative of one or more variable values ​​associated with the interface, in response to receiving an input event representing that a change in value of at least one of the variable values ​​has occurred, the interface is updated to replace the first QR code with a second QR code encoding data representative of the change.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a non-provisional patent application that derives priority benefit from U.S. Provisional Patent Application No. 63 / 315,077, filed on February 28, 2022, and entitled "Dynamic QR Code Embedding in a Media Interface," and U.S. Provisional Patent Application No. 63 / 359,754, filed on July 8, 2022, and entitled "Interactive Broadcast Overlay System," both of which are incorporated by reference in their entireties herein.

[0002] The present application relates to methods and systems for two-way communication between a screen used to display a broadcast and a computer server associated with that broadcast, possibly facilitated by the use of QR codes, and more particularly to methods and systems for creating an interactive user interface that is overlaid on a broadcast and responds to local user input or other events, and that can be enhanced by embedding QR codes in the interface and updating the QR codes in response to user input or other events. [Background technology]

[0003] In traditional television broadcast systems, communication is exclusively unidirectional, with multiple signals from one or more broadcast sources transmitted over the airwaves or cable network, and a television antenna or cable box separating one signal for viewing and consumption by that television viewer. Modern cable television offers many more options for more personalized entertainment, such as the ability to purchase movies and pay-per-view events, but the basic principle of presenting the same stream to all viewers remains common.

[0004] The introduction of the Advanced Television Systems Committee standards ("ATSC 3.0"), and similar Connected Television ("CTV") standards for "smart" televisions, has made available Internet Protocol (IP) based systems for broadcast transmission. A stream of multiplexed packets can carry information for both the underlying audiovisual stream and additional data related to internet-connected applications running on the television or other device. CTV standards are expected to continue to evolve over time and to incorporate additional features in future versions of the ATSC standards, or in any future standards that may be developed in parallel with or on behalf of ATSC to interface with smart televisions or other display devices.

[0005] This combined data stream offers a unique opportunity to implement interactive services that receive, analyze, and respond to additional data packets while displaying the audiovisual broadcast transmission, although such applications have not yet been widely deployed or adopted.

[0006] Additionally, as part of any display, it may be necessary to communicate information from the display to another computing device. Strings generated as part of a display may be used for a variety of purposes during software execution outside of the display, including hyperlink Uniform Resource Locators (URLs) for navigation by a web browser, tokens or access codes for two-factor authentication, or information on a passport, boarding pass, or other document.

[0007] When a hyperlink or other information is presented within one app on a personal computer or smartphone, it is easy to copy and paste the other information into another app on the same device due to a shared clipboard feature of all apps on the device. In contrast, the lack of a shared clipboard makes it tedious at best and impossible at worst for a human user to copy long or dense text information presented in a non-digital medium or on one device to another device.

[0008] Quick Response (QR) codes are rapidly becoming the primary way to encode hyperlinks or other information for copying across media. QR codes allow an optical scanner, such as a smartphone camera, to take a photo containing the QR code. Software can process the image and decode the encoded content without error, even in images taken in poor lighting conditions or from any orientation, rotation, or angle. The decoded content can then be copied to the smartphone's clipboard for use in another app, or to follow a hyperlink in the smartphone's browser.

[0009] However, the main limitation of most QR codes, especially those printed on physical media, is that they are completely static: once a QR code is generated and included in some medium, the hyperlink encoded within it always points to the same destination, even if the destination becomes inaccessible, deprecated, outdated, or no longer usable for its original purpose. This limitation can be partially overcome by a method sometimes called a "dynamic QR code". The QR code itself does not change, and always encodes a fixed destination URL that is the same as the placeholder, but that URL simply points to a server that performs server-side processing and redirects the request to the new final destination URL upon receiving a request for the placeholder URL.

[0010] This "dynamic" solution also has drawbacks. Because the placeholder or deprecated URL is the same for each recipient of the QR code, it cannot contain any client-side event logging or identification when the user takes the photo and is redirected to the final destination URL. The server-side redirection functionality must always be on and listening, otherwise the user will not be redirected and the QR code will simply appear broken. The server-side functionality must be updated every time a change of the final destination URL is desired.

[0011] Each of these limitations could be circumvented if the QR code itself - a pattern of black and white bits - could be changed, rather than relying on an external processor to reinterpret the meaning of fixed encoded values.

[0012] As a result of all the above contexts, there is an advantage to developing new interactive applications that offer viewers capabilities never before experienced on television, and to incorporating truly dynamic QR codes into such applications to provide these new capabilities, without relying on server-side redirection. Summary of the Invention

[0013] To address the opportunities presented by these technologies, a system and method for creating an interactive user interface associated with an audiovisual broadcast is disclosed. The method begins with transmitting, together with the audiovisual broadcast, an instruction to generate a first user interface to be overlaid on the audiovisual broadcast by a device displaying the audiovisual broadcast, and then receiving a first command from a viewer of the audiovisual broadcast that results in updating one or more values ​​associated with the first user interface. In response, the method then creates an identifier in the generated user interface overlay that is used to access a second user interface on a second device. The user captures the identifier with the second device. The method then continues with generating a second user interface on the second device, receiving a second command from the viewer as the viewer uses the second device, again updating the one or more values, and finally updating the first user interface to display information related to the one or more values ​​updated from within the second user interface.

[0014] Such a system could fundamentally change the functionality available on a smart TV, enabling shopping carts, auctions, multiplayer gaming, social networking, and many other use cases requiring two-way communication and previously limited to personal computers or smartphones. By adding a user interface to the video stream, nearly any use case that previously required a separate window to display the video and another window to display the user interface can be realized.

[0015] Whether as part of the system described above or in a separate application, it may be desirable to dynamically update the QR code itself, i.e., to actually change the visible, scannable pattern regardless of where the QR code is already distributed or displayed, rather than simply redirecting the original URL destination to a new or different URL destination. In addition to the URL itself, QR codes can also encode many types of data, including, as one non-limiting example, data that can be added to a URL as a parameter that improves the results of traversing the URL to its destination.

[0016] This data may originate from any number of sources or causes, and its transmission may be desirable for many reasons, such as to transmit real-time status information about a machine or software system, to transmit confirmation of a user's identification or access rights, to transmit information about a user's interaction with a web page on which the QR code is placed, or to transmit information about a user's interaction with a digital advertisement that includes a QR code.

[0017] This scenario, where the QR code pattern itself is dynamic, rather than simply the URL redirection being dynamic, represents a truly "dynamic QR code." The form of "dynamic QR code" common in existing applications is actually a misnomer, since it is the redirection that is dynamic, not the QR code itself. In contrast, this disclosure describes a QR code that is itself dynamic, changing in response to local conditions or events in the environment in which the code is placed. This disclosure teaches a truly dynamic QR code, namely "TDQRC."

[0018] A system and method for presenting a digital display incorporating a truly dynamic QR code is disclosed, the method including generating a first set of one or more images including a first QR code, encoding data representative of one or more variable values ​​associated with a user interface or environment in which the QR code is displayed, receiving an input event representative of at least one of the one or more variable values ​​having a value change or at least one additional value being added or modified, and generating a second set of one or more images as a continuation of the stream that comprised the first set of one or more images, where the first QR code is replaced with the second QR code encoding data representative of one or more variable values ​​associated with the digital display, including the change in value of the at least one variable value.

[0019] Other aspects, features, and advantages will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings, which are provided for illustrative purposes only, without limiting the scope of any embodiments. [Brief description of the drawings]

[0020] [Figure 1] FIG. 13 illustrates a sample broadcast user interface incorporating a QR code or other trigger to initiate interaction. [Diagram 2] FIG. 13 illustrates another sample user interface incorporating a truly dynamic QR code. [Diagram 3] FIG. 1 illustrates a back-end system for broadcasting digital content to human users and facilitating interactive communication. [Figure 4] FIG. 1 illustrates a more general computing system for delivering digital content, including dynamic QR codes, to human users. [Diagram 5]FIG. 1 illustrates a simplified diagram of a method for generating a dynamic QR code and subsequently updating the QR code in response to events internal or external to the digital interface enhanced by the QR code. [Figure 6] FIG. 2 shows a simplified diagram of how interaction can be initiated from a broadcast. [Figure 7] 7A, 7B, 7C and 7D show user interfaces that are updated in response to a viewer's interaction with a secondary device. [Figure 8] 8A, 8B, 8C and 8D show separate successive moments of a truly dynamic QR code from the sample interface shown in FIG. [Figure 9] FIG. 1 illustrates a typical computing device for performing some of the functions described above. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] To take advantage of the new broadcast media, a series of related applications or use cases are disclosed that rely on the general principle of including instructions for generating a user interface on the device being used to display the broadcast. The instructions take the form of markup languages ​​and scripts that generate a web page-like interface (such as those created in HTML and JavaScript) that are executed by a parser operated by the display device. The display device displays the underlying audiovisual stream while also operating a dynamic user interface that overlays the audiovisual stream and responds to various user actions or inputs and / or directs the user to specific web pages or applications on a secondary device that the user interacts with while viewing the broadcast on the primary device.

[0022] FIG. 1 shows a sample broadcast user interface incorporating a QR code or other trigger to initiate an interaction.

[0023] When the smart TV 100 is used to watch a valid broadcast 105 , the stream of packets in that broadcast is split into audiovisual transport packets 110 and other IP packets 115 .

[0024] The audiovisual transmission packets 110 are used to generate the underlying display 120, and the other IP packets 115 are used to generate the overlay display 125. The overlay display 125 preferably includes a QR code 130 or other similar bar code, graphical encoding, or visual trigger that, when detected by a camera or other sensor of the secondary device 135, allows a viewer operating the secondary device 135 to initiate an interaction that directly or indirectly affects the user interface of the overlay display 125. This is preferably accomplished by incorporating a URL into the QR code 130 that the secondary device 135 visits after the URL is decoded from the QR code. The URL forms part of an API that takes one or more actions when the secondary device 135 makes an HTTP request for the URL, as described further below in connection with Figures 3 and 4 and the following paragraphs.

[0025] In one exemplary embodiment, the following code initializes a WebSocket connection with an identifiable SESSION_ID between the video and server 215 (shown in FIG. 2). This code also creates a URL with the SESSION_ID embedded in it, which is preferably accessed via QR code 130, directing secondary device 135 to a landing page hosted by server 215 or another server. / ** *Initialize by creating a WebSocket to the Communicator server. * / function initialize(){ / / Create a WebSocket object to connect. / / The SESSION_ID value is dynamically generated. var socket = new WebSocket( 'wss: / / communicator.linkstorm.net / ', SESSION_ID+"_ad" ); / / Open a WebSocket connection to the Communicator server. socket.addEventListener("open", () =>{ var msg = '{"SessionID":"'+SESSION_ID+'", "SessionType": "ad"}'; socket.send(msg); }); / / Handle incoming messages from the Communicator socket.onmessage = function(e) { / / Run custom code on the TV that responds to messages sent via Communicator from the landing page. }; } / ** * *Returns the URL the QR code will redirect to * *The video's session ID is passed to the landing page as the connection ID * * / function qrCode(){ return qr_code_destination_url = DOMAIN + " / cart.html?connection_id=" + SESSION_ID + "&data="; }

[0026] In some embodiments, the QR code 130 or similar trigger is the same code for each viewer of a broadcast, directing the secondary device 135 to the same landing page, where the viewer is then identified by login, by source IP address, by a cookie present on the secondary device 135, or by any other identifying characteristic of the viewer or secondary device. In other embodiments, the QR code 130 already uniquely identifies the viewer or the TV 100 itself when communicating with the source device, for example, by including information accessible to a script running on the smart TV in a variable appended to the URL encoded by the QR code. In some embodiments, the QR code is a truly dynamic QR code that responds to initial input by changing the encoded URL, according to a process described in more detail in the following portion of this disclosure.

[0027] Although all preferred embodiments are described as associating the display with an audiovisual broadcast, it is not strictly necessary that the broadcast include both audio and visual content. Even if the broadcast only includes audio content and the visual content is only white noise, still image frames, completely black frames, or other displays without useful information, the visual display 125 can be overlaid on the unchanging or uninformative visual aspect. Similarly, if the broadcast only includes video content and is silent or has white noise or other uninformative sound, the functionality of the visual display 125 is not affected in any way by the absence of sound. In some embodiments, instead of using a visual code 130 to connect the secondary device 135 to the broadcast origination server, it is also conceivable to use an audio code that does not interfere with the listener's enjoyment of the broadcast, such as a series of tones at infrasonic or ultrasonic frequencies detectable by the microphone of the secondary device 135.

[0028] Throughout this disclosure, device 100 will be referred to as a television, although any other device compatible with the protocol for mixing additional code data with the broadcast video stream may be used instead. This may include a laptop computer, a desktop computer and monitor, a tablet, a game console, a handheld gaming device, a display in the dashboard of a vehicle or other display in a car, a smart watch, a personal digital assistant, a kiosk monitor, or any other device capable of simultaneously displaying video and also simultaneously executing software. Similarly, while secondary device 135 is primarily envisioned as a smartphone, in other embodiments the device may be a smart watch, smart glasses, a camera connected to a smart home system, or any other device having a camera or similar optical scanner capable of picking up a QR code at a frequency at which the QR code reflects radiation, or any other type of sensor capable of capturing data from another form of code 130.

[0029] The ATSC 3.0 standard is preferred as the protocol for transmitting the broadcast and associated code to the smart television, although any other method that may become common for simultaneously transmitting an audiovisual channel or stream and a data channel or stream to a single device may instead be used, such that the device can decode both streams simultaneously, interpret one to generate a user interface, and display the other below the user interface.

[0030] As discussed above, the QR code 130 in the display may be a truly dynamic QR code, and furthermore, a truly dynamic QR code may be incorporated into a display other than a broadcast video stream. In another example, Figure 2 shows a sample user interface incorporating a dynamic QR code.

[0031] In the video player or other display 200, a menu 210 may be overlaid on top of the underlying video 205 to allow user interaction while the video 205 is being played. The functionality of this form of enhanced video is further described in U.S. Patent Application Serial No. 16 / 953,096, "VIDEO ADVERTISEMENT AUGMENTATION WITH DYNAMIC WEB CONTENT," now allowed as U.S. Patent No. 11,430,019.

[0032] After the user selects an item 215 from the menu 210, a cart icon 220 may be displayed, but more importantly, a QR code 225 is generated and displayed within the interface.

[0033] In this particular use case, the QR code 225 encodes not only a hyperlink to the website of the merchant selling the item displayed in the underlying content 205, but also text describing the item currently added to the cart by interacting with the menu 210. When a second item 215 is clicked and thereby added to the cart, the QR code 225 changes to reflect the updated cart contents.

[0034] The result is an enhanced interface 230 that includes a truly dynamic QR code 225. At any point during the user's shopping and interaction with the menu 210, by taking a picture of the QR code, the device that took the picture can navigate to the merchant's website on that device with the item already added to the cart and ready for checkout, as described in more detail in FIG. 3 and subsequent figures.

[0035] FIG. 3 illustrates a back-end system for broadcasting digital content to human users and facilitating interactive communication.

[0036] The broadcast origination server 300 coordinates between an audio / video store 305 and a code data store 310 to determine what to send to the smart television 100 and combines the input data from each source into a stream 105. Such transmission may be accomplished in some embodiments by radio signals originating at a transmission tower 320 and ultimately passed to an antenna connected to the television 100, or via a satellite repeater and satellite dish connected to the television 100. In other embodiments, a physical data network 325, such as the Internet, Ethernet, or a cable television network, may connect the origination server 300 to the smart television 100.

[0037] As mentioned above, the user interface on the smart TV may include a QR code or other means of directing the secondary device 135 to the web server 315. In one example embodiment, the web server 315 may receive a query or transmission from the secondary device 135 and execute code that associates the query or transmission with the particular video being viewed at the time. const WebSocketServer = require('ws'); / / Communicator object const Communicator = function(wsPort){ / ** *Map session IDs to advertising websocket and landing page websocket connections *You can associate a specific video websocket with the websocket created on landing page load * / const that = this; this.connectionMap = new Map(); this.wss = new WebSocketServer.Server({port: wsPort}); / ** *When a new connection request is received, parse the header to determine the session ID and type of connection (ad or landing page) * / that.wss.on("connection", function(ws, req){ console.log("A new client has connected"); let headers; let sessionID; let clientType; try { headers = req.headers['sec-websocket-protocol'].split("_"); sessionID = headers[0]; clientType = headers[1]; if(clientType != "ad" && clientType != "landing"){ console.log("Invalid connection"); ws.close(); return; } }catch(e){ console.log("Invalid connection"); ws.close(); } that.handleIncomingConnection(ws, sessionID, clientType); }); / ** * Handles incoming connection requests by checking if the session ID is present in the map and adding it if not. *Call setupNewClientForDualWS to create the necessary event handlers for message and close events on the new websocket connection. * @param {*} ws * @param {*} sessionID * @param {*} clientType * / this.handleIncomingConnection = function(ws, sessionID, clientType){ if(!that.connectionMap.has(sessionID)){ let dualWS = {}; that.setupNewClientForDualWS(dualWS, ws, clientType, sessionID); that.connectionMap.set(sessionID, dualWS); } else { let mapWS = that.connectionMap.get(sessionID); that.setupNewClientForDualWS(mapWS, ws, clientType, sessionID); that.connectionMap.set(sessionID, mapWS); if(clientType == "landing" && typeof mapWS.adWS != "undefined"){ mapWS.adWS.send("bark"); } } } / ** *Receive the dualWS object (websocket for the ad and websocket for the landing page), *Returns the updated dualWS object after adding the new websocket to the object. * @param {*} dualWS * @param {*} ws * @param {*} clientType * @param {*} sessionID * @returns * / this.setupNewClientForDualWS = function(dualWS, ws, clientType, sessionID){ if(clientType == "ad" && typeof dualWS.adWS === "undefined"){ that.setupAdClient(sessionID, ws) dualWS.adWS = ws; }else if(clientType == "landing" && typeof dualWS.landingWS === "undefined"){ that.setupLandingClient(sessionID, ws); dualWS.landingWS = []; dualWS.landingWS.push(ws); }else if(clientType == "landing"){ that.setupLandingClient(sessionID, ws); dualWS.landingWS.push(ws); } return dualWS; } / ** *Set up an event handler for the ad websocket connection. * @param {*} sessionID * @param {*} newClient * / this.setupAdClient = function(sessionID, newClient){ newClient.on("message", function(data){ console.log(`Client has sent us: ${(data)}`); }); newClient.on("close", function(){ let dualWS = that.connectionMap.get(sessionID); dualWS.adWS = undefined; that.connectionMap.set(sessionID, dualWS); console.log("Client has been disconnected"); }); } / ** *Set up an event listener for the landing page's websocket connection. * @param {s} sessionID * @param {*} newClient * / this.setupLandingClient = function(sessionID, newClient){ newClient.on("message", function(data){ console.log(`Client sent: ${(data)}`); }); newClient.on("close", function(){ let dualWS = that.connectionMap.get(sessionID); const index = dualWS.landingWS.indexOf(this); if(index > -1) { dualWS.landingWS.splice(index, 1); } if(dualWS.landingWS.length == 0) dualWS.landingWS = undefined; that.connectionMap.set(sessionID, dualWS); console.log("Client has been disconnected"); }); } } / ** *Start the Communicator Websocket server. * / const server = new Communicator(3333); console.log("WebSocket server running on port 3333");

[0038] Upon receiving a query or transmission from the secondary computing device 135, the web server 315 serves a web page or other application to the secondary computing device 135. In one example embodiment, the web page may include the following code: <title> TEST< / title> <script type="text / javascript"> / / 渡されたurl変数からハッシュを作成するヘルパー関数。function QtoH(){try{var s = location.href.substring(location.href.indexOf('?')+1);s = s.split('&');var h = [];for(var i = 0; i < s.length;i++){var p = s[i].split('=');h[p[0]] = p[1];}return h;}catch(e){console.log(e)}}; / / すべてのurl変数を含むハッシュを作成する。var connection = QtoH(); / / 接続するWebSocketオブジェクトを作成する。 / / connection_idハッシュ値は、QRコードのURLを介してランディングページに渡される。var socket = new WebSocket('wss: / / communicator.linkstorm.net / ',connection['connection_id']+"_landing"); / / CommunicatorサーバへのWebSocket接続を開く。socket.addEventListener("open", () =>{var msg = '{"SessionID":"' + connection['connection_id']+"_landing" + '", "SessionType": "landing"}';socket.send(msg);}); / / WebSocket接続を閉じるsocket.addEventListener("close", function(){console.log("接続が閉じられました");});< / script>

[0039] The web server 315 also communicates with the broadcast origination server 300 and instructs the broadcast origination server to further communicate with the smart TV 100 to update the user interface, as further described below in connection with FIG. 6 and the description of subsequent figures.

[0040] While a preferred configuration of devices is shown in Figure 3, in some embodiments any combination of these devices may be on a single device. For example, a single data store may store both the audio / video data for broadcast and the code for user interface generation (305 and 310). A single server may generate the broadcast stream and listen for incoming API requests to service the second user interface (300 and 315). If desired, all four of these computing devices, or any other subset of them, may also be combined into a single device.

[0041] Furthermore, in some embodiments, rather than a single device performing the described functions, a cloud-based solution may provide functionality through the action of multiple devices in a cluster or distributed geographically. For example, audio / video data and code may be stored in a content delivery network made up of many access points 305 and 310, and the server 300 may pull from any access point that is active and available at a given moment. Multiple servers 300 or 315 may operate in a load-balanced cluster or in different geographic areas to serve a large number of distributed users simultaneously. Single Device Interactivity

[0042] In some embodiments, it may be desirable to prevent a viewer from hopping from a first interface to a second interface on a different device. In such instances, it may be possible to complete useful functionality using the overlay generated for the broadcast alone, without the need for an external mobile device such as a cell phone. For example, one may add an item to a cart overlaid on an advertisement, and then complete the transaction and purchase the item within the same overlaid interface. It is not strictly necessary to use a QR code or other identifier to move the cart to a new interface or device. Multi-device, unidirectional interactivity

[0043] In other embodiments, a QR code or other identifier may be used to navigate to a new interface or device without necessarily creating a feedback loop that affects the original user interface or underlying broadcast.

[0044] FIG. 4 illustrates a more general computing system for delivering digital content, including dynamic QR codes, to a human user.

[0045] Computing device 400 may include instructions for generating enhanced interface 230 or may otherwise receive instructions for generating enhanced interface 230 from web server 405 or other server. Computing device 400 may be, depending on different embodiments, a laptop computer, a desktop computer, a smart television, a game console, a handheld gaming device, a smart watch, a personal digital assistant, an electronic slot machine or other regulated gaming device, a kiosk monitor, a self-checkout retail device, or any other device capable of executing software and having a visual output display that can be controlled and updated by software during execution.

[0046] After the computing device 400 begins displaying the enhanced interface 230, the QR code 225 of the enhanced interface 230 may be repeatedly updated according to a method further described below in conjunction with FIG.

[0047] At any stage between repeated updates, a second computing device 410 may be used by a viewer or user of the extended interface 230 to capture an image of the QR code 225. The second computing device 410 is preferably a smartphone owned by the user of the first computing device 400. In other embodiments, the second computing device 410 may be a smart watch, smart glasses, a camera connected to a smart home system, or any other device equipped with a camera or similar optical scanner capable of picking up a QR code at a frequency at which the QR code reflects radiation.

[0048] The QR code 225 may point to a URL of a different web server 415, such as a web server for a merchant's website, and the second computing device 410 may be used to continue two-way communication with the different web server 415 (e.g., to complete a transaction initiated within an advertisement and check out). In other embodiments, the QR code 225 may instead point to the same server 405, and there may be no need to distinguish between servers 405 and 415. Truly dynamic QR code generation

[0049] To address the shortcomings of static QR codes, various graphical user interfaces or digital media presentations may be augmented with QR codes that initially encode a first information content, but that can be replaced with a new QR code that encodes a second information content that is different from the first information content. This replacement may occur in response to a human user's interaction with the display in which the QR code is embedded, in response to a push communication from a central server to which the display is listening, or in response to other events detected by scripts running within the display medium.

[0050] FIG. 5 illustrates in a simplified manner how a truly dynamic QR code can be generated and then updated in response to events internal or external to the digital interface that is enhanced by the QR code.

[0051] First, a digital display 230 is generated (step 500). As previously mentioned, the digital display 230 may be a video player with an interactive menu overlaid. In other embodiments, other forms of graphical user interfaces may be used, such as a web page, a dashboard displaying information or statistics, a shopping cart, a widget on an operating system desktop or a mobile phone, etc. Finally, a graphical user interface may be used that displays information and / or options to a human user.

[0052] In some embodiments, the QR code may already be generated when the digital display is first displayed (step 505). In other embodiments, no QR code may be displayed until a triggering event causes a QR code to be generated and displayed. In such embodiments, the digital display need only reserve an area of ​​the display in which the QR code can eventually be displayed, or have the ability to rearrange elements of the display to make room for the QR code.

[0053] The digital display is configured to listen for one or more types of events that require a change to the QR code to ensure that it represents the current state of the information associated with or represented by the digital display (step 510). For example, if the digital display contains a shopping cart, changing the number of instances of an item in the cart may require the content of the QR code to change accordingly. For any text, numeric, or other variable associated with the display (or the environment in which the display contains information) that may change over time, such an event may be that such a variable has changed. Alternatively, if the QR code stores a log of a sequence of events, actions, or choices, each additional such event, action, or choice may trigger the addition of information to that log.

[0054] In response to detecting one of the events being listened for (step 515), the QR code is updated to represent the current state of the display (variables, logs, tokens, etc.) (step 520).

[0055] Included below is sample code that generates a QR code and re-executes whenever a trigger event occurs. / ** * * @param {String} QR code destination URL * @param {Array} An array containing all items currently in the shopping cart. * @param {HTMLElement} The HTML element where the QR code will be rendered. * / this.drawDynamicQRCode = function(QRCodeDestionationURL, ShoppingCartItems, QRCodeParentMenu){ / / QR code container var QRcodeHTMLContainer = QRCodeParentMenu.getElementsByClassName("ls-qrcode")[0]; / / Empty the QRCode container QRcodeHTMLContainer.innerHTML = ""; / / Build a string of order data var order_data = ""; / / Loop through the order items and append the product and quantity to the url / / Format: ProductID_QUANTITY|ProductID_QUANTITY|ProductID_QUANTITY| for (var i = 0; i < ShoppingCartItems.length; i++){ if(ShoppingCartItems[i].quantity > 0) { / / Add each product and add quantity order_data += (ShoppingCartItems[i].id + "_" + ShoppingCartItems[i].quantity + "|"); } } / / Remove the trailing "|" character order_data = order_data.substring(0, order_data.length - 1); / / Encode the order_data in base64 order_data = btoa(order_data); / / Set a new destination URL with the order data appended to the end QRCodeDestionationURL = QRCodeDestionationURL + order_data; / / Fade in the QR code container after 0.5 seconds setTimeout(function(){ QRCodeParentMenu.style.opacity = "1"; }, 500) / / Pass in the container and the new QR code destination URL to generate the code / / The QR code is placed in an HTML container. / / QR codes are generated using the following open source libraries: / / https: / / github.com / davidshimjs / qrcodejs new QRCode( QRcodeHTMLContainer, { / / Container for the QR code text: QRCodeDestionationURL, / / URL where QRCode redirects width: 150, height: 150, colorDark : "#000000", colorLight : "#ffffff", correctLevel : QRCode.CorrectLevel.H } ); / / Return the destination URL for further processing, if desired. return QRCodeDestionationURL; }

[0056] In some embodiments, the change in the QR code may be intended to be very noticeable. The entire QR code may disappear and then a new QR code may appear. The change may be complemented by other elements of the interface or visual effects, including one or more of messages, highlighting effects, blinking effects, use of color, etc. In other embodiments, the QR code may be intentionally made invisible to a human user that it has been changed. For example, a fade effect or other animation may cause the displayed QR code to gradually change from the original code to the new code, making the change unnoticeable to a human user unless the human user is looking directly at the QR code during the transition.

[0057] After updating the QR code, the display continues to listen for subsequent events and updates (returning to step 510) in a continuous loop that maintains the relevance and accuracy of the QR code's intended meaning.

[0058] In some embodiments, it is possible for the QR code to not be displayed once the user has used it to transfer information to a new medium. In these embodiments, the web server of the destination URL encoded by the QR code may send a message in response to being accessed by the user's computing device 410. If the display is listening for this communication, it will stop displaying the QR code once the communication is received, allowing the user interface to be cleaned up once the embedded QR code is no longer needed. Interacting with the Broadcast

[0059] FIG. 6 shows a simplified diagram of how interaction can be initiated from a broadcast, as described above.

[0060] First, the broadcast origination server 300 begins transmitting video and other IP data to the smart TV 100 (step 600). The IP data includes instructions for creating a user interface that is overlaid on top of the video.

[0061] In response to receiving and executing the instructions, the smart television 100 creates a user interface over the video (step 605), and an exemplary generated user interface suitable for one embodiment of the method is shown in Figure 7A in an initial state, and in Figures 7B, 7C, and 7D in subsequent states.

[0062] As shown in Figure 7B, a unique QR code or other identifier, preferably embedding a URL, can be incorporated into the user interface immediately or after the first interaction with the user interface (step 610). For example, in Figure 7B, the QR code is added to the user interface of Figure 7A only if the viewer adds an item to their cart during video playback. In other embodiments, the QR code may already be present at the time the user interface is created.

[0063] The viewer then makes the data from the user interface available to the secondary device 135 (step 615). In a most preferred embodiment, the data is made available by extracting a URL embedded in the QR code. In other embodiments, the data may be extracted from another form of barcode, two-dimensional code, bitmap, or pattern that is visually identifiable in the user interface, or via text extraction with optical character recognition applied to the user interface. As previously mentioned, it is also contemplated to use a microphone rather than a camera to capture an audio cue instead of a visual cue. The cue may be an infrasonic or ultrasonic cue that is inaudible to the viewer, or it may be within the audible range and designed to interface with other software on a smartphone or other device that listens for audible cues, such as voice commands to Amazon's "Alexa" or Apple's "Siri". Additionally, instead of automatic capture by the camera or microphone, the viewer may be able to enter a URL, passcode, token, or other data string displayed in the user interface via a keyboard, touch screen, or other input method. If the data string does not directly contain a URL, an application running on the secondary device may be able to convert the data string into the expected URL format.

[0064] Using data from the user interface, the secondary device determines a URL to send the HTTP request to and sends it (step 620). The URL may be part of an API hosted and operated by the web server 315, and a request for a given URL constitutes a command or request to be executed by the server. If additional optional parameters are appended to the URL, they may constitute additional parameters or arguments associated with the command or request.

[0065] The server 315 receives the HTTP request from the secondary device and is thereby notified that a secondary device interaction is taking place (step 625). Depending on the purpose of the API operated by the server, the server may parse the URL to determine either the command or the request and its optional parameters and act according to the API. The server may not send any response to the HTTP request, may send a simple positive response such as an HTTP status 200 "OK" response code, may return a structured response such as a JSON array or other file containing the requested data in response to a query, or may provide a landing page or other webpage to be displayed by the secondary device 135. The viewer may continue to interact with the landing page or other webpage in parallel with and independent of subsequent steps.

[0066] Upon being notified of the interaction with the secondary device, the server 315 sends a message to the smart TV 100 (step 630). The message may simply contain the fact that a secondary device near the smart TV has contacted the server, or it may contain more detailed information, such as parameters that were set or changed during the interaction between the viewer, the secondary device, and the web server.

[0067] Code executing locally on the smart TV 100 acts on the received message (step 635) and updates the video or user interface in some way. For example, as shown in FIG. 7C, the user interface may be updated to display a message encouraging the viewer to continue interacting with the secondary device 135. If the original secondary device 135 is lost or unavailable, but another secondary device is found and can be located, the user interface may be updated and a QR code or some other alternative on-ramp may again be provided and interaction may continue again. Other responses or actions may include changing the underlying video being played, displaying additional user interface elements, changing the content or appearance of existing user interface elements, hiding user interface elements, or any other changes necessary to achieve the various functions described below.

[0068] At any point, the viewer's continued interaction with the smart TV's user interface through the secondary device or with the landing page or other web page displayed on the secondary device may cause the secondary device to make additional transmissions to the server (returning to step 620), which may create a constant loop in which both the smart TV's user interface and any user interface rendered on the secondary device are updated in response to viewer selections or inputs. For example, as shown in FIG. 7D, ongoing communication between software running locally on the smart TV and the web server may reveal whether a particular expected communication between the secondary device and the web server has occurred during a given period of time. As described in more detail below, subsequent messages or content may be displayed to the viewer to reward the viewer for taking an action or to encourage the viewer to take an action that they have not yet taken. Specific Applications of Broadcast Overlay Technology

[0069] As shown in Figures 7A, 7B, 7C, and 7D, a shopping cart can be overlaid on a video about a product (see Figure 7A). When an item is selected using input on the smart TV, a QR code is displayed (see Figure 7B) that contains not only the URL of the merchant selling that item, but also URL parameters embedded therein indicating which item has been added to the shopping cart. When the QR code is scanned with a smartphone or similar device, the shopping cart is replaced with a message prompting the viewer to complete the transaction on the merchant's web page (see Figure 7C) and / or to resume the transaction if it was interrupted in some way before completion. The viewer can purchase the item on the web page that is generated and sent to the smartphone. The software running locally on the television continues to communicate with the web server, and if a predetermined time (e.g., a one-minute or five-minute interval) passes without the viewer completing the purchase of the items in the cart, a subsequent message may be displayed to the viewer reminding the viewer that the purchase is incomplete (see FIG. 7D), and optionally encouraging the viewer to complete the purchase by offering a discount, allowing the viewer to skip the currently playing advertisement, offering a small additional product in the cart for free, displaying a prompting message, or otherwise offering something of objective or subjective value to the viewer.

[0070] In another embodiment, the home shopping channel may be used to facilitate an online auction for items displayed on the channel. Upon scanning a QR code or other visual trigger displayed in association with a product on the television screen, the viewer may use a smartphone to access and bid on the online auction. The display on the television may display any number of alerts or data as the viewer watches, such as the current bid price, the highest bid the viewer has previously submitted, whether the viewer is the current high bidder, an alert that the viewer has lost high bidder status, a countdown timer until the end of the auction, or any other useful information related to the auction. As a result, the viewer may continue to participate in the auction without having to continually divide their attention between the television and the phone, only needing to return to the phone when the television alerts them that they have lost high bidder status.

[0071] In another embodiment, a real-time collective vote of the viewers can be performed via the secondary device and displayed on the smart TV. For example, a question can be presented to all viewers, along with a QR code or other visual trigger code that directs the secondary device to a URL where the secondary device can select an answer to the vote. Once the web server receives the viewer's selection, the TV display is also updated to show the cumulative total of other viewers' responses as well as the selections recorded by the individual viewer, so the viewer knows that their vote has been received. This system allows for interactive polls and political surveys, as traditionally done by pollsters, but also interaction with reality and game shows. Viewers can vote for or against contestants and their submissions in real time (as is typical for television shows such as Big Brother, Eurovision, The Sing-Off, The Masked Singer, Project Runway, Top Chef, etc., which are typically done by phone votes only). Viewers may also provide real-time feedback during an ongoing performance or interaction, such as positive or negative reactions to individual statements in a political debate, or to form a critical mass to reject an ongoing performance (similar to how a Got Talent performance is terminated when all four judges vote to eject a contestant). In each example, the current feedback the viewer is providing is displayed to the viewer, along with optional other information, such as an overall vote tally or additional information about the option the viewer selected.

[0072] In another embodiment, real-time data regarding collective efforts, both individual and group, may be displayed on the television screen. For example, during a fundraiser, a QR code or other code may be displayed prompting a donation to a particular charity, non-profit organization, public radio or television station, etc. Once a smartphone or other secondary device is used to pledge or actually send a donation to a recipient, the television screen will display the viewer's personal donation amount, as well as the total amount of funds pledged or donated and a progress bar or "thermometer" showing progress toward the goal. If there are other benefits associated with the donation, such as access to a content stream without advertisements, access to a content stream without requests for additional donations, or access to coupon codes provided by the charity's partners, all of that data will also be provided instantly via the television.

[0073] In another embodiment similar to the previous two, collective action can be encouraged by displaying content, coupon codes, or other materials only if participation by all users meets a certain threshold. For example, if a television show is showcasing a product that may or may not be commercialized in the future (such as those featured on Shark Tank or similar entrepreneurial shows), viewers may be prompted to use a QR code or other code to go to an online page to pre-order the product. If enough pre-orders are made, viewers may be given a guarantee that the product will be produced and that their pre-orders will be filled, that an investor on the show will partner with the product's owner, or that the price will be discounted compared to the price non-backers will pay when the product is produced. Discounts may be offered only on the backend by associating a particular viewer with a pre-order commitment, or may be offered additionally or alternatively by providing a coupon code or other information that allows for a discount in the future.

[0074] In another embodiment, an interactive game may be displayed over the underlying video content. The interactive game may be exclusively local (e.g., two players watching TV in the same room) or remote (one player watching TV can play against an opponent in any location). After scanning a QR code or other visual trigger with the game interface to link with the smartphone, the viewer can input a move or selection and see the result on the TV along with the underlying video content. The TV can visually inform the viewer that it is their turn in a turn-based game, so that the viewer does not have to look at the smartphone to see if the game state is progressing without being distracted from the video content. If the game is instead a real-time game, the TV can display the underlying video content and the current game state in a split screen.

[0075] In another embodiment, the television can be linked to a social networking site on which the viewer has an account by scanning a QR code or other visual trigger. The viewer can then see new posts, comments, content, upvotes, likes, retweets, reactions, or other information associated with others' use of the social network in real time while watching the underlying video content.

[0076] In another embodiment, by scanning a QR code or other visual trigger, a smartphone can be linked to a television, allowing the smartphone to act as a remote control, with inputs on the smartphone communicated to a back-end server and sent to the television, allowing changes to be made to the channel, volume, or other settings associated with the television.

[0077] In another embodiment, the message queue available on the smartphone and associated with the viewer can also be linked to the television by scanning a QR code or other visual trigger: calendar items, to-do tasks, unread text messages, unread emails, voicemails, or other messages or reminders can continue to be displayed to the viewer along with the underlying video content.

[0078] In another embodiment, a public service announcement, public safety announcement, emergency weather alert or other announcement, or advertisement may be displayed to the viewer along with a QR code or other visual trigger for its associated website. If the viewer is willing to access the website by scanning the QR code or other visual trigger with a smartphone, the video may be short-circuited and the viewer may return to the content that was playing before the video was interrupted. Dynamic QR Code Example

[0079] 8A, 8B, 8C and 8D show successive instants which differ from the embodiment shown in FIG.

[0080] Initially, the video is displayed with an overlaid shopping menu icon in the top left corner and a shopping cart icon in the top right corner, as shown in Figure 8A.

[0081] Hovering over or clicking on the menu icon will reveal a cascading, multi-level menu displaying the various items that can be added to the cart, as shown in Figure 8B.

[0082] When the viewer clicks "Add" to add an item to their cart, the cart icon updates (now showing one item worth $100) and a QR code is generated in the bottom right corner, as shown in Figure 8C. When the QR code is scanned by a smartphone or other computing device at this stage, the smartphone navigates to a cart checkout page on the merchant's website, and the item is automatically added to the cart using a token contained in the URL encoded by the QR code.

[0083] When the viewer clicks "Add" for the second item, the cart icon updates again (now showing two items worth a total of $155) and the QR code updates so that it no longer contains the same token information as before. When the QR code is now scanned by a smartphone or other computing device, the smartphone is directed to a cart checkout page on the merchant's website and both items have been automatically added to the cart. Additional Dynamic QR Code Embodiments

[0084] As shown in Figures 2, 8C, and 8D, dynamic QR codes allow a shopping cart to be embedded into a web page without the need for a user to log in or use cookies. All information about the cart contents can be encoded into the URL itself, rather than being stored in a cookie or tied to a user's login.

[0085] A video player, or a website that serves the video (e.g., YouTube, TikTok, etc.), may display a QR code in association with the video, where the QR code contains information about the current timestamp of the video. If a viewer of the video needs to pause the video and step away from their computer, the viewer can use their smartphone to scan the QR code and open the same video on their smartphone already advanced to the same timestamp, without the viewer needing to log into the website that is displaying the video or requiring any other tracking device.

[0086] Digital ads displayed on websites, mobile devices, televisions, or VR environments may be augmented with a QR code that contains a hyperlink to a specific page on the advertiser's website. After the website and ad are loaded, a script on the hosting website may contact the advertiser's website with information about the user, such as identification information, demographic information, detected cookies, or other data. The advertiser's website may respond by providing another, more targeted hyperlink that is encoded by the QR code. If a user then scans the QR code to access the advertiser's website on their smartphone, the user may be presented with a different page than the page the generic version of the ad would direct them to, and / or the user may be tracked by a token embedded in the URL that the QR code encodes.

[0087] Digital ads may include QR codes that are automatically updated to contain data recording the user's interaction with the ad to that point, such as which elements of the ad were interacted with, for how long, and in what order.

[0088] An automated concierge or kiosk display may be configured to receive a set of choices of where a user would like to go and generate a QR code containing the locations / directions. When the QR code is scanned, a map app on the user's smartphone can automatically load the list of locations and directions to help the user navigate to the location.

[0089] A smartwatch, smartphone, or other portable display that also includes sensors can be used as a kind of medical "black box" that can be accessed by emergency personnel or other personnel who encounter an unconscious person. A QR code displayed on the smartwatch or smartphone screen can contain information such as the individual's pulse rate over the past hour, the time the individual last ate, the time the individual was last conscious, or other information that can be used to differentially diagnose an individual who is unconscious or unable to provide a medical history. By scanning the QR code, emergency personnel can instantly access that history, thereby providing better medical care to the patient. For example, an unconscious person having a heart attack may be indicated by a rapid and irregular pulse before a sudden loss of consciousness is detected by an accelerometer in a wearable device, while an unconscious person in a diabetic coma may be indicated by how many hours have passed since their last meal, or hypoglycemia may be indicated by a smart medical device constantly monitoring the individual's blood glucose levels. All of these variables can be invisibly displayed at all times by a dynamic QR code displayed on the smartwatch or smartphone. Similar "black box" functionality can be built into devices such as automobiles that have been crashed, appliances that have stopped functioning, or appliances that are still functioning but display status information or error messages to the appliance's user.

[0090] Computers, printers, machines, any other type of device or equipment can display a QR code that incorporates real-time information about the device's operating status, such as whether a component has failed, what is causing the slowdown, error codes, other diagnostic information, refueling or charging requirements, power usage, temperature information, etc. When the QR code is scanned with a smartphone or other device, the smartphone can respond by displaying specific instructions to a repair or maintenance worker, requesting a software update or patch, ordering a replacement part, etc.

[0091] A QR code may represent a digital ID, a digital signature, the encoding of access or permissions, or other object for the purpose of verifying the identity of an individual or device or document, and may or may not be accompanied by data that further informs the condition or state of the encoding of the object, such as a timestamp accompanying the signature, blockchain history, a snapshot of the current device state, or other information supporting the identification, verification, or current state. When any element of the recorded information is changed or added, the QR code is similarly updated to reflect the changed information.

[0092] QR codes can also be used to augment the actual carts, trays, and other containers used in selecting items in a brick-and-mortar store, cafeteria, or other facility. For example, the cart may include an RFID chip reader that detects a chip in each item as it is placed in the cart. As each item is collected, a QR code on a display associated with the cart is updated to include information about the contents and the total price of the contents. When the customer takes the cart to the register to check out, the cashier only needs to scan the QR code rather than handling and scanning each item individually. This can significantly speed up the checkout process, reduce errors, and improve the customer experience.

[0093] In a casino, an electronic gaming machine such as a slot machine or video poker may display a QR code when a player wins a prize, and then continually update the QR code as the player continues to play using the prize funds. For example, if a player first wins $1000 in poker, the QR code may initially represent that fact, but if the player uses this prize to pay a $20 ante on the next hand, the QR code may update to represent that the player has now won $980. If the player wins another $1000 on the next hand, the QR code may update again to $1980. At any point when the player wants to stop playing, they can either use their smartphone to capture the QR code and have the prize automatically deposited into an account of the player's choice, or they can take their smartphone to a cash-out window and prove (via the QR code and security token included in the balance) that the player is entitled to the prize. As a result, players do not need to carry or handle chips that can be lost or stolen. Computing technology in general

[0094] 9 is a high-level block diagram of a representative computing device that may be utilized to implement various functions and processes described herein, such as, for example, the secondary computing device 135, the broadcast origination server 300, the data stores 305 and 310, the web server 315, the computing device 400, the web server or other interface source device 405, the scanning computing device 410, or other web server 415 functionality. A computing device may be described in the general context of computer system executable instructions, such as program modules, executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types.

[0095] 9, a computing device is depicted in the form of a special purpose computer system. Components of the computing device may include (but are not limited to) one or more processors or processing units 900, a system memory 910, and a bus 915 that couples various system components including the memory 910 to the processor 900.

[0096] Bus 915 represents any one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a high-speed graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and without limitation, such architectures include Industry Standard Architecture (ISA) bus, MicroChannel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0097] The processing unit 900 may execute computer programs stored in memory 910. Any suitable programming language may be used to implement the routines of a particular embodiment, including C, C++, Java, assembly language, etc. A variety of programming techniques may be employed, such as procedural or object-oriented. The routines may be executed on a single computing device or multiple computing devices. Additionally, multiple processors 900 may be used.

[0098] A computing device typically includes a variety of computer system readable media, which can be any available media that can be accessed by the computing device and includes both volatile and nonvolatile media, removable and non-removable media.

[0099] The system memory 910 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 920 and / or cache memory 930. The computing device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 940 may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, typically referred to as a "hard drive"). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from and writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical medium, may also be provided. In such an example, each may be connected to the bus 915 by one or more data media interfaces. As further illustrated and described below, the memory 910 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments described in this disclosure.

[0100] Programs / utilities 950 having a set (at least one) of program modules 955 may be stored in memory 910, as well as, by way of example and not limitation, an operating system, one or more application software, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or combinations thereof, may include implementations of a network environment.

[0101] The computing device may also communicate with one or more external devices 970, such as a keyboard, a pointing device, a display, one or more devices that allow a user to interact with the computing device, and / or any devices that allow the computing device to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communications may occur through input / output (I / O) interface 960.

[0102] Additionally, as discussed above, the computing device may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (such as the Internet) via a network adapter 980. As shown, the network adapter 980 communicates with other components of the computing device via a bus 915. Although not shown, it should be understood that other hardware and / or software components may be used in conjunction with the computing device. Examples include (but are not limited to) microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data archival storage systems, and the like.

[0103] The description of various embodiments of the present invention is presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification have been selected to best explain the principles of the embodiments, practical applications or technical improvements to the technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0104] The present invention may be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0105] A computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, or a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards, or raised structures in grooves with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be construed as being a transitory signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a wave guide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.

[0106] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may comprise copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0107] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages ​​including object oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can utilize state information of the computer readable program instructions to execute computer readable program instructions to personalize the electronic circuitry in order to carry out aspects of the invention.

[0108] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0109] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram blocks. These computer readable program instructions can also be stored on a computer readable medium capable of directing a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer readable medium on which the instructions are stored includes an article of manufacture including instructions that implement aspects of the functions / acts specified in the flowchart and / or block diagram blocks.

[0110] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and a sequence of operational steps executed on the computer, other programmable apparatus, or other device to generate a computer-implemented process, where the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram blocks.

[0111] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown in the blocks may be performed in a different order than that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, as well as combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts, or executes a combination of dedicated hardware and computer instructions.

Claims

1. A system for creating an interactive user interface associated with an audiovisual broadcast, comprising: one or more servers for transmitting said audiovisual broadcasts to a display device; a processor of one or more of the one or more servers and the display device; When executed by the one or more processors, it causes the one or more processors to: transmitting, together with said audiovisual broadcast, instructions to generate, by a device displaying said audiovisual broadcast, a first user interface to be overlaid on said audiovisual broadcast; receiving a first command from a viewer of the audiovisual broadcast that results in an update of one or more values ​​associated with the first user interface; creating an identifier in the generated user interface overlay that is used to access a second user interface on a second device; Capturing the identifier at the second device; generating the second user interface on the second device; receiving a second command from the viewer when the viewer uses the second device; updating said one or more values ​​again; updating the first user interface to display information related to the updated one or more values ​​from within the second user interface. A non-transient memory for storing instructions; A system including:

2. The system of claim 1 , wherein the second device is a mobile computing device and the identifier is captured via a camera of the mobile computing device.

3. The system of claim 1 , wherein the identifier is encoded as a Quick Response (QR) code.

4. 4. The system of claim 3, wherein the quick response (QR) code is dynamic and changes after its initial creation in response to commands subsequent to the first command from the viewer of the audiovisual broadcast.

5. When executed by the one or more processors, the instructions further cause the one or more processors to: determining that an expected command has not been received within the second user interface; updating the first user interface to display a prompt to resume interaction with the second user interface or a new user interface on a new device different from the second device; The system of claim 1 .

6. When executed by the one or more processors, the instructions further cause the one or more processors to: determining, via the capture of the identifier at the second device, that the viewer operating the second device is viewing the audiovisual broadcast; Varying future output of said audiovisual broadcast in response to said determination. The system of claim 1 .

7. When executed by the one or more processors, the instructions further cause the one or more processors to: using the generated user interface overlay as a shopping cart; upon receiving a selection of an item by a user in the generated user interface overlay, tracking the insertion of the item into the shopping cart; Completing the shopping cart transaction within the generated user interface overlay. The system of claim 1 .

8. A method for creating an interactive user interface associated with an audiovisual broadcast, comprising the steps of: transmitting, together with the audiovisual broadcast, instructions to generate a first user interface to be overlaid on the audiovisual broadcast by a device displaying the audiovisual broadcast; receiving a first command from a viewer of the audiovisual broadcast that results in an update of one or more values ​​associated with the first user interface; creating an identifier within the generated user interface overlay that is used to access a second user interface on a second device; capturing the identifier at the second device; and generating the second user interface on the second device; receiving a second command from the viewer when the viewer uses the second device; updating said one or more values ​​again; and updating the first user interface to display information related to the updated one or more values ​​from within the second user interface; The method includes:

9. The method of claim 8 , wherein the second device is a mobile computing device and the identifier is captured via a camera of the mobile computing device.

10. The method of claim 8 , wherein the identifier is encoded as a Quick Response (QR) code.

11. 11. The method of claim 10, wherein the quick response (QR) code is dynamic and changes after its initial creation in response to commands subsequent to the first command from the viewer of the audiovisual broadcast.

12. When executed by the one or more processors, the instructions further cause the one or more processors to: determining that an expected command has not been received within the second user interface; updating the first user interface to display a prompt to resume interaction with the second user interface or a new user interface on a new device different from the second device; The method according to claim 8.

13. When executed by the one or more processors, the instructions further cause the one or more processors to: determining, via the capture of the identifier at the second device, that the viewer operating the second device is viewing the audiovisual broadcast; Varying future output of said audiovisual broadcast in response to said determination. The method according to claim 8.

14. When executed by the one or more processors, the instructions further cause the one or more processors to: using the generated user interface overlay as a shopping cart; upon receiving a selection of an item by a user in the generated user interface overlay, tracking the insertion of the item into the shopping cart; Completing the shopping cart transaction within the generated user interface overlay. The method according to claim 8.

15. 1. A method for streaming one or more images incorporating a truly dynamic QR code, comprising: generating a first set of one or more images including a first QR code and encoding data representing one or more variable values ​​associated with a user interface or environment in which the QR code is displayed; receiving an input event indicating that a value change has occurred to at least one of the one or more variable values ​​or that at least one additional value has been added or changed; generating a second set of one or more images as a continuation of the stream comprising said first set of one or more images, wherein said first QR code is replaced with a second QR code encoding data representative of said one or more variable values ​​associated with said digital display, including said change in value of said at least one variable value; The method includes:

16. 16. The method of claim 15, wherein the second QR code is scanned by a camera of a mobile computing device and the one or more variable values ​​are communicated to another user interface displayed on the mobile computing device.

17. 16. The method of claim 15, wherein the first QR code and the second QR code lead to the same URL but have different parameters encoded in the request for that URL.

18. 16. The method of claim 15, wherein an operator of a user interface can change the target of a QR code in a user interface by replacing the first QR code with the second QR code experienced on the client side, rather than creating a server-side redirection of the target of the first QR code.

19. 16. The method of claim 15, wherein animation is used to hide that a first QR code is no longer present and has been replaced by the second QR code.

Images