Participating and synchronizing using received audio or visual cues

Abstract

Methods, apparatus, and computer readable media for engagement and synchronization using received audio or visual cues. A method for content synchronization includes receiving, by an electronic device (300), video content for display on a display (355) associated with the electronic device; identifying, by the electronic device, an indicator included in the video content (606); and in response to identifying the indicator, identifying, by an electronic device, additional content to be included with the video content displayed on the display (608). The method further includes determining, by the electronic device, when the additional content is to be included with the video content based on the video content; and providing, by the electronic device, the additional content for display on the display together with the video content based on the determination.

Description

Technical Field

[0001] The present disclosure relates generally to electronic devices and more particularly to engagement and synchronization using received audio or visual cues. Background Art

[0002] The use of electronic devices has greatly expanded due in large part to their usability, convenience, computing power, etc. One result of recent technological developments is that electronic devices have become increasingly compact, while the number of functions performed by a given device has increased. For example, some electronic devices support functions such as digital media playback, game play, e-book reading, digital image capture, calendar scheduling, wireless communications, Internet searches, etc. (to name a few). With the rise of mobile computing technology, users can interact with a variety of functions and be provided with new information. Summary of the invention

[0003] Embodiments of the present disclosure allow for participation and synchronization using received audio or visual cues.

[0004] In one embodiment, a method for content synchronization is provided. The method includes: receiving, by an electronic device, video content for display on a display associated with the electronic device; identifying, by the electronic device, an indicator included in the video content; and in response to identifying the indicator, identifying, by the electronic device, additional content to be included with the video content displayed on the display. The method also includes: determining, by the electronic device, based on the video content, when to include the additional content with the video content; and providing, by the electronic device, the additional content for display on the display together with the video content based on the determination.

[0005] In another embodiment, an electronic device is provided. The electronic device includes: a communication interface configured to receive video content for display on a display associated with the electronic device; and a processor operably coupled to the communication interface. The processor is configured to: identify an indicator included in the video content; in response to identifying the indicator, identify additional content to be included with the video content displayed on the display; based on the video content, determine when to include the additional content with the video content; and based on the determination, provide the additional content for display on the display with the video content.

[0006] In yet another embodiment, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium includes program code, which, when executed by a processor of an electronic device, causes the electronic device to: receive video content for display on a display associated with the electronic device; identify an indicator included in the video content; in response to identifying the indicator, identify additional content to be included with the video content displayed on the display; based on the video content, determine when to include the additional content with the video content; and based on the determination, provide the additional content for display on the display with the video content.

[0007] Other technical features may be readily apparent to those skilled in the art from the following drawings, descriptions and claims.

[0008] As used herein, the term "coupling" and its derivatives refer to any direct or indirect communication between two or more elements, whether or not these elements are in physical contact with each other. The terms "send", "receive" and "communicate" and their derivatives include both direct and indirect communication. The terms "include (include) and include (comprise)" and their derivatives mean including but not limited to. The term "or" is inclusive, meaning and / or. The phrase "associated with ......" and its derivatives mean including ..., included in ..., interconnected with ..., included in ..., connected to ... or connected with ..., coupled to ... or coupled with ..., communicable with ..., collaborative with ..., interlaced, juxtaposed, close to ..., bound to ... or bound with ..., have, have ... property, with ... related to or with ... have a relationship, etc. The term "controller" means any device, system or part thereof that controls at least one operation. Such a controller can be implemented by hardware or a combination of hardware and software and / or firmware. The function associated with any specific controller can be centralized or distributed, whether local or remote. When used with a list of items, the phrase "at least one of" means that different combinations of one or more of the listed items may be used, and only one of the items in the list may be required. For example, "at least one of A, B, and C" includes any of the following combinations: A; B; C; A and B; A and C; B and C; and A and B and C.

[0009] In addition, the various functions described below can be implemented or supported by one or more computer programs, each of which is formed by a computer-readable program code and is embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, processes, functions, objects, classes, instances, related data or a part thereof suitable for implementation with a suitable computer-readable program code. The phrase "computer-readable program code" includes any type of computer code, including source code, object code and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disc (CD), a digital video disc (DVD) or any other type of memory. "Non-transitory" computer-readable media excludes wired, wireless, optical or other communication links that transmit temporary electrical signals or other signals. Non-transitory computer-readable media include media in which data can be stored permanently and media in which data can be stored and overwritten later, such as rewritable optical discs or erasable memory devices.

[0010] Definitions for certain other words and phrases are provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most, instances, such definitions apply to prior, as well as future uses of such defined words and phrases. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:

[0012] Figure 1 An example communication system according to one embodiment of the present disclosure is illustrated;

[0013] Figure 2 and Figure 3 An example electronic device according to one embodiment of the present disclosure is illustrated;

[0014] Figure 4A illustrates an example timing diagram according to one embodiment of the present disclosure;

[0015] Figure 4B An example method for engaging in accordance with one embodiment of the present disclosure is illustrated;

[0016] Figure 5A and Figure 5B An example method for synchronizing content in a video according to one embodiment of the present disclosure is illustrated; and

[0017] Fig. 6A , Figure 6B and Figure 6C An exemplary diagram illustrating an electronic device displaying content according to an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0018] Discussed below Figures 1 to 6C The various embodiments used to describe the principles of the present disclosure in this patent document are only illustrative and should not be interpreted in any way as limiting the scope of the present disclosure. Those skilled in the art will appreciate that the principles of the present disclosure can be implemented in any appropriately arranged system or device.

[0019] According to the embodiments of the present disclosure, the electronic device may include a user equipment (UE) such as a mobile phone. The electronic device may also refer to any component such as a mobile station, a subscriber station, a remote terminal, a wireless terminal, a receiving point, a vehicle or a user device. The electronic device may be a mobile phone, a smart phone, a monitoring device, an alarm device, a fleet management device, an asset tracking device, a car, a desktop computer, an entertainment device, an infotainment device, a vending machine, an electric meter, a water meter, a gas meter (gas meter), a safety device, a sensor device, a household appliance, etc. In addition, the electronic device may include a personal computer (such as a laptop computer, a desktop computer), a workstation, a server, a television, a household appliance, etc. In certain embodiments, the electronic device may be a portable electronic device, such as a portable communication device (such as a smart phone or a mobile phone), a laptop computer, a tablet computer, an e-book reader (such as an e-reader), a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a virtual reality headset, a portable game console, a camera and a wearable device, etc. In addition, the electronic device may be at least one of a part of a piece of furniture or a building / structure, an electronic board, an electronic signature receiving device, a projector or a measuring device. The electronic device is one or a combination of the devices listed above. In addition, the electronic devices as disclosed herein are not limited to the devices listed above, and may include new electronic devices according to the development of technology. It should be noted that as used herein, the term "user" may refer to a person using an electronic device or another device (such as an artificial intelligence electronic device).

[0020] The embodiment of the present disclosure considers that content developers strive to provide novel and unique methods for user interaction with electronic devices.For example, successful content development can provide useful content and the information to users at the time when users may want information.That is, if useful information is provided at a time that is inconvenient for the user or at a time when the user is not interested in the content, the content provided may be ignored by the user.For example, if a user is watching a video, and content related to another scene in the video is provided to the user (such as by pop-up notification, overlay, etc.), the user may ignore the information because the information is irrelevant to the current scene.However, if a user is watching a video (including pre-recorded events or live broadcasts) and a certain event occurs, the user may find that a certain information related to the event is useful when the event occurs.Similarly, if a user is playing a video game, if the user is not at a certain position in the game, then simply displaying information at a certain time instance in the game may be useless to the user.In addition, if the user's information in the game may be a useful position, but a certain object is not displayed, the information may still be useless to the user.

[0021] Embodiments of the present disclosure provide systems and methods for synchronizing when providing information to a user.For example, if a video is a live broadcast of a live event that can also be viewed at a later time, embodiments of the present disclosure describe how to provide information at a time relevant to the user.

[0022] Embodiments of the present disclosure also provide systems and methods for including indicators in video streams. Video streams can be accessed on video-based platforms (or streaming services) such as YOUTUBE, TWITCH, FACEBOOK, etc. As disclosed herein, an indicator can be included at a certain location in the video that, when detected on a viewing device associated with a user, can cause an overlay or pop-up to display certain information that the viewer may find useful.

[0023] Embodiments of the present disclosure also provide systems and methods for providing a reward system for interacting with a user of an electronic device. For example, a content developer can provide rewards (physical or digital) to a user based on certain actions performed by the user. For example, a content developer can provide a reward to a user for passing by or viewing certain content at a certain location.

[0024] The embodiments of the present disclosure also provide systems and methods for providing physical feedback to users. In a VR or augmented reality (AR) environment, video stimulation can be provided via a head-mounted display (HMD), and audio stimulation can be provided through headphones or speakers. The embodiments of the present disclosure describe using tactile feedback on a controller held by a user to provide another immersive element (in addition to video and / or auditory stimulation) to the user. If the user is playing a displayed video game, the user interface device can provide tactile feedback to the user. For example, an electronic device (such as an egg, an action figure, an action figure, a user interface controller (such as a mouse or a game controller)) with an input device (such as a microphone, a light sensor, etc.) can vibrate in response to receiving an input (such as when receiving a certain sound or detecting a certain light color, wavelength, flash pattern, etc.). For example, the controller can vibrate in response to the controller detecting a certain light-based signal (such as a predefined pulse light pattern, a certain color with a certain frequency pulse). For another example, when the controller detects a certain sound output from a speaker when the user is playing a video game, the controller can vibrate. Vibrations may occur in selected situations in response to detecting specific sounds, further immersing the user in the game play (in addition to the visual and audio stimulation).

[0025] Figure 1 An example networked system 100 is illustrated in which various embodiments of the present disclosure may be implemented. Figure 1 The embodiment of the networking system 100 shown in FIG. 1 is for illustration only. Other embodiments of the networking system 100 may be used without departing from the scope of the present disclosure.

[0026] As in Figure 1 As shown in FIG. 1 , system 100 includes a network 101 that facilitates communication between various components in system 100. For example, network 101 can communicate Internet Protocol (IP) packets or other information between network addresses. Network 101 can include one or more local area networks (LANs); metropolitan area networks (MANs); wide area networks (WANs); virtual private networks (VPNs); all or part of a global network (such as the Internet); or any other one or more communication systems located at one or more locations.

[0027] The network 101 facilitates communication between various servers 102-104 and various client devices 106-114. Each of the servers 102-104 can be any suitable (one or more) electronic computing or processing device that can provide computing services (including software for one or more client devices 106-114). For example, each of the servers 102-104 may include one or more processing devices, one or more memories storing instructions and data, and one or more network interfaces that facilitate communication through the network 101. For example, as discussed in more detail below, the server 102 can use one or more of the client devices 106-114 to: provide instructions for synchronizing content in media content or manage instructions for synchronizing content in media content, provide user feedback (such as tactile feedback) or manage providing user feedback (such as tactile feedback), etc. For another example, the server 102 can provide or manage a reward system based on notifications received from one or more of the client devices 106-114. The reward system can be associated with an account associated with a user and linked to one or more of the client devices 106-114.

[0028] Each client device 106-114 represents any suitable electronic computing or processing device that interacts with at least one server or (one or more) other computing devices through the network 101. In this embodiment, the client devices 106-114 include a desktop computer 106, a mobile phone or smart phone 108, a tablet computer 110, a laptop computer 112, a video game console 114; a set-top box and / or a television, etc. However, any other or additional client devices can be used in the networked system 100. For example, any Internet or network-connectable device or Internet of Things (IoT) device (e.g., a smart TV, a refrigerator, a Raspberry PI, etc.) can be used as one of the client devices 106-114 in the system 100. For another example, a peripheral device (such as a game controller, a mouse, a keyboard, etc.) can also be used as one of the client devices 106-114 in the system 100. As discussed below, in various embodiments, the client devices 106-114 participate in processing various tasks under the coordination of the server 102, such as sending and receiving signals for user participation purposes. For example, as discussed herein, the client devices 106-114 may run various video games or interactive applications that include the ability to display content in a synchronized manner, perform feedback to the user (such as tactile feedback), etc. based on events in the video game. In addition, in various embodiments, any of the client devices 106-114 may generate, send, and / or analyze signals such as sound signals, radio frequency (RF) signals, light signals (such as various colors, varying wavelengths, various frequencies of light pulses, etc.), other types of electromagnetic signals or waves, etc. In some embodiments, the individual client devices 106-114 may communicate with each other or with a server directly or indirectly using, for example, a peer-to-peer, ad hoc, and / or mesh-based network (with or without a centralized server). In certain embodiments, the individual client devices 106-114 may communicate with each other or with a server using Wi-Fi, near field communication (NFC), Bluetooth, infrared, sound, or any other RF signal.

[0029] although Figure 1 One embodiment of the networked system 100 is illustrated, but may be used for Figure 1 For example, system 100 may include any number of each component in any suitable arrangement, and each of servers 102-104 and client devices 106-114 may represent any number of servers and / or client devices that are part of system 100. In general, computing and communication systems have a wide variety of configurations, and Figure 1 The scope of the present disclosure is not limited to any specific configuration. Figure 1One operating environment is illustrated in which the various features disclosed in this patent document may be used, but these features may also be used in any other suitable system.

[0030] Figure 2 and Figure 3 An example computing device in a networked system according to various embodiments of the present disclosure is illustrated. Specifically, Figure 2 An example server 200 is illustrated, and Figure 3 An example client device 300 is illustrated. In this exemplary embodiment, the server 200 represents Figure 1 Any one of the servers 102-104 in the embodiment, and the electronic device 300 may represent Figure 1 One or more of the client devices 106-114 in.

[0031] As in Figure 2 As shown in , the server 200 includes a bus system 205 that supports communication between (one or more) processors 210, storage devices 215, communication interfaces 220, and input / output (I / O) units 225. (One or more) processors 210 execute instructions that may be loaded into memory 230. (One or more) processors 210 may include any (one or more) suitable number and type of processors or other devices in any suitable arrangement. Example types of (one or more) processors 210 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuit systems.

[0032] Memory 230 and persistent storage 235 are embodiments of storage device 215, which represents any (one or more) structures capable of storing (temporarily or permanently) and facilitating retrieval of information (such as data, program code, and / or other suitable information). Memory 230 may represent random access memory or any (one or more) other suitable volatile or non-volatile storage device. Persistent storage 235 may include one or more components or devices that support long-term storage of data, such as read-only memory, hard disk, flash memory, or optical disk. For example, persistent storage 235 may store one or more data databases, client applications, reward systems (including accumulated rewards), video data, text data, audio data, etc., for distribution via network 101.

[0033] Communication interface 220 supports communications with other systems or devices. For example, communication interface 220 may include a network interface card or a wireless transceiver that facilitates communication over network 101. Communication interface 220 may support communication over any (one or more) suitable physical or wireless communication links.

[0034] I / O unit 225 allows input and output of data. For example, I / O unit 225 can provide a connection for user input through a keyboard, mouse, keypad, touch screen, or other suitable input device. I / O unit 225 can also send output to a display, printer, or other suitable output device.

[0035] although Figure 2 One embodiment of the server 200 is illustrated, but the Figure 2 For example, the following can be combined, further subdivided or omitted according to specific needs. Figure 2 The various components in and additional components may be added. As a specific example, although depicted as one system, the server 200 may include multiple server systems that may be located remotely. In another example, according to various embodiments of the present disclosure, different server systems may provide some or all of the processing, storage and / or communication resources for synchronization and participation purposes.

[0036] Figure 3 An example client device 300 is illustrated according to an embodiment of the present disclosure. Figure 3 The embodiment of the electronic device 300 illustrated in FIG. 1 is for illustration only, and Figure 1 One or more of the client devices 106-114 may have the same or similar configuration. However, client devices have a variety of configurations, and Figure 3 The scope of the present disclosure is not limited to any particular implementation of an electronic device.

[0037] As in Figure 3 As shown in FIG. 3 , the electronic device 300 includes an antenna 305, a communication interface 310, a microphone 320, and a speaker 330. The electronic device 300 also includes a processor 340, an I / O interface 345, an input device 350, a display 355, a memory 360, and (one or more) sensors 365. The memory 360 includes an operating system (OS) 361 and one or more applications 362.

[0038] The communication interface 310 may be similar to Figure 2The communication interface 220. The communication interface 310 supports communication with other systems or devices. For example, the communication interface 310 may include a network interface card or a wireless transceiver that facilitates communication through the network 101. The communication interface 310 may support communication through any (one or more) suitable physical or wireless communication links. For embodiments utilizing wireless communication, the communication interface 310 may include an RF transceiver, a Bluetooth transceiver, a WI-FI transceiver, a ZIGBEE transceiver, an infrared transceiver, a radio frequency identifier (RFID) tag (e.g., active or passive) and various other transceivers for receiving and / or sending wireless communication signals. The communication interface 310 may receive (via antenna 305) incoming RF signals using a variety of wireless communication protocols (such as Bluetooth, Wi-Fi, cellular, LTE communication protocols, etc.). The signal may be sent from an access point (such as a base station, a WI-FI router or a Bluetooth device) or other device of a network 101 (such as WI-FI, Bluetooth, cellular, 5G, LTE, LTE-A, WiMAX or any other type of wireless network). Communication interface 310 may also receive data from the processor to be sent via RF signals to an access point or another device of network 101 .

[0039] Microphone 320 according to one embodiment can receive sound and convert the sound into an electrical signal. The received sound can be in the audible spectrum, the non-audible spectrum, or both.

[0040] The speaker 330 according to one embodiment can output the electrical signal as sound. For example, the speaker 330 can output the sound signal to the outside of the electronic device 300. The speaker 330 can be used for general purposes, such as playing audio content. In some embodiments, the speaker 330 can output sounds in both the audible spectrum and the non-audible spectrum.

[0041] Processor 340 may include one or more processors or other processing devices. Processor 340 may execute instructions stored in memory 360 (such as OS 361) to control the overall operation of electronic device 300. Processor 340 may include any (one or more) suitable number and type of processors or other devices in any suitable arrangement. For example, in certain embodiments, processor 340 includes at least one microprocessor or microcontroller. Example types of processor 340 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuit systems.

[0042] The processor 340 is also capable of executing other processes and programs resident in the memory 360, such as operations to receive and store data. The processor 340 can move data into or out of the memory 360 as required by the execution process. In some embodiments, the processor 340 is configured to execute one or more applications 362 based on the OS 361 or in response to a signal received from (one or more) external sources or operators. For example, the application 362 may include a video game, a video, a location recognition application (such as a global positioning system (GPS)), a VR or AR application, a camera application (for still images and video), a video phone call application, an email client, a social media client, an SMS messaging client, a virtual assistant, etc.

[0043] Processor 340 is also coupled to I / O interface 345, which provides the electronic device 300 with the ability to connect to other devices, such as client devices 106-114. I / O interface 345 is the communication path between these accessories and processor 340.

[0044] The processor 340 is also coupled to an input device 350 and a display 355. An operator of the electronic device 300 can use the input device 350 to input data or input information into the electronic device 300. The input device 350 can be a keyboard, a touch screen, a mouse, a trackball, a controller (such as a video game controller), a joystick, a voice input device, a sound device, or other devices that can act as a user interface to allow the user to interact with the electronic device 300. For example, the input device 350 may include voice recognition processing, thereby allowing the user to enter voice commands. In another embodiment, the input device 350 may include a touch panel, a (digital) pen sensor, a key, or an ultrasonic input device. The touch panel can recognize, for example, touch input in at least one scheme (such as a capacitive scheme, a pressure sensitive scheme, an infrared scheme, or an ultrasonic scheme). The input device 350 can be associated with (one or more) sensors 365 and / or cameras by providing additional input to the processor 340. In some embodiments, the sensor 365 includes one or more inertial measurement units (IMUs) (such as accelerometers, gyroscopes, and magnetometers), motion sensors, optical sensors, cameras, pressure sensors, heart rate sensors, altimeters, etc. The input device 350 may also include a control circuit. In a capacitive scheme, the input device 350 may recognize touch or proximity.

[0045] Display 355 can be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED), an active matrix OLED (AMOLED), or other display capable of rendering text and / or graphics (such as from a website, video, game, image, etc.). Display 355 can be sized to fit within the HMD. Display 355 can be a single display screen or multiple display screens capable of creating a stereoscopic display. In some embodiments, display 355 is a head-up display (HUD, head-up display). Display 355 can display 3D objects, such as animated 3D objects.

[0046] Memory 360 is coupled to processor 340. A portion of memory 360 may include RAM, and another portion of memory 360 may include flash memory or other ROM. Memory 360 may include a persistent storage device (not shown), which represents any (one or more) structures capable of storing and facilitating retrieval of information (such as data, program code, and / or other suitable information). Memory 360 may include one or more components or devices that support long-term storage of data, such as a read-only memory, a hard drive, flash memory, or an optical disk. Memory 360 may also include media content. Media content may include various types of media, such as images, videos, three-dimensional content, VR content, AR content, animations, and static 3D objects, etc.

[0047] The electronic device 300 also includes one or more sensors 365, which can measure physical quantities or detect the activation state of the electronic device 300 and convert the measured or detected information into electrical signals. For example, the sensor 365 may include one or more buttons for touch input, cameras, gesture sensors, IMU sensors (such as gyroscopes or gyroscope sensors and accelerometers), eye tracking sensors, air pressure sensors, magnetic sensors or magnetometers, grip sensors, proximity sensors, color sensors, biophysical sensors, temperature / humidity sensors, lighting sensors, ultraviolet (UV) sensors, electromyography (EMG) sensors, electroencephalography (EEG) sensors, electrocardiography (ECG) sensors, infrared sensors, ultrasonic sensors, iris sensors, fingerprint sensors, color sensors (such as red, green, and blue (RGB) sensors / cameras), depth sensors, D-RGB sensors (depth red, green, and blue sensors / cameras), position sensors (such as GPS), RFID receivers, etc. The sensor 365 may also include a control circuit for controlling any sensor included therein.

[0048] As discussed in more detail below, one or more of these sensors 365 may include a tactile sensor. A tactile sensor converts an electrical signal into a mechanical stimulus (e.g., vibration or motion) or an electrical stimulus that can be recognized by a user via their sense of touch or kinesthetic sense. According to one embodiment, the tactile sensor may include, for example, a motor, a piezoelectric element, an electrical stimulator, etc.

[0049] although Figure 2 and Figure 3 An embodiment of an electronic device is illustrated, but the Figure 2 and Figure 3 For example, the following can be combined, further subdivided or omitted according to specific needs. Figure 2 and Figure 3 As a specific embodiment, the processor 340 may be divided into multiple processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). In addition, as with computing and communication, electronic devices and servers may adopt a variety of configurations, and Figure 2 and Figure 3 The present disclosure is not limited to any specific electronic device or server.

[0050] Embodiments of the present disclosure describe various methods for engaging users, such as by providing proof of experience, including providing tactile feedback when certain sounds or signals are detected, recording the user's attendance, and rewards. For example, proof of experience can provide user credit, which can enhance the user experience by trading in physical and digital objects with accumulated credit from one or more previous engagements. Users can be provided credit for viewing certain content, being at a certain location, etc. For another example, proof of experience can also provide tactile feedback to users. This is described in the following examples and embodiments (such as Figure 4A and Figure 4B are described in those of .

[0051] Figure 4A An example timing diagram 400 is illustrated according to one embodiment of the present disclosure. Figure 4B An example method 470a for participating is illustrated according to one embodiment of the present disclosure. Figure 4B The steps of method 470a may be performed by Figure 1 Any of the client devices 106-114 (such as Figure 3 The timing diagram 400 and the method 470a are for illustration only, and other implementations may be used without departing from the scope of the present disclosure.

[0052] Figure 4A The timing diagram 400 of FIG. 4 illustrates an electronic device 410, an electronic device 420, and a server 430. The electronic device 410 and the electronic device 420 may be similar to Figure 1 Any of the client devices 106-114, and including similar Figure 3 The internal components of the electronic device 300. Similarly, the server 430 can be similar to Figure 1 Any one of the servers 102-104, and including a similar Figure 2 Internal components of server 200. In some embodiments, server 430 can be omitted from timing diagram 400.

[0053] In some embodiments, electronic device 410 is a portable device, while electronic device 420 is stationary. In this embodiment, server 430 can provide a reward to an account associated with a user of electronic device 410, such as when electronic device 410 moves within a predefined distance from electronic device 420.

[0054] In this embodiment, electronic device 410, which is a portable device, sends a signal (step 442). For example, electronic device 410 may move with the user. For example, electronic device 410 may be carried by the user (such as in the user's hand), located in a pocket of clothing worn by the user, attached to clothing worn by the user, located in a bag carried by the user, etc. The signal sent by electronic device 410 may be emitted by a speaker (such as a Figure 3 The signal transmitted by the electronic device 410 may be an RF signal, an infrared (IR) signal, a sound signal, a light signal, or the like. The signal may be continuously transmitted, or transmitted based on a pattern or a detected event. In some embodiments, the electronic device 410 may include an RFID tag to actively or passively communicate such a signal. For example, an RFID receiver at a location may detect the presence of an electronic device 410 near the location to receive a signal that may not necessarily be actively transmitted by the electronic device 410.

[0055] In step 444, electronic device 420 receives a signal sent by electronic device 410. In this embodiment, electronic device 420 is fixed and located at a specific location. For example, if electronic device 420 is located at a concert or conference and receives a signal sent from electronic device 410, it can be determined that electronic device 410 is located at the same location as electronic device 420.

[0056] In step 450, electronic device 420 performs an action. In this embodiment, the action may be to generate and send a notification to server 430. The notification may be sent to server 430 via network 101. The notification may inform server 430 that electronic device 410 and electronic device 420 are located at the same or similar location.

[0057] In step 460, server 430 receives the notification sent by electronic device 420. In step 462, server 430 performs operations. For example, server 430 can determine the location of electronic device 410 based on the known location of electronic device 420. Then, server 430 can find the reward account associated with the user of electronic device 410. When there is a reward account associated with the user of electronic device 410, the server provides a reward to the reward account. In some embodiments, the reward can have a monetary value. In some embodiments, the reward can be linked to the user's profile. The user's profile can indicate how many locations the user has visited. In some embodiments, the reward can be expressed as points. For example, points can be spent in a game to upgrade a role, improve the avatar representing the user, obtain digital content, etc.

[0058] In some embodiments, the electronic device 410 is fixed and the electronic device 420 may be fixed or portable. In this embodiment, the server 430 may provide a reward to an account associated with the user of the electronic device 420. It should be noted that this embodiment is different from the above embodiment, which previously described (i) the electronic device 410 is portable and the electronic device 420 is fixed, and (ii) the server 430 provides the reward to the electronic device 410.

[0059] In this embodiment, the electronic device 410 is fixed and is transmitted through a speaker (such as Figure 3 For example, the electronic device 410 may communicate with a speaker (such as a speaker 330) in an event (such as a concert environment, a business environment, an industrial environment, a residential environment, etc.) to produce a sound (step 442). Figure 3 The sound may be audible (such as a musical instrument or speech sound), inaudible, or both. In addition to or in lieu of sound waves, the electronic device 410 may also transmit RF signals.

[0060] As described above, the electronic device 420 can be fixed or portable. For example, the electronic device 420 can be carried by the user (such as in the user's hand), located in the pocket of the clothes worn by the user, attached to the clothes worn by the user, located in the bag carried by the user, etc. For another example, the electronic device 420 can be fixed, such as a listening device that is kept at a certain location to listen to certain sounds. For another example, the electronic device 420 can be used to interact with another electronic device (such as a game controller or a mouse).

[0061] In step 444, the electronic device 420 receives a signal sent by the electronic device 410. For example, the electronic device 420 may include a Figure 3 The microphone 320 of the microphone is used to convert the sound into an electrical signal. For another example, the electronic device 420 may include a microphone similar to Figure 3 The receiver (or transceiver) of the communication interface 310 is used to receive RF signals.

[0062] In step 450, the electronic device 420 performs an action. For example, based on the type of signal or an indicator associated with the signal, the electronic device 420 can perform various actions. In one embodiment, the electronic device 420 can send a notification to the server 430. The notification can inform the server that the electronic device 420 has received certain signals from another device (identified as the electronic device 410). For example, the notification can inform the server 430 that the electronic device 420 is located at a similar location to the electronic device 410 (because it can receive the signal sent by the electronic device 410). The notification can also inform the server 430 that the user of the electronic device 420 is listening to a certain audio. In another embodiment, the electronic device can provide tactile feedback to the user when receiving the signal. For example, the electronic device 420 can be a game controller (such as a mouse) that vibrates when a certain sound (sent from the electronic device 410) is detected when the user is playing a video game. In another embodiment, the action can involve unlocking the properties of the electronic device 420 or the properties associated with the electronic device 420. For example, the electronic device 420 can be an egg-shaped object or action figure associated with a game or video. Based on the reception of a certain number of received or detected chirps or other sounds (e.g., as evidence that a user with electronic device 420 was present at a location for a certain amount of time or consumed digital content for a certain amount of time), the egg-shaped object may hatch, or abilities associated with the action figure may be unlocked or enhanced.

[0063] In some embodiments, electronic device 420 may be associated with a digital item, such as a non-fungible token (NFT). In such an embodiment, electronic device 420 may be a personal electronic device of the owner of the digital item, and may detect that the user of electronic device 420 will receive a reward (e.g., for content consumption or location attendance, as previously discussed). As a result, server 430 may determine to improve or modify the value, appearance, or other quality of the digital item as a reward. In some embodiments, the digital item may be an action figure, character, or object in a game or video. In some embodiments, improving or modifying the quality of the digital item may involve modifying the appearance or abilities of the action figure in the game or video.

[0064] In other embodiments of actions performed, the actions may be associated with either or both of the physical and digital representations. For example, in addition to receiving or modifying a digital item, a physical object such as a badge or game object may have a modified physical property (such as light or color) and be associated with a digital item having a corresponding modified digital property.

[0065] Block 470 including step 444 and step 450 is Figure 4B is described in more detail in method 470a.

[0066] In step 460 , the server 430 receives the notification sent by the electronic device 420 .

[0067] In step 462, server 430 performs operations. For example, server 430 can determine the location of electronic device 420 based on the known location of electronic device 410. Server 430 can find a reward account associated with the user of electronic device 420. When there is a reward account associated with the user of electronic device 420, server 430 provides a reward to the reward account, for example, a reward can be given in exchange for the user being located in a similar location to electronic device 410. In some embodiments, the reward can have a monetary value. In some embodiments, the reward can be linked to the user's profile. For example, the user's profile can indicate how many locations the user has visited. In some embodiments, the reward can be represented as points. For example, points can be spent in a game to upgrade a role, improve an avatar representing a user, obtain digital content, obtain or modify a digital item, etc.

[0068] It should be noted that if the action performed by the electronic device 420 in step 450 is not to send a notification, steps 460 and 462 can be omitted. For another example, the electronic device 420 can store the information about the event locally for a period of time. The information about the event can be sent in response to another event occurring, such as when the electronic device is scanned at a later point in time.

[0069] Figure 4B The method 470a is described in more detail Figure 4A Block 470. In step 444, the electronic device 420 receives a signal. The signal may be sent from the electronic device 410. The signal may be audible, inaudible, RF, etc.

[0070] In step 446, the electronic device 420 identifies information related to the received signal. The information may be based on the signal itself, a sensor associated with the electronic device 420 (such as Figure 3 The information may be identified based on the type of signal (whether it is audio, RF, or a certain RF signal). For example, the relevant information may be identified based on the type of signal (whether it is audio, RF, or a certain RF signal). For example, a Bluetooth signal may indicate something different from an audible sound signal. Similarly, the signal itself may represent the relevant information. For another example, the relevant information may be identified based on whether the sensor of the electronic device 420 detects a physical measurement value (such as light, darkness, pressure, altitude, motion, etc.). For example, the relevant information may indicate whether the electronic device 420 is portable (compared to fixed) based on the detection of motion at the time when the proximity signal is received. For another example, the relevant information may be based on the device type of the electronic device 420. For example, the relevant information may indicate that the electronic device 420 is an interface item such as a mouse, keyboard, game controller, etc.

[0071] In step 448, the electronic device 420 determines the signal type. The signal may represent a proof of experience. The proof of experience may result in tactile feedback or physical movement of a peripheral device. The proof of experience may also provide a reward to the user that may be redeemed at a later date and / or location. The determination of the signal type may be based on the relevant information and received signal. For example, the determination of the signal type may be based on whether the signal is Bluetooth as opposed to audio-based sound. For another example, the determination of the signal type may be based on whether the electronic device 420 is mobile or stationary. For another example, the determination of the signal type may be based on the device class of the electronic device 420.

[0072] After determining (in step 448) that the signal is classified as a first signal type, then the electronic device 420 performs step 450a. For example, step 450a can be performed based on determining the location of the electronic device 420 as indicated by one or more sensors (such as at a concert venue or at a location where the electronic device 410 is located). For example, the location can be identified via a location-type sensor (such as GPS) of the electronic device 420. For another example, step 450a can be performed based on content associated with the signal. For example, the signal itself can specify that it is a first type of signal.

[0073] In step 452, electronic device 420 generates a notification. The notification may indicate the presence of electronic device 410. For example, the notification may indicate that electronic device 410 is within a predefined distance of electronic device 420. The notification may also indicate information associated with the content of the received signal. For example, the notification may indicate that the user of electronic device 420 is listening to certain content (based on the sound signal received from step 444). For example, when the user of electronic device 420 and electronic device 420 are co-located (such as when electronic device 420 is worn by the user), electronic device 420 may generate a notification via a microphone (such as a microphone). Figure 3 The microphone 320 of the electronic device 420 obtains the same sound signal as heard by the user. Like this, the generated notification can specify what the user of the electronic device 420 heard. Based on this indication, the server 430 can provide a reward to a reward account associated with the user of the electronic device 420.

[0074] In step 454, the electronic device sends a notification to the server 430. Based on the instructions included in the notification, the server 430 provides a reward account associated with the user of the electronic device 420.

[0075] After determining (in step 448) that the signal is classified as the second signal type, the electronic device 420 then performs step 450b. For example, step 450b may be performed based on content associated with the signal. For example, the signal itself may specify that it is a second type of signal. For another example, when the electronic device is classified as a certain device type (such as, a user interface controller similar to a mouse), step 450b may be performed.

[0076] In step 456, the electronic device 420 performs tactile feedback based on the received signal. In some embodiments, the electronic device 420 can perform different vibration patterns based on the different signals received. In other words, multiple signals can be classified as a second signal type, wherein each signal indicates that the electronic device 420 is to perform a different vibration pattern.

[0077] In these embodiments, users of electronic devices 410 / 420 are given rewards and / or attendance credits for passively viewing content or attending events without providing information or registering for the provided events or content. In addition, providers or sponsors of events or content can better identify participants and their interest in events and interact with participants without having participants register directly with providers or sponsors of events or content.

[0078] although Figure 4A A timing diagram 400 is illustrated and Figure 4B Method 470a is illustrated, but may be used for Figure 4A and Figure 4BVarious changes may be made. For example, although method 470a is shown as a series of steps, various steps may overlap, occur in parallel, occur in a different order, or occur multiple times. In another embodiment, a step may be omitted or replaced by another step. For example, the steps of method 470a may be performed in a different order.

[0079] Embodiments of the present disclosure describe various methods for synchronizing content in a video. This is described in the following examples and embodiments (such as Figures 5A-6C ).

[0080] Figure 5A and Figure 5B 5 and 510 respectively illustrate example methods 500 and 510 for synchronizing content in a video according to an embodiment of the present disclosure. Specifically, Figure 5A Method 500 describes a method for synchronizing content in a video performed by a server according to one embodiment of the present disclosure. Figure 5B Method 510 describes a method for synchronizing content in a video performed by a user device according to one embodiment of the present disclosure. Fig. 6A , Figure 6B and Figure 6C An exemplary diagram of an electronic device 602 displaying content according to one embodiment of the present disclosure is illustrated. Figure 5A The steps of method 500 may be performed by Figure 1 Any one of the servers 102-104 (such as Figure 2 Server 200) executes. Figure 5B The steps of method 510 may be performed by Figure 1 Any of the client devices 106-114 (such as Figure 3 The electronic device 300) is executed. Methods 500 and 510 and Figure 6A-6C The figures are for illustration only, and other embodiments may be used without departing from the scope of the present disclosure.

[0081] Figure 5A Method 500 of the present invention describes a server including an indicator for synchronizing content in a video stream. In step 502, server 300 obtains content associated with a predefined event in a video. The video may be live or pre-recorded. The video may be used for streaming to an end user's electronic device (such as a Figure 1 Any client device 106-114 and Figure 3 In some embodiments, the video may be VR or AR. The content may be generated by a third party unrelated to the creator of the video itself.

[0082] In step 504, the server 200 embeds the indicator into the video. The indicator is embedded (or added as an overlay) at a certain time instance corresponding to the predefined event in the video. In some embodiments, the indicator can be a pixel with a predefined color at a certain position added in the video frame corresponding to the predefined event. In some embodiments, the indicator can be a predefined shape at a certain position embedded in the video frame corresponding to the predefined event. In some embodiments, the indicator can be a barcode or a quick response (QR) code. After the video is embedded with the indicator, the video is received and watched by the end user.

[0083] In step 506, the server 200 receives notification that the pointer has been recognized by the viewing device. In step 508, the server 200 sends the obtained content to the viewing device for display. This enables the viewing device to display the content at a specific instance in response to the pixels appearing on the display of the viewing device.

[0084] Figure 5B Method 510 describes that a viewing device displays synchronized content in response to identifying an indicator in a video stream. For example, if different users watch a video stream at different time instances, synchronized content can be presented to the users based on a certain time or event occurrence in the video, rather than all viewers receiving content at the same time instance.

[0085] In step 512, the electronic device 300 receives and displays the video. The video may be a live event or a pre-recorded event. The video may be streamed. In step 514, the electronic device 300 identifies an indicator embedded at a certain point in the video while displaying the video. The indicator may be a certain pixel with a predefined color at a certain position in a frame or a series of frames.

[0086] In step 516, the electronic device 300 sends a notification to the server that a certain indicator is included in the video. The notification generated by the electronic device 300 may be sent to a specific server (such as Figure 2 Server 200 or Figure 4A The indicator parameters may include color, position in the frame, size, shape, etc. By sending the indicator parameters to the server, the server can determine what information to transmit to the electronic device 300. For example, the server may store the indicator parameters in its memory (such as Figure 2The memory 215 of the video controller 210 may include a lookup table that associates certain information to be sent as synchronous content with specific parameters of the indicator. In other embodiments, the electronic device 300 may use the parameters of the identified indicator for the electronic device 300 to determine which content to include in the video, without the need to send the parameters to the server for determination. In another embodiment, the indicator and / or the parameters of the indicator enable the electronic device 300 to know when to show content so that the additional content is synchronized with what is displayed. For example, this additional content may be preloaded on the client, wait for execution, or the indicator may indicate when to request additional content.

[0087] In step 518, the electronic device 300 can receive content from the server. In step 520, the electronic device 300 displays the received content. In some embodiments, the electronic device 300 displays content on the video as an overlay. For example, the creator of the content can request the viewer to vote for or provide feedback on the action associated with the content (for example, the action to be taken in the video game). Since the content may not be consumed when the content is created (for example, whether due to on-demand viewing or due to a small difference in time caused by transmission and / or processing delays), the electronic device 300 can present the overlay at the appropriate time during the display content, regardless of when the content is created. In other embodiments, the overlay content can be supplementary content, so that the viewing experience for the viewer is personalized or improved for the viewing experience for the viewer. For example, the supplementary content can be a modification of the appearance of an object or character in the content or reward content, so that some viewers can have a different or improved viewing experience.

[0088] In other embodiments, the content may already be on the electronic device 300. For example, the electronic device 300 sees the indicator, and the electronic device 300 can react and display the content without having to signal the server to obtain the requested content. For example, the content can be preloaded on the client, and the electronic device 300 uses the indicator as a way to synchronize when to display the local content, for example, to reduce the delay caused by having to request the content.

[0089] In some embodiments, the indicator identified in step 514 may be a coded symbol, such as a barcode or QR code. The electronic device 300 may scan (or read) the indicator and display the content as an overlay, as described above in step 520. In this embodiment, steps 516 and 518 may be omitted. The electronic device 300 uses the Internet (or a web browser such as a web browser) to access the content. Figure 1The embedded indicator can be read and interpreted by the electronic device 300 thereon, and the overlay display can be started immediately without having to be communicated back to any server. For example, the electronic device 300 can recognize a "start game" command via the embedded indicator, and then the electronic device 300 will interpret the command to make an overlay running locally on the browser of the electronic device 300 to pop up a "game start" menu.

[0090] In other embodiments, step 516 may be performed without step 518. For example, the electronic device 300 may perform signaling at 516 that is synchronized with the server streaming code, so that the signaling back to the server does let the server know when the content at the electronic device 300 begins. For example, in a video streaming scenario, this signaling can let the live streamer know when their audience members have finished the current event. For example, if the streamer wants to select a winner from a drawing, the streamer will need to know when everyone has finished entering their name, for example, which means that the streamer may need to wait for the audience members to see the beginning and end portions of the event being streamed on the video, and then select the winner from the entrants.

[0091] Fig. 6A , Figure 6B and Figure 6C An electronic device 602 is illustrated that displays content on a display 604. The electronic device 602 may be similar to Figure 1 Any client device 106-114, Figure 3 The electronic device 300, the electronic device 410, the electronic device 420, etc. The display 604 may be similar to Figure 3 The display 355 of the electronic device 300.

[0092] Fig. 6A The electronic device 602 is illustrated at time T0. Figure 6B The electronic device 602 is illustrated at time T1. Figure 6C The electronic device 602 is illustrated at time T2. It should be noted that times T0, T1, and T2 represent consecutive time instances, wherein T0 <T1<T2。

[0093] As in Fig. 6A As illustrated in FIG. 6 , electronic device 602 is displaying content on display 604. This may be similar to Figure 5B Step 512. Figure 6BAlso illustrated is that electronic device 602 is displaying content on display 604 and indicator 606. Indicator 606 is included in the received content being displayed. In some embodiments, indicator 606 is a QR code. Figure 6C An electronic device 602 is illustrated that displays video content and content 608. In some embodiments, content 608 can be an overlay displayed on the video content. For example, as discussed above, the overlay can be associated with a request for audience feedback, such as a vote or a request to perform certain actions during a video game or other content. The size of the video content can also be adjusted so that content 608 does not overlap with any video content.

[0094] although Figure 5A A method 500 is illustrated, Figure 5B The method 510 is illustrated, and Figure 6A-6C Various diagrams are shown, but Figures 5A-6C Various changes may be made. For example, although method 500 and method 510 are shown as a series of steps, various steps may overlap, occur in parallel, occur in a different order, or occur multiple times. In another embodiment, a step may be omitted or replaced by another step. For example, the steps of method 500 and method 510 may be performed in a different order.

[0095] The flowcharts above illustrate example methods that can be implemented according to the principles of the present disclosure, and various changes can be made to the methods illustrated in the flowcharts herein. For example, although shown as a series of steps, the various steps in each figure can overlap, occur in parallel, occur in different orders, or occur multiple times. In another embodiment, a step can be omitted or replaced by another step.

[0096] In various embodiments, a method for content synchronization is provided. The method includes: receiving, by an electronic device, video content for display on a display associated with the electronic device; identifying, by the electronic device, an indicator included in the video content; in response to identifying the indicator, identifying, by the electronic device, additional content to be included with the video content displayed on the display; determining, by the electronic device, when to include the additional content with the video content based on the video content; and providing, by the electronic device, the additional content for display on the display together with the video content based on the determination.

[0097] In any of the above embodiments, identifying additional content to be included with the video content further comprises: sending, by the electronic device, a notification identifying the indicator; and receiving, by the electronic device, information associated with the additional content to be included with the video content.

[0098] In any of the above embodiments, identifying the additional content to be included with the video content also includes: identifying, by the electronic device, parameters of the indicator included in the video content; and identifying, by the electronic device, the additional content to be included with the video content based on the parameters of the indicator.

[0099] In any of the above embodiments, determining when to include the additional content with the video content also includes: identifying a time in the displayed video content based on when the indicator is displayed in the video content; and determining when to include the additional content for display on the display with the video content based on the identified time in the video content.

[0100] In any of the above embodiments, determining when to include the additional content with the video content also includes: identifying an event occurring in the displayed video content based on when the indicator is displayed in the video content; and determining when to include the additional content for display on the display with the video content based on the identified event occurring in the video content.

[0101] In any of the above embodiments, the method further includes: receiving, by the electronic device, a signal associated with an event; and sending, by the electronic device, a notification associated with detecting the signal associated with the event.

[0102] In any of the above embodiments, the method further includes: identifying, by the electronic device, receipt or modification of a digital item associated with a user of the electronic device as a result of detecting a signal associated with the event.

[0103] Although the drawings illustrate different embodiments of user equipment, various changes may be made to the drawings. For example, the user equipment may include any number of each component in any suitable arrangement. In general, the drawings do not limit the scope of the present disclosure to any (one or more) specific configurations. In addition, although the drawings illustrate operating environments in which various user equipment features disclosed in this patent document may be used, these features may also be used in any other suitable system. The descriptions in this application should not be construed as implying that any specific element, step, or function is an essential element that must be included in the scope of the claims.

[0104] Although the present disclosure has been described using an exemplary embodiment, various changes and modifications may be suggested to one skilled in the art. The present disclosure is intended to encompass such changes and modifications as fall within the scope of the appended claims.

Claims

1. An electronic device (300), comprising: a communication interface (310) configured to receive video content for display on a display (355) associated with the electronic device; as well as A processor (340) operatively coupled to the communication interface, the processor being configured to: identifying an indicator included in the video content (606); In response to identifying the indicator, identifying additional content to be included with the video content displayed on the display (608); determining, based on the video content, when to include the additional content with the video content; as well as Based on the determination, the additional content is provided for display on the display along with the video content.

2. The electronic device according to claim 1, wherein: The communication interface is further configured to: sending a notification identifying the indicator; and receiving information associated with the additional content to be included with the video content, and The processor is further configured to identify the additional content to be included with the video content based on the received information.

3. The electronic device according to claim 1, wherein: To identify the additional content to be included with the video content, the processor is further configured to: identifying parameters of the indicator included in the video content; and The additional content to be included with the video content is identified based on the parameters of the indicator.

4. The electronic device according to claim 1, wherein: To determine when to include the additional content with the video content, the processor is further configured to: identifying a time in the displayed video content based on when the indicator is displayed in the video content; as well as Based on the identified time in the video content, a determination is made as to when to include the additional content for display on the display with the video content.

5. The electronic device according to claim 1, wherein: To determine when to include the additional content with the video content, the processor is further configured to: identifying an event occurring in the displayed video content based on when the indicator is displayed in the video content; as well as Based on the identified events occurring in the video content, it is determined when to include the additional content for display on the display with the video content.

6. The electronic device according to claim 1, wherein: The communication interface is further configured to: receiving a signal associated with the event; and A notification associated with detecting the signal associated with the event is sent.

7. The electronic device according to claim 13, wherein: The processor is further configured to identify receipt or modification of a digital item associated with a user of the electronic device as a result of detecting the signal associated with the event.

8. A non-transitory computer-readable medium (360) comprising program code (362) which, when executed by a processor (340) of an electronic device (300), causes the electronic device to: receiving video content for display on a display (355) associated with the electronic device; identifying an indicator included in the video content (606); In response to identifying the indicator, identifying additional content to be included with the video content displayed on the display (608); determining, based on the video content, when to include the additional content with the video content; as well as Based on the determination, the additional content is provided for display on the display along with the video content.

9. The non-transitory computer readable medium of claim 8, wherein: The program code that, when executed, causes the electronic device to identify the additional content to be included with the video content also includes: program code that, when executed by the processor of the electronic device, causes the electronic device to: sending a notification identifying the indicator; and Information associated with the additional content to be included with the video content is received.

10. The non-transitory computer readable medium of claim 8, wherein: The program code that, when executed, causes the electronic device to identify the additional content to be included with the video content also includes: program code that, when executed by the processor of the electronic device, causes the electronic device to: identifying, by the electronic device, a parameter of the indicator included in the video content; and The additional content to be included with the video content is identified by the electronic device based on the parameter of the indicator.

11. The non-transitory computer readable medium of claim 8, wherein: The program code that when executed causes the electronic device to determine when to include the additional content with the video content also includes: program code that when executed by the processor of the electronic device causes the electronic device to: identifying a time in the video content being displayed based on when the indicator is displayed in the video content; and Based on the identified time in the video content, a determination is made as to when to include the additional content for display on the display with the video content.

12. The non-transitory computer readable medium of claim 8, wherein: The program code that when executed causes the electronic device to determine when to include the additional content with the video content also includes: program code that when executed by the processor of the electronic device causes the electronic device to: identifying an event occurring in the displayed video content based on when the indicator is displayed in the video content; and Based on the identified events occurring in the video content, it is determined when to include the additional content for display on the display with the video content.

13. The non-transitory computer-readable medium of claim 8, further comprising program code that, when executed by the processor of the electronic device, causes the electronic device to: Receive signals associated with events; sending a notification associated with detecting the signal associated with the event; and As a result of detecting the signal associated with the event, receipt or modification of a digital item associated with a user of the electronic device is identified.

14. A method for an electronic device to perform processing according to any one of claims 1-7.

Images