Music service with motion video

Abstract

To combine presentation of video content with presentation of audio content.SOLUTION: Techniques of providing motion video content along with audio content are disclosed. In some example embodiments, a computer-implemented system is configured to perform operations comprising: receiving primary audio content; determining that at least one reference audio content satisfies a predetermined similarity threshold based on comparison of the primary audio content with the at least one reference audio content; for each one of the at least one reference audio content, identifying motion video content based on the motion video content being stored in association with one of the at least one reference audio content and not stored in association with the primary audio content; and causing the identified motion video content to be displayed on a device concurrently with presentation of the primary audio content on the device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 15 / 475,488, filed March 31, 2017, which is incorporated by reference herein in its entirety.

[0002] TECHNICAL FIELD This application relates generally to the field of data processing and, in various embodiments, to a method and system for providing video content along with audio content. [Background technology]

[0003] The presentation of audio content often lacks corresponding video content, and combining such presentation of video content with the presentation of audio content poses many technical challenges, including but not limited to determining which video content to use and how to combine the video content with the audio content. Summary of the Invention

[0004] Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram illustrating a network environment suitable for providing video content along with audio content, according to some example embodiments. [Diagram 2] 4 illustrates a comparison of a dominant audio content with multiple reference audio contents, according to some example embodiments. [Figure 3A] 1 is a conceptual diagram illustrating synchronization of video content with primary audio content, according to some example embodiments. [Figure 3B] 1 is a conceptual diagram illustrating synchronization of video content with primary audio content, according to some example embodiments. [Figure 4A] 1 is a conceptual diagram illustrating synchronization of distinct video content with a primary audio content, according to some example embodiments. [Figure 4B] 1 is a conceptual diagram illustrating synchronization of distinct video content with a primary audio content, according to some example embodiments. [Diagram 5] 1 is a flowchart illustrating a method for providing video content along with audio content, according to some example embodiments. [Figure 6] 1 is a flow chart illustrating a method for displaying video content on a device simultaneously with the presentation of audio content on the device, according to some example embodiments. [Figure 7] 1 is a block diagram illustrating a mobile device, according to some example embodiments. [Figure 8] FIG. 1 is a block diagram of an example computer system on which the methodologies described herein may be implemented, according to some example embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] An example method and system for providing video content together with audio content is disclosed. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the example embodiments. However, it will be apparent to one skilled in the art that the present embodiments may be practiced without these specific details.

[0007] In some example embodiments, a computer-implemented method includes receiving primary audio content, determining that the at least one reference audio content meets a predetermined similarity threshold based on a comparison of the primary audio content with the at least one reference audio content, identifying video content for each one of the at least one reference audio content based on the video content being stored in association with the one of the at least one reference audio content and not stored in association with the primary audio content, and displaying the identified video content on the device concurrently with presentation of the primary audio content on the device. In some example embodiments, the primary audio content includes music.

[0008] In some example embodiments, the comparison includes comparing the fingerprint data of the primary audio content with fingerprint data of at least one reference audio content.

[0009] In some example embodiments, the identification of the video content is further based on a user profile associated with the device.

[0010] In some example embodiments, causing the identified video content to be displayed on the device concurrently with the presentation of the primary audio content on the device includes synchronizing at least one reference audio content data with the primary audio content data, and synchronizing the identified video content with the primary audio content based on the synchronization of the at least one reference audio content data with the primary audio content data. In some example embodiments, the synchronization of the at least one reference audio content data with the primary audio content data is based on a comparison of the at least one reference audio content fingerprint data with a fingerprint of the primary audio content.

[0011] In some example embodiments, the at least one reference audio content includes at least two reference audio contents, each one of the at least two reference audio contents being stored in association with a different video content, and the identified video content includes a portion of each of the different video contents.

[0012] The methods or embodiments disclosed herein may be implemented as a computer system having one or more modules (e.g., hardware modules or software modules). Such modules may be executed by one or more processors of the computer system. The methods or embodiments disclosed herein may be embodied as instructions stored on a machine-readable medium that, when executed by the one or more processors, causes the one or more processors to execute the instructions.

[0013] 1 is a block diagram illustrating a network environment 100 suitable for providing video content along with audio content, according to some example embodiments. Network environment 100 includes a content provider 110, one or more devices 130, and one or more data sources 140 (e.g., data source 140-1 through data source 140-N), all of which are communicatively coupled to one another via a network 120. Content provider 110, device(s) 130, and data source(s) 140 may each be implemented in whole or in part in a computer system, as described below with respect to FIG.

[0014] 1 is a user 132. The user 132 may be a human user (e.g., a human), a machine user (e.g., a computer configured by a software program to interact with the device 130), or any suitable combination thereof (e.g., a human assisted by a machine or a machine supervised by a human). The user 132 may not be part of the network environment 100 but is associated with and a user of the device 130. For example, the device 130 may be a desktop computer, a vehicle computer, a tablet computer, a navigation device, a portable media device, or a smartphone that belongs to the user 132.

[0015] Any of the machines, providers, modules, databases, devices, or data sources depicted in FIG. 1 may be implemented in a computer modified (e.g., configured or programmed) by software to be a special purpose computer to perform one or more of the functions described herein for that machine, provider, module, database, device, or data source. For example, a computer system capable of implementing any one or more of the methodologies described herein is discussed below with respect to FIG. 8. As used herein, a "database" is a data storage resource and may store data structured as a text file, a table, a spreadsheet, a relational database (e.g., an object-relational database), a triple store, a hierarchical data store, or any suitable combination thereof. Moreover, any two or more of the databases, devices, or data sources depicted in FIG. 1 may be combined into a single machine, and the functionality described herein for any single database, device, or data source may be further divided among multiple databases, devices, or data sources.

[0016] Network 120 may be any network that enables communication between or among machines, databases, and devices. Thus, network 120 may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof. Network 120 may include one or more portions that constitute a private network, a public network (e.g., the Internet), or any suitable combination thereof. Thus, network 120 may include one or more portions that incorporate a local area network (LAN), a wide area network (WAN), the Internet, a mobile phone network (e.g., a cellular network), a wired telephone network (e.g., a plain old telephone system (POTS) network), a wireless data network (e.g., a WiFi network or a WiMax network), or any suitable combination thereof. Any one or more portions of network 190 may communicate information via a transmission medium. As used herein, "transmission medium" should be considered to include any intangible medium, including digital or analog communication signals or other intangible media, capable of storing, encoding, or executing instructions for execution by a machine and facilitating the communication of such software.

[0017] Content provider 110 includes a computer system configured to provide audio and video content to devices, such as device 130. In some example embodiments, content provider 110 includes any combination of one or more of criteria determination module 112, video identification module 114, presentation module 116, and one or more databases 118. In some example embodiments, modules 112, 114, and 116, and database(s) 118 reside on a machine having a memory and at least one processor. In some example embodiments, modules 112, 114, and 116, and database(s) 118 reside on the same machine, while in other example embodiments, one or more of modules 112, 114, and 116, and database(s) 118 reside on separate remote machines that communicate with each other over a network, such as network 120.

[0018] In some example embodiments, the criteria determination module 112 is configured to receive a primary audio content. The audio content may include music, such as a recording of a single song. However, other types of audio content are contemplated to be within the scope of this disclosure. In some example embodiments, the criteria determination module 112 is configured to identify or otherwise determine at least one reference audio content that meets a predetermined similarity threshold based on a comparison of the primary audio content with the reference audio content. For example, the criteria determination module 112 may search a plurality of reference audio content stored in the database(s) 118 for reference audio content that meets a predetermined similarity threshold. Additionally or alternatively, the criteria determination module 112 may search one or more external data sources 140 for reference audio content that meets a predetermined similarity threshold. The external data sources 140 may include data sources that are separate and independent from the content provider 110.

[0019] In some example embodiments, the comparison of the primary audio content with the reference audio content includes a comparison of the data of the primary audio content with the data of the reference audio content. The compared data may include fingerprint data that uniquely identifies or characterizes the corresponding audio content. FIG. 2 illustrates a comparison of the primary audio content with a plurality of reference audio contents, according to some example embodiments. In FIG. 2, the fingerprint data 212 of the primary audio content 210 is compared with a plurality of fingerprint data 222 (e.g., fingerprint data 222-1, ..., fingerprint data 222-N) of a plurality of reference audio contents 220 (e.g., reference audio content 220-1, ..., reference audio content 220-N). In some example embodiments, each comparison generates corresponding statistical data indicative of a level of similarity between the primary audio content and the reference audio content. One example of such statistical data is a bit error rate. However, other statistical data are contemplated to be within the scope of the present disclosure. In some example embodiments, the criteria determination module 112 determines whether the statistical data corresponding to the reference audio content 220 meets a predetermined threshold.

[0020] In some example embodiments, the criteria determination module 112 uses an exact fingerprint match between the fingerprint data 212 of the primary audio content 210 and the fingerprint data 222 of the reference audio content 220 as a predetermined threshold. For example, the criteria determination module 112 searches multiple reference audio contents 220 to match one version of an audio recording (e.g., compressed or noisy) with an uncorrupted version of the same audio recording.

[0021] In some example embodiments, the criteria determination module 112 is configured to use a fuzzy fingerprint match between the fingerprint data 212 of the primary audio content 210 and the fingerprint data 222 of the reference audio content 220 as a predetermined threshold. For example, the criteria determination module 112 may search multiple reference audio contents 220 and may match a recording of a song (or a theatrical performance or narration, etc.) with a different performance or recording of the same song (or a theatrical performance or narration, etc.).

[0022] In some example embodiments, the criteria determination module 112 is configured to use as a predetermined threshold a match between audio characteristics, such as chords, rhythmic features, and instrument onsets of such audio characteristics of the primary audio content 210 and the reference audio content 220. For example, the criteria determination module 112 may search through multiple reference audio contents 220 and match one audio recording to another audio recording simply based on a certain level of similarity between the audio characteristics of the different audio recordings, such as matching two different songs based on a determination of a high level of similarity between the rhythmic features of the two different songs.

[0023] In some example embodiments, for one or more of the reference audio content 220 determined to meet the similarity threshold, the video identification module 114 identifies the video content based on the video content being stored in association with the reference audio content and not in association with the primary audio content. In some example embodiments, the video identification module 114 is configured to identify the video content based also on a profile of a user 132 associated with the device 130 to which the combination of the primary audio content and the identified video content is presented. In some example embodiments, the user profile is stored in the database(s) 118. The profile of the user 132 may include any combination of one or more of a history of audio content listened to by the user 132, an indication that the user 132 prefers a particular type or category of audio content, a purchasing history of audio content, a history of video content viewed by the user 132, an indication that the user 132 prefers a particular type or category of video, and demographic information about the user 132 (e.g., gender, age, geographic location). Other types of information indicative of potential preferences for particular types of audio content or particular types of video content may also be included in the profile of user 132. In a scenario in which several different video contents meet a similarity threshold, the video identification module 114 may use the profile of user 132 to select one or more video contents based on a determination of which video content is most relevant to user 132.

[0024] In some example embodiments, presentation module 116 is configured to cause video content identified by video identification module 114 to be displayed on device 130 concurrently with the presentation of the primary audio content on device 132. In some example embodiments in which the primary audio content includes songs, content provider 110 thus dynamically generates music videos for songs for which content provider 110 has stored music videos.

[0025] In some example embodiments, the presentation module 116 is configured to synchronize the reference audio content data with the primary audio content data and then synchronize the identified video content with the primary audio content based on the synchronization of the reference audio content data with the primary audio content data. In some example embodiments, the synchronization of the reference audio content data with the primary audio content data is based on a comparison of the reference audio content fingerprint data with the primary audio content fingerprint data.

[0026] 3A-3B are conceptual diagrams illustrating synchronization of video content with primary audio content, according to some example embodiments. In FIG. 3A, primary audio content 210 is shown as being comprised of audio segments 310-1, 310-2, 310-3, and 310-4, and reference audio content 220 is shown as being comprised of audio segments 320-1, 320-2, 320-3, and 320-4. Reference audio content 220 is also shown as being stored in association with video content 320, which is shown as being comprised of video segments 322-1, 322-2, 322-3, and 322-4. Other segmentation configurations are contemplated to be within the scope of this disclosure. In FIG. 3A, audio segments 310 of primary audio content 210 and audio segments 320 of reference audio content 220 are adjusted along the time domain according to their respective timestamps as a result of presentation module 116 synchronizing them. Similarly, a video segment 322 of the video content 320 is coordinated with the audio segment 320 of the reference audio content 220 with which it is associated.

[0027] In FIG. 3B, using synchronization of the audio segments 320 of the reference audio content 220 with the audio segments 310 of the main audio content 210, along with synchronization of the video segments 322 of the video content 320 with the audio segments 320 of the reference audio content 220, the presentation module 116 synchronizes the video segments 322 of the video content 320 with the audio segments 310 of the main audio content 210.

[0028] In some example embodiments, portions from different video contents associated with different reference audio contents are combined with the primary audio content. Figures 4A-4B are conceptual diagrams illustrating the synchronization of different video contents with the primary audio content, according to some example embodiments. In Figure 4A, similar to Figure 3A, the primary audio content 210 is shown as being composed of audio segments 310-1, 310-2, 310-3, and 310-4, and the reference audio content 220 is shown as being composed of audio segments 320-1, 320-2, 320-3, and 320-4. The reference audio content 220 is also shown as being stored in association with the video content 320, which is shown as being composed of video segments 322-1, 322-2, 322-3, and 322-4. The audio segments 310 of the primary audio content 210 and the audio segments 320 of the reference audio content 220 are adjusted along the time domain according to their respective timestamps as a result of the presentation module 116 synchronizing them. Similarly, a video segment 322 of the video content 320 is coordinated with the audio segment 320 of the reference audio content 220 with which it is associated.

[0029] 4A, another reference audio content 420 is shown as being comprised of audio segments 420-1, 420-2, 420-3, and 420-4. The reference audio content 420 is also shown as being stored in association with video content 420, which is shown as being comprised of video segments 422-1, 422-2, 422-3, and 422-4. The audio segments 420-1, 420-2, 420-3, and 420-4 of the reference audio content 420 and the video segments 422-1, 422-2, 422-3, and 422-4 are coordinated with the audio segments 310-1, 310-2, 310-3, and 310-4 of the primary audio content 210.

[0030] Using synchronization, the presentation module 116 generates video content 425 from portions of video content 320 and portions of video content 420. As a result, video segment 322-1 is synchronized with audio segment 310-1, video segment 322-2 is synchronized with audio segment 310-2, video segment 422-3 is synchronized with audio segment 310-3, and video segment 422-4 is synchronized with audio segment 310-4.

[0031] In some example embodiments, the presentation module 116 is configured to synchronize the audio segments 310 of the primary audio content 210 with the audio segments 320 of the reference audio content 220 based on an exact fingerprint match between the audio segments 310 of the primary audio content 210 and the audio segments 320 of the reference audio content 220. For example, the presentation module 116 may synchronize the audio segments 310 of the primary audio content 210 with the audio segments 320 of the reference audio content 220 based on a match between one version of an audio recording (e.g., compressed or noisy) and a non-degraded version of the same audio recording.

[0032] In some example embodiments, the presentation module 116 is configured to synchronize the audio segments 310 of the primary audio content 210 with the audio segments 320 of the reference audio content 220 based on a fuzzy fingerprint match between the audio segments 310 of the primary audio content 210 and the audio segments 320 of the reference audio content 220. For example, the presentation module 116 may synchronize the audio segments 310 of the primary audio content 210 with the audio segments 320 of the reference audio content 220 based on a match between a recording of a song (or a theatrical performance or narration, etc.) and a different performance or recording of the same song (or a theatrical performance or narration, etc.).

[0033] In some example embodiments, the presentation module 116 is configured to synchronize the audio segments 310 of the primary audio content 210 with the audio segments 320 of the reference audio content 220 using matches between audio characteristics of the audio segments 310 of the primary audio content 210 and such audio characteristics of the audio segments 320 of the reference audio content 220, such as chords, rhythmic features, and instrument onsets. For example, the presentation module 116 may synchronize the audio segments 310 of the primary audio content 210 with the audio segments 320 of the reference audio content 220 based on a particular level of similarity between audio characteristics of the different audio recordings, such as synchronizing two different songs based on a determination of a high level of similarity between rhythmic features of the two different songs.

[0034] In some example embodiments, the video identification module 114 and the presentation module 116 are configured to identify different video content to synchronize and display simultaneously with the same primary audio content, thereby changing the video experience for the same primary audio content from one rendition to the next. The change in the video experience from one rendition of the primary audio content to the next may be partial, such as by maintaining at least one video segment or one scene from one rendition to the next, while swapping out one video segment or one scene for another video segment or another scene. Alternatively, the change in the video experience from one rendition of the primary audio content to the next may be total, such as by replacing all of the video segments used for the presentation of the primary audio content with completely different video segments for the subsequent presentation of the primary audio content. For example, on one date, video content covering an entire live performance of a song may be synchronized and displayed simultaneously with the primary audio content, and then on a later date, video content covering an entire live performance of the same song in a studio (e.g., different from the live performance) may be synchronized and displayed simultaneously with the same primary audio content in place of the video content covering the entire live performance of the song. Such changes in video experience may be based on detected changes in the popularity of the video content (e.g., changes in the total number of YouTube® views of video content on a daily basis), or may be based on detected changes in the preferences or behavior of users who are to be presented with video content along with primary audio content (e.g., changes in users' browsing habits of video content on YouTube®), or may be random. It is contemplated that other factors may be used to cause the video content to change from one presentation to another.

[0035] 5 is a flow chart depicting a method 500 of providing video content along with audio content, according to some example embodiments. Method 500 can be performed by processing logic that can include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions running on a processing device), or a combination thereof. In one example embodiment, method 500 is performed by content provider 110 of FIG. 1, or any combination of one or more of its components or modules.

[0036] At operation 510, the content provider 110 receives the primary audio content. In some example embodiments, the primary audio content includes music (e.g., a song). At operation 520, the content provider 110 determines that the at least one reference audio content meets a predetermined similarity threshold based on a comparison of the primary audio content with the at least one reference audio content. In some example embodiments, the comparison includes a comparison of fingerprint data of the primary audio content with fingerprint data of the at least one reference audio content. At operation 530, for each one of the at least one reference audio content, the content provider 110 identifies video content based on the video content being stored in association with one of the at least one reference audio content and not stored in association with the primary audio content. In some example embodiments, the identification of the video content is further based on a profile of a user associated with the device. At operation 540, the content provider 110 causes the identified video content to be displayed on the device concurrently with the presentation of the primary audio content on the device. It is contemplated that any of the other features described within this disclosure can be incorporated into the method 500.

[0037] 6 is a flow chart illustrating a method 600 for causing video content to be displayed on a device simultaneously with the presentation of audio content on the device, according to some example embodiments. Method 600 can be performed by processing logic that can include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions running on a processing device), or a combination thereof. In one example embodiment, method 600 is performed by content provider 110 of FIG. 1, or any combination of one or more of its components or modules.

[0038] At operation 610, the content provider synchronizes the at least one reference audio content data with the primary audio content data. At operation 620, the content provider 110 synchronizes the identified video content with the primary audio content based on the synchronization of the at least one reference audio content data with the primary audio content data. In some example embodiments, the synchronization of the at least one reference audio content data with the primary audio content data is based on a comparison of the at least one reference audio content fingerprint data with the primary audio content fingerprint data. It is contemplated that any of the other features described within this disclosure can be incorporated into method 600.

[0039] Mobile devices as an example FIG. 7 is a block diagram depicting a mobile device 700 according to an example embodiment. The mobile device 700 may include a processor 702. The processor 702 may be any of a variety of different types of commercially available processors suitable for the mobile device 700 (e.g., an XScale architecture microprocessor, a microprocessor without an Interlocked Pipeline Stages (MIPS) architecture processor, or another type of processor). A memory 704, such as a random access memory (RAM), flash memory, or other type of memory, is typically accessible to the processor 702. The memory 704 may be adapted to store an operating system (OS) 706 as well as application programs 708, such as a mobile location-enabled application that may provide a LBS to a user. The processor 702 may be coupled, either directly or through suitable intermediary hardware, to a display 710 and one or more input / output (I / O) devices 712, such as a keypad, a touch panel sensor, and a microphone. Similarly, in some embodiments, the processor 702 may be coupled to a transceiver 714 that interfaces with an antenna 716. The transceiver 714 can be configured to both transmit and receive cellular network signals, wireless data signals, or other types of signals via an antenna 716, depending on the nature of the mobile device 700. Additionally, in some configurations, a GPS receiver 718 can also utilize the antenna 716 to receive GPS signals.

[0040] Modules, Components, and Logic Certain embodiments are described herein as logic, or as some components, modules, or mechanisms. A module can constitute either a software module (e.g., code embodied on a machine-readable medium or in a transmission signal) or a hardware module. A hardware module is a tangible unit capable of performing a particular operation and can be configured or arranged in a particular manner. In an example embodiment, one or more computer systems (e.g., standalone, client, or server computer systems), or one or more hardware modules of a computer system (e.g., a processor or a group of processors), can be configured by software (e.g., an application or application portion) as a hardware module that operates to perform a particular operation as described herein.

[0041] In various embodiments, a hardware module can be implemented mechanically or electrically. For example, a hardware module can include dedicated circuitry or logic (e.g., a special purpose processor such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) that is permanently configured to perform certain operations. A hardware module can also include programmable logic or circuitry (e.g., as incorporated within a general purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It is recognized that the decision to mechanically implement a hardware module in dedicated and permanently configured circuitry or temporarily configured circuitry (e.g., configured by software) can be driven by cost and time considerations.

[0042] Thus, the term "hardware module" should be understood to encompass tangible entities, which are physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner and / or perform particular operations described herein. Considering embodiments in which the hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, if the hardware modules include a general-purpose processor configured using software, the general-purpose processor can be configured as each different hardware module at different times. Thus, the software can, for example, configure the processor to configure a particular hardware module at one instance in time and configure a different hardware module at a different instance in time.

[0043] Hardware modules can provide information to and receive information from other hardware modules. Thus, the described hardware modules can be considered to be communicatively coupled. When multiple such hardware modules are present simultaneously, communication can be achieved through signal transmission (e.g., through appropriate circuits and buses) connecting the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, through storage and retrieval of information in a memory structure to which the multiple hardware modules have access. For example, one hardware module can perform an operation in a memory device to which it is communicatively coupled and store the output of that operation. An additional hardware module can then access the memory device at a later time to retrieve and process the stored output. Hardware modules can also initiate communication with input or output devices and can operate on resources (e.g., collections of information).

[0044] Various operations of example methods described herein may be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the associated operations. Such processors, whether temporarily or permanently configured, may constitute processor-implemented modules that operate to perform one or more operations or functions. Modules referred to herein may, in some example embodiments, include processor-implemented modules.

[0045] Similarly, the methods described herein can be at least partially implemented by a processor. For example, at least some of the operations of the methods can be performed by one or more processors or processor-implemented modules. The execution of certain operations can be distributed among one or more processors deployed across several machines as well as within a single machine. In some example embodiments, the processor or processors can be located at a single location (e.g., in a home environment, in an office environment, or as a server farm), while in other embodiments, the processors can be distributed across several locations.

[0046] The one or more processors may also operate to support performance of related operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations may be performed by a group of computers (as an example of a machine that includes a processor), and those operations may be accessible over a network and via one or more suitable interfaces (e.g., APIs).

[0047] Exemplary embodiments can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Exemplary embodiments can be implemented using a computer program product, e.g., a computer program tangibly embodied in an information carrier wave, e.g., a machine-readable medium for execution by or to control the operation of a data processing apparatus, e.g., a programmable processor, a computer, or a plurality of computers.

[0048] A computer program can be written in any type of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site, or can be distributed across multiple sites, interconnected by a communication network.

[0049] In example embodiments, the operations may be performed by one or more programmable processors executing computer programs to perform functions by operating on input data and generating output. The operations of the methods may also be performed by, and apparatus of example embodiments may be implemented as, special purpose logic circuitry (e.g., an FPGA or an ASIC).

[0050] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communications network. The relationship of client and server arises by reason of computer programs running on the respective computers and having a client-server relationship to each other. In embodiments deploying a programmable computing system, it is recognized that it is worth considering both hardware and software architectures. In particular, it is recognized that the choice of whether to implement a particular functionality in permanently configured hardware (e.g., ASICs), in temporarily configured hardware (e.g., a combination of software and programmable processors), or in a combination of permanently and temporarily configured hardware can be a design choice. The following illustrates hardware (e.g., machine) and software architectures that can be deployed in various example embodiments.

[0051] FIG. 8 is a block diagram of a machine in an example form, computer system 800, on which instructions 824 may be executed to cause the machine to perform any one or more of the methodologies discussed herein, according to an example embodiment. In alternative embodiments, the machine may operate as a standalone device or may be connected to other machines (e.g., networked). In a networked deployment, the machine may operate in the capacity of a server or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, network router, switch or bridge, or any machine capable of executing (in sequence or otherwise) instructions that specify actions to be taken by the machine. Additionally, although a single machine is depicted, the term "machine" should also be considered to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0052] The example computer system 800 includes a processor 802 (e.g., a central processing unit (CPU), a graphic processing unit (GPU), or both), a main memory 804, and a static memory 806, which communicate with each other via a bus 808. The computer system 800 may further include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 800 also includes an alphanumeric input device 812 (e.g., a keyboard), a user interface (UI) navigation (or cursor control) device 814 (e.g., a mouse), a disk drive unit 816, a signal generating device 818 (e.g., a speaker), and a network interface device 820.

[0053] Disk drive unit 816 includes a machine-readable medium 822 on which is stored one or more sets of data structures and instructions 824 (e.g., software) that embody or are utilized by any one or more of the methodologies or functions described herein. The instructions 824 may also reside, completely or at least partially, within main memory 804 and / or within processor 802 during execution thereof by computer system 800, main memory 804, and processor 802, which also constitute machine-readable media. The instructions 824 may also reside, completely or at least partially, within static memory 806.

[0054] While the machine-readable medium 822 is shown in the example embodiment as being a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more instructions 824 or data structures. The term "machine-readable medium" should also be considered to include any tangible medium capable of storing, encoding, or carrying instructions for execution by a machine, causing a machine to perform any one or more of the methodologies of the present embodiments, or storing, encoding, or carrying data structures utilized by or associated with such instructions. Accordingly, the term "machine-readable medium" should be considered to include, but is not limited to, solid-state memory, and optical and magnetic media. Particular examples of machine-readable media include, by way of example, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), and flash memory devices), magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and non-volatile memory including compact disk read-only memory (CD-ROM) and digital versatile disk (or digital video disk) read-only memory (DVD-ROM) disks.

[0055] The instructions 824 may further be transmitted or received over a communications network 826 using a transmission medium. The instructions 824 may be transmitted using a network interface device 820 and any one of several well-known transfer protocols (e.g., HTTP). Examples of communications networks include LANs, WANs, the Internet, cellular networks, POTS networks, and wireless data networks (e.g., WiFi and WiMAX networks). The term "transmission medium" should be considered to include any intangible medium capable of storing, encoding, or carrying instructions for execution by a machine, including digital or analog communications signals, or other intangible media that facilitate communication of such software.

[0056] Although the embodiments have been described with reference to certain example embodiments, it is apparent that various modifications and changes can be made thereto without departing from the broader spirit and scope of the present disclosure. Accordingly, the specification and drawings are to be regarded in an illustrative and not a restrictive sense. The accompanying drawings, which form a part hereof, show, by way of example, and not by way of limitation, specific embodiments in which the subject matter may be practiced. The represented embodiments are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments can be utilized and derived therefrom, such that structural and logical substitutions and changes can be made without departing from the scope of the present disclosure. Accordingly, this detailed description is not to be considered in a limiting sense, and the scope of the various embodiments is defined solely by the appended claims, with the full scope as such claims are entitled to.

[0057] Although specific embodiments have been shown and described herein, it should be recognized that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiment shown. The present disclosure is intended to cover any and all adaptations or variations of the various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reference to the above description.

Claims

1. Receiving primary audio content; and determining, by at least one hardware processor, that the at least one reference sound content meets a predetermined similarity threshold based on a comparison of the dominant sound content with at least one reference sound content; identifying, for each one of the at least one reference audio content, video content based on the video content being stored in association with the one of the at least one reference audio content and not being stored in association with the primary audio content; displaying the identified video content on the device concurrently with the presentation of the primary audio content on the device; A computer-implemented method comprising:

2. The computer-implemented method of claim 1 , wherein the primary audio content includes music.

3. The computer-implemented method of claim 1 , wherein the comparing comprises comparing fingerprint data of the primary audio content with fingerprint data of the at least one reference audio content.

4. The computer-implemented method of claim 1 , wherein the identification of video content is further based on a user profile associated with the device.

5. Causing the identified video content to be displayed on the device simultaneously with the presentation of the primary audio content on the device includes: synchronizing data of the at least one reference audio content with data of the primary audio content; and synchronizing the identified video content with the primary audio content based on synchronization of the data of the at least one reference audio content with the data of the primary audio content.

10. The computer implemented method of claim 1.

6. 6. The computer-implemented method of claim 5, wherein the synchronization of the data of the at least one reference audio content with the data of the primary audio content is based on a comparison of fingerprint data of the at least one reference audio content with fingerprint data of the primary audio content.

7. 2. The computer-implemented method of claim 1, wherein the at least one reference audio content includes at least two reference audio contents, each one of the at least two reference audio contents being stored in association with a different video content, and the identified video content includes a portion of each of the different video content.

8. At least one processor; and a non-transitory computer-readable medium having executable instructions stored thereon, the executable instructions, when executed, causing the at least one processor to: Receiving primary audio content; and determining, based on a comparison of the dominant sound content with at least one reference sound content, that the at least one reference sound content meets a predetermined similarity threshold; identifying, for each one of the at least one reference audio content, video content based on the video content being stored in association with the one of the at least one reference audio content and not being stored in association with the primary audio content; displaying the identified video content on the device concurrently with the presentation of the primary audio content on the device; A system that causes a system to perform an operation including:

9. The system of claim 8 , wherein the primary audio content comprises music.

10. The system of claim 8 , wherein the comparison comprises comparing fingerprint data of the primary audio content with fingerprint data of the at least one reference audio content.

11. The system of claim 8 , wherein the identification of the video content is further based on a profile of a user associated with the device.

12. Causing the identified video content to be displayed on the device simultaneously with the presentation of the primary audio content on the device includes: synchronizing data of the at least one reference audio content with data of the primary audio content; and synchronizing the identified video content with the primary audio content based on synchronization of the data of the at least one reference audio content with the data of the primary audio content. The system of claim 8.

13. The system of claim 12, wherein the synchronization of the data of the at least one reference audio content with the data of the primary audio content is based on a comparison of fingerprint data of the at least one reference audio content with fingerprint data of the primary audio content.

14. 9. The system of claim 8, wherein the at least one reference audio content includes at least two reference audio contents, each one of the at least two reference audio contents being stored in association with a different video content, and the identified video content includes a portion of each of the different video content.

15. A non-transitory machine-readable storage medium tangibly embodied with a set of instructions that, when executed by at least one processor, cause the at least one processor to: Receiving primary audio content; and determining, by at least one hardware processor, that the at least one reference sound content meets a predetermined similarity threshold based on a comparison of the dominant sound content with at least one reference sound content; identifying, for each one of the at least one reference audio content, video content based on the video content being stored in association with the one of the at least one reference audio content and not being stored in association with the primary audio content; displaying the identified video content on the device concurrently with the presentation of the primary audio content on the device; A non-transitory machine-readable storage medium for causing a user to perform operations including:

16. 16. The non-transitory machine-readable storage medium of claim 15, wherein the comparison comprises a comparison of fingerprint data of the primary audio content with fingerprint data of the at least one reference audio content.

17. 20. The non-transitory machine-readable storage medium of claim 15, wherein the identification of the video content is further based on a profile of a user associated with the device.

18. Causing the identified video content to be displayed on the device simultaneously with the presentation of the primary audio content on the device includes: synchronizing data of the at least one reference audio content with data of the primary audio content; and synchronizing the identified video content with the primary audio content based on synchronization of the data of the at least one reference audio content with the data of the primary audio content.

16. The non-transitory machine-readable storage medium of claim 15.

19. 20. The non-transitory machine-readable storage medium of claim 18, wherein the synchronization of the data of the at least one reference audio content with the data of the primary audio content is based on a comparison of fingerprint data of the at least one reference audio content with fingerprint data of the primary audio content.

20. 16. The non-transitory machine-readable storage medium of claim 15, wherein the at least one reference audio content includes at least two reference audio contents, each one of the at least two reference audio contents being stored in association with a different video content, and the identified video content includes a portion of each of the different video content.

Images