Systems and methods for presenting supplemental content in augmented reality

Abstract

To provide a suitable system and method for presenting supplemental content in augmented reality.SOLUTION: Systems and methods provided for presenting supplemental content in an augmented reality environment where an object within a field of view of an augmented reality device of a user is identified and processed to detect a reference related to a participant in an event. A user profile or user social network is searched to identify a message from the user about the participant. The message may be combined with the object in the augmented reality field of view. Capturing an image includes using camera circuitry and storing pixels representing the object.SELECTED DRAWING: Figure 4

Description

[Background technology]

[0001] A user can use an augmented reality device to provide an enhanced or modified version of the user's environment by adding features or information within the user's field of view. Existing augmented reality applications include, for example, games that overlay the user's environment, or information services that may obtain information for objects within the field of view. It would be useful if such information could be tailored to be relevant to the user. Summary of the Invention [Problem to be solved by the invention]

[0002] Thus, a system and method are described for presenting complementary content in augmented reality. Augmented reality is a modified version of reality that may include enhanced information overlaid over a real-world view. In general, an augmented reality environment can be accessed using a dedicated device such as smart glasses or a headset. An augmented reality environment can also be accessed using an application on a multi-purpose portable device such as a smartphone or tablet. An augmented reality environment may use a camera to take a picture of the real-world view and a computer or processor to analyze the picture and detect objects in the picture that may be identified as having some relevance to the viewer or user and provide information about, or interactive options regarding, one or more objects. [Means for solving the problem]

[0003] In examples, an augmented reality user may use an augmented reality device, such as smart glasses, a headset, a mobile phone, or a smartphone, to view a scene. The augmented reality device (e.g., smart glasses or a smartphone) may capture a photo or video image of the scene and search for an object in the image. In some scenarios, the user may specifically focus on a particular object in the scene, for example, by pointing the augmented reality device in the direction of the object. The media guidance application may analyze one or more objects in the image and determine whether any of the objects have relevance to the user. For any object determined to have relevance to the user, the media guidance application may identify complementary content related to the object for the user. In some scenarios, to identify an association between the user and the object, the media guidance application may search a user profile, a user history, the user's social media, or another source of information about the user. The complementary content identified by the media guidance application may be media content related to the object, such as messages, video clips, television programs, movies, images, books, or other media content related to the object that may be obtained, for example, from a social network or social media.

[0004] In an example, a method for presenting complementary content in augmented reality may be provided by identifying objects within a field of view of a user's augmented reality device. The field of view may be any environment the user desires to use. For example, the field of view may be a location where the user is present. For example, the field of view may be a room where the user is present, or a street scene where the user is walking, or a location where the user is looking for but may not be present. The field of view may also be an image of a sign, a photograph, a magazine advertisement, etc. The augmented reality device may capture an image of such a field of view. The image may be a video or a still image that captures the field of view. The media guidance application may process the image and detect objects within the image. The media guidance application may detect references on the object that may relate to participants of the event. For example, the media guidance application may detect that there is a sports team logo in the image, such as a sign, a vehicle, clothing, or another location in the image that may relate to members of the sports team, and that the sports team may have a scheduled match or another upcoming event.

[0005] The media guidance application may search the user's profile and find an association between the user and the participant. For example, the media guidance application may search social networks for messages by the user about the participant. In an example, a user may be posting on Twitter about a participant on a sports team, or about the team, or about the sport. If the media guidance application can identify social media messages by the user about the participant or team, the media guidance application may overlay the messages about the participant using an augmented reality device. For example, the media guidance application may overlay the user's social media posts about a sports player over objects related to the player in an image as seen in an augmented reality environment.

[0006] The message overlay in the augmented reality environment may be the content of a social media message by the user regarding the participant, such that the message may be combined in the augmented reality image with a reference detected on the object. In some scenarios, the message may be identical to that used in the social media post. In other scenarios, the message may be shortened, omitted, or otherwise based on the social media post message. In some examples, the location of the overlaid message may be on or near the location of the object within the field of view of the augmented reality device.

[0007] In another example, a method is provided for presenting complementary content in augmented reality in which a user's augmented reality device can be positioned over an object in the field of view of the augmented reality device. For example, a user may look at a particular field of view with smart glasses and position the field of view over the object. In another example, a user may place a portable device with augmented reality functionality over the object in the field of view. For example, a user may point a smartphone camera generally in the direction of the object. A user may also use the smartphone camera to zoom in on the object.

[0008] The media guidance application may identify an object within the field of view of the user's augmented reality device. For example, the media guidance application may detect an object that is the focus of a smartphone camera. In some examples, the object may be in the center of the screen. In another example, the object may be an object stored in a user profile as being associated with the user in some way.

[0009] The media guidance application may capture an image of the object. The image may be captured within an augmented reality device, for example, by taking a photograph or screenshot. The image may also be captured using camera circuitry and store pixels representing the object. The media guidance application may then search a database to identify content that matches at least a portion of the image of the object. For example, the media guidance application may use an image search tool to search one or more image databases to find matches for portions of the object image.

[0010] Searching the database to identify suitable content may also include searching a database that includes entries that associate images with content. Such a database may be predefined to include entries that are relevant to the user and that include interesting complementary content that may arise from time to time. Configuring such a database can improve search results and reduce the time to identify suitable complementary content for the user. Such a database may be searched in addition to searching a general media database, which may have many more content items but is not defined with associated images. The database may store multiple entries that associate images with content. The image may be analyzed by control circuitry of the media guidance application to detect multiple distinct items within the captured image of the object. The distinct items may be, for example, logos, faces, brand names, or other objects and items. The distinct items may be matched with the associated content and entries associating the distinct items and content may be stored in the database. The entries may be searched for the item or object and a matching entry that substantially corresponds to one of the distinct items may be identified. When the media guidance application makes such a match, content within the entry associated with a distinct item that is part of the captured image of the object and therefore also related to the object image may be retrieved by the media guidance application and provided to the augmented reality environment for the user.

[0011] If content matching the object image or portion of an image is found in the media database, the media guidance application may retrieve metadata for the matching content. The metadata may be analyzed and used to detect information or references related to participants of the event. For example, the media guidance application may find a matching sports team logo image and the associated metadata may include one or more sports team player names. The media guidance application may then search a list of messages previously transmitted by the user to a social network and identify messages that contain content matching the participants. For example, the media guidance application may search the user's social media posts for messages related to players on a sports team with a matching logo.

[0012] The media guidance application may analyze the field of view within the augmented reality device to find the location or position of objects and references. In particular, the media guidance application may identify the location on the object of a reference associated with the participant. In a sports team logo example, the media guidance application may identify an area of ​​the object in the field of view that has the logo. The media guidance application may determine an area of ​​the object in the field of view that includes a portion of an image that matches the matching content. In another example, the media guidance application may identify a player's name on a jersey or sign and determine an area of ​​the object in the field of view that includes the name.

[0013] The identified location may then be used by the media guidance application to position a message within the augmented reality device. For example, the media guidance application may cause the augmented reality device to overlay a message in the field of view for the user at the location of the object. The content of the message may be combined with the reference detected on the object. In an example, a user may have previously posted to a social network about a player on a sports team, and such a social media message may be superimposed in the augmented reality field of view over the reference object.

[0014] In an illustrative example, the image in the field of view may be sports related. An example of a content, object, or item in the field of view may be a sports logo. The search for related complementary content may include detecting the sports logo and determining the name of the sports team associated with the sports logo, for example, by searching entries in a database for the associated team name and logo or by using metadata for the logo.

[0015] In another example, the event may be a sporting event. If the media guidance application determines that the event is a sporting event, the media guidance application may identify competitors participating in the sporting event. The competitors may be identified by searching a media database for information or metadata about the event. The media guidance application may also determine the teams featured in the sporting event in the same manner. The competitor information may be used to determine a second team participating in the sporting event. For example, the competitor information may include metadata for each player that includes team information. If a second team is identified, the media guidance application may select a message to include in the augmented reality view related to the first team, any of the competitors, or the second team.

[0016] In another example, an event may have a time associated with it. For example, a sporting event may be scheduled at a particular time or following another event in a particular order. The time for the event may be identified by the media guidance application. Messages selected by the media guidance application to include in the augmented reality view may be those transmitted to the user's social network within a threshold of time that are relevant or suitable for the event time. For example, a message about a sports team that is a year old may not be relevant to a game scheduled in the near future. Instead, a more recent message about the sports team may be selected by the media guidance application.

[0017] After a message is combined with a reference in an object in the field of view for the augmented reality device, the user may wish to capture an image of the view. The media guidance application may store a photograph representing the view through the augmented reality device of a given message combined with the reference detected on the object for the user. The photograph may be presented to the user in a display generated by the media guidance application along with an option for the user to transmit the photograph to a social network.

[0018] In another illustrative example, a user may use an augmented reality device to view another individual wearing clothing having an association with a sports team. For example, the second individual's clothing may include a graphic logo or another representation of the sports team. If such an image is within the augmented reality field of view and thus captured by the media guidance application, the resulting composite message and object view generated by the media guidance application for the user may also be transmitted to the second individual. For example, the media guidance application may cause a message combined with the image to be sent to the second user's computer or smartphone, or augmented reality device.

[0019] It should be noted that the systems, methods, apparatus, and / or aspects described above may be applied to or used in accordance with other systems, methods, apparatus, and / or aspects described in this disclosure. The present specification also provides, for example, the following items: (Item 1) 1. A method for presenting complementary content in augmented reality, the method comprising: Identifying an object within a field of view of a user's augmented reality device; processing an image of the object and detecting references on the object that relate to participants of the event; searching the user's profile to identify messages about the participant previously transmitted by the user to a social network; using the augmented reality device to overlay content of the message transmitted by the user to the social network at the location of the object and combine the message with the reference detected on the object; A method comprising: (Item 2) 1. A method for presenting complementary content in augmented reality, the method comprising: Positioning an augmented reality device of a first user over an object within a field of view of the augmented reality device; identifying the object within a field of view of the first user's augmented reality device; capturing an image of the object; searching a database to identify content that matches at least a portion of the image of the object; in response to identifying content matching at least a portion of the image of the object, reading metadata associated with the content and detecting references on the object related to participants of the event; searching a list of messages previously transmitted by the first user to a social network to identify a given message that includes content matching the participant; identifying a location on the object of the reference associated with the participant by determining an area of ​​the object within the field of view that includes a portion of the image that matches the identified content; combining the given message with the reference detected on the object at a position relative to the identified location using the augmented reality device; A method comprising: (Item 3) 3. The method of claim 2, wherein capturing the image includes using a camera circuit and storing pixels representing the object. (Item 4) Searching the database includes: storing a plurality of entries in the database associating images with content; detecting a plurality of distinct items within a captured image of the object; identifying a given one of the plurality of entries that includes a given image that substantially corresponds to a given one of the distinct items; Retrieving the content associated with the given image; The method according to item 2, comprising: (Item 5) 5. The method of claim 4, wherein the content comprises a logo of a first team, and further comprising determining a name of the first team based on a database search of the logo and based on the metadata. (Item 6) the event is a sporting event; identifying a plurality of competitors in the sporting event; determining that the first team is featured in the sporting event; searching the plurality of competitors to identify a second team featured in the sporting event; selecting one of the messages including the second team as the given message; The method of claim 5, further comprising: (Item 7) identifying a time associated with the event; selecting a subset of the list of messages that were transmitted to the social network within the time threshold; The method according to item 2, further comprising: (Item 8) storing a photograph representing a view through an augmented reality device of the given message combined with the reference detected on the object; generating a display for the user with an option to transmit the photo to the social network; The method according to item 2, further comprising: (Item 9) 3. The method of claim 2, wherein the object includes an article of clothing worn by a second user, the portion of the image includes a graphic on the article of clothing worn by the second user, and the content includes a team represented by the graphic. (Item 10) 3. The method of claim 2, further comprising transmitting a message including the composite content of the given message and the reference found on the object to a second user associated with the object. (Item 11) 3. The method of claim 2, wherein the augmented reality device comprises a mobile phone or a headset worn by a user. (Item 12) 1. A system for presenting complementary content in augmented reality, the system comprising: an input / output (I / O) interface circuit configured to receive user input; A control circuit comprising: Positioning an augmented reality device of a first user over an object within a field of view of the augmented reality device; identifying the object within a field of view of the first user's augmented reality device; capturing an image of the object; searching a database to identify content that matches at least a portion of the image of the object; in response to identifying content matching at least a portion of the image of the object, reading metadata associated with the content and detecting references on the object related to participants of the event; searching a list of messages previously transmitted by the first user to a social network to identify a given message that includes content matching the participant; identifying a location on the object of the reference associated with the participant by determining an area of ​​the object within the field of view that includes a portion of the image that matches the identified content; combining the given message with the reference detected on the object at a position relative to the identified location using the augmented reality device; a control circuit configured to A system comprising: (Item 13) Item 13. The system of item 12, wherein the control circuitry configured to capture the image uses a camera circuit and stores pixels representing the object. (Item 14) The control circuitry configured to search the database further comprises: storing a plurality of entries in the database associating images with content; detecting a plurality of distinct items within a captured image of the object; identifying a given one of the plurality of entries that includes a given image that substantially corresponds to a given one of the distinct items; Retrieving the content associated with the given image; Item 13. The system of item 12, configured to perform the following: (Item 15) Item 15. The system of item 14, wherein the content comprises a logo of a first team, and the control circuitry is further configured to determine a name of the first team based on a database search of the logo and based on the metadata. (Item 16) the event is a sporting event, and the control circuitry further comprises: identifying a plurality of competitors in the sporting event; determining that the first team is featured in the sporting event; searching the plurality of competitors to identify a second team featured in the sporting event; selecting one of the messages including the second team as the given message; Item 16. The system of item 15, configured to perform the steps of: (Item 17) The control circuit further comprises: identifying a time associated with the event; selecting a subset of the list of messages that were transmitted to the social network within the time threshold; Item 13. The system of item 12, configured to perform the following: (Item 18) The control circuit further comprises: storing a photograph representing a view through an augmented reality device of the given message combined with the reference detected on the object; generating a display for the user with an option to transmit the photo to the social network; Item 13. The system of item 12, configured to perform the following: (Item 19) Item 13. The system of item 12, wherein the object includes an item of clothing worn by the second user, the portion of the image includes a graphic on the item of clothing worn by the second user, and the content includes a team represented by the graphic. (Item 20) Item 13. The system of item 12, wherein the control circuitry is further configured to transmit a message including the composite content of the given message and the reference found on the object to a second user associated with the object. (Item 21) Item 13. The system of item 12, wherein the augmented reality device comprises a mobile phone or a headset worn by a user. (Item 22) 1. A system for presenting complementary content in augmented reality, the system comprising: means for positioning the first user's augmented reality device over an object within a field of view of the augmented reality device; means for identifying the object within a field of view of the first user's augmented reality device; means for capturing an image of the object; means for searching a database to identify content that matches at least a portion of the image of the object; means for, in response to identifying content matching at least a portion of an image of the object, reading metadata associated with the content and detecting references on the object related to participants of the event; means for searching a list of messages previously transmitted by the first user to a social network and identifying a given message that includes content that matches the participant; means for identifying a location on the object of the reference associated with the participant by determining an area of ​​the object within the field of view that includes a portion of the image that matches the identified content; means for combining the given message with the reference detected on the object at a position relative to the identified location using the augmented reality device; A system comprising: (Item 23) 23. The system of claim 22, wherein the means for capturing an image includes using a camera circuit and storing pixels representative of the object. (Item 24) The means for searching the database comprises: means for storing a plurality of entries in said database associating images with content; means for detecting a plurality of distinct items within a captured image of the object; means for identifying a given one of the plurality of entries that includes a given image that substantially corresponds to a given one of the distinct items; means for retrieving the content associated with the given image; 23. The system of claim 22, comprising: (Item 25) 25. The system of claim 24, wherein the content comprises a logo of a first team, and further comprising means for determining a name of the first team based on a database search of the logo and based on the metadata. (Item 26) the event is a sporting event; means for identifying a plurality of competitors in the sporting event; means for determining that the first team is featured in the sporting event; means for searching the plurality of competitors and identifying a second team featured in the sporting event; means for selecting one of the messages that includes the second team as the given message; 26. The system of claim 25, further comprising: (Item 27) means for identifying a time associated with said event; means for selecting a subset of the list of messages transmitted to the social network within the time threshold; 23. The system of claim 22, further comprising: (Item 28) means for storing a picture representing a view through an augmented reality device of the given message combined with the reference detected on the object; means for generating a display for the user with an option to transmit the photo to the social network; 23. The system of claim 22, further comprising: (Item 29) 23. The system of claim 22, wherein the object includes an item of clothing worn by the second user, the portion of the image includes a graphic on the item of clothing worn by the second user, and the content includes a team represented by the graphic. (Item 30) 23. The system of claim 22, further comprising means for transmitting a message including the composite content of the given message and the references found on the object to a second user associated with the object. (Item 31) 23. The system of claim 22, wherein the augmented reality device comprises a mobile phone or a headset worn by a user. (Item 32) A non-transient computer readable medium having non-transient computer readable instructions encoded on the non-transient computer readable medium for presenting complementary content in an augmented reality, the non-transient computer readable medium comprising: instructions for positioning the first user's augmented reality device over an object within a field of view of the augmented reality device; instructions for identifying the object within a field of view of the first user's augmented reality device; instructions for capturing an image of the object; instructions for searching a database to identify content that matches at least a portion of the image of the object; instructions for, in response to identifying content matching at least a portion of an image of the object, reading metadata associated with the content and detecting references on the object related to participants of an event; instructions for searching a list of messages previously transmitted by the first user to a social network to identify a given message that includes content that matches the participant; instructions for identifying a location on the object of the reference associated with the participant by determining an area of ​​the object within the field of view that includes a portion of the image that matches the identified content; instructions for combining the given message with the reference detected on the object at a position relative to the identified location using the augmented reality device; 1. A non-transitory computer-readable medium comprising: (Item 33) Item 33. The non-transitory computer-readable medium of item 32, wherein the instructions for capturing the image use a camera circuit and store pixels representing the object. (Item 34) The instructions for searching the database include: instructions for storing a plurality of entries in the database associating images with content; instructions for detecting a plurality of distinct items within a captured image of the object; instructions for identifying a given one of the plurality of entries that includes a given image that substantially corresponds to a given one of the distinct items; instructions for retrieving the content associated with the given image; 33. The non-transitory computer-readable medium of item 32, comprising: (Item 35) 35. The non-transitory computer-readable medium of claim 34, wherein the content comprises a logo of a first team and further comprises instructions for determining a name of the first team based on a database search of the logo and based on the metadata. (Item 36) the event is a sporting event; instructions for identifying a plurality of competitors in the sporting event; instructions for determining that the first team is featured in the sporting event; instructions for searching the plurality of competitors to identify a second team featured in the sporting event; instructions for selecting one of the messages including the second team as the given message; Item 36. The non-transitory computer-readable medium of item 35, further comprising: (Item 37) instructions for identifying a time associated with the event; instructions for selecting a subset of the list of messages transmitted to the social network within the time threshold; Item 33. The non-transitory computer-readable medium of item 32, further comprising: (Item 38) instructions for storing a photograph representing a view through an augmented reality device of the given message combined with the reference detected on the object; instructions for generating a display for the user of an option to transmit the photo to the social network; Item 33. The non-transitory computer-readable medium of item 32, further comprising: (Item 39) Item 33. The non-transitory computer-readable medium of item 32, wherein the object includes an article of clothing worn by a second user, the portion of the image includes a graphic on the article of clothing worn by the second user, and the content includes a team represented by the graphic. (Item 40) Item 33. The non-transitory computer-readable medium of item 32, further comprising instructions for transmitting a message including the combined content of the given message and the references found on the object to a second user associated with the object. (Item 41) Item 33. The non-transitory computer-readable medium of item 32, wherein the augmented reality device comprises a mobile phone or a headset worn by a user. (Item 42) 1. A method for presenting complementary content in augmented reality, the method comprising: Positioning an augmented reality device of a first user over an object within a field of view of the augmented reality device; identifying the object within a field of view of the first user's augmented reality device; capturing an image of the object; searching a database to identify content that matches at least a portion of the image of the object; in response to identifying content matching at least a portion of the image of the object, reading metadata associated with the content and detecting references on the object related to participants of the event; searching a list of messages previously transmitted by the first user to a social network to identify a given message that includes content matching the participant; identifying a location on the object of the reference associated with the participant by determining an area of ​​the object within the field of view that includes a portion of the image that matches the identified content; combining the given message with the reference detected on the object at a position relative to the identified location using the augmented reality device; A method comprising: (Item 43) Item 43. The method of item 42, wherein capturing the image includes using a camera circuit and storing pixels representing the object. (Item 44) Searching the database includes: storing a plurality of entries in the database associating images with content; detecting a plurality of distinct items within a captured image of the object; identifying a given one of the plurality of entries that includes a given image that substantially corresponds to a given one of the distinct items; Retrieving the content associated with the given image; 44. The method according to any one of items 42-43, comprising: (Item 45) 45. The method of any one of claims 42-44, wherein the content comprises a logo of a first team, and further comprising determining a name of the first team based on a database search of the logo and based on the metadata. (Item 46) the event is a sporting event; identifying a plurality of competitors in the sporting event; determining that the first team is featured in the sporting event; searching the plurality of competitors to identify a second team featured in the sporting event; selecting one of the messages including the second team as the given message; 46. ​​The method according to any one of items 42-45, further comprising: (Item 47) identifying a time associated with the event; selecting a subset of the list of messages that were transmitted to the social network within the time threshold; 47. The method according to any one of items 42-46, further comprising: (Item 48) storing a photograph representing a view through an augmented reality device of the given message combined with the reference detected on the object; generating a display for the user with an option to transmit the photo to the social network; 48. The method according to any one of items 42-47, further comprising: (Item 49) Item 49. The method of any one of items 42-48, wherein the object includes an item of clothing worn by the second user, the portion of the image includes a graphic on the item of clothing worn by the second user, and the content includes a team represented by the graphic. (Item 50) A method according to any one of items 42-49, further comprising transmitting a message including the composite content of the given message and the reference found on the object to a second user associated with the object. (Item 51) 51. The method of any one of claims 42-50, wherein the augmented reality device comprises a mobile phone or a headset worn by a user.

[0020] The above and other objects and advantages of the present disclosure will become apparent from a consideration of the following detailed description taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 shows an illustrative example of a user of an augmented reality device, according to some embodiments of the present disclosure.

[0022] [Diagram 2] FIG. 2 shows an illustrative example of an augmented reality display according to some embodiments of the present disclosure.

[0023] [Diagram 3] FIG. 3 shows an illustrative example of a user's social media history, according to some embodiments of the present disclosure.

[0024] [Figure 4] 4-7 show illustrative examples of augmented reality displays according to some embodiments of the present disclosure. [Diagram 5] 4-7 show illustrative examples of augmented reality displays according to some embodiments of the present disclosure. [Figure 6] 4-7 show illustrative examples of augmented reality displays according to some embodiments of the present disclosure. [Figure 7] 4-7 show illustrative examples of augmented reality displays according to some embodiments of the present disclosure.

[0025] [Figure 8] 8-9 show other illustrative examples of display screens generated by a media guidance application in accordance with some embodiments of this disclosure. [Figure 9] 8-9 show other illustrative examples of display screens generated by a media guidance application in accordance with some embodiments of this disclosure.

[0026] [Figure 10] FIG. 10 is a block diagram of an illustrative user equipment device in accordance with some embodiments of the disclosure.

[0027] [Figure 11] FIG. 11 is a block diagram of an illustrative media system according to some embodiments of the disclosure.

[0028] [Figure 12] FIG. 12 is a flowchart of an illustrative process for adding a message to an object in an augmented reality view, according to some embodiments of the disclosure.

[0029] [Figure 13] FIG. 13 is a flowchart of another illustrative process for adding a message to an object in an augmented reality view, according to some embodiments of the disclosure.

[0030] [Figure 14] FIG. 14 is a flowchart of an illustrative process for generating social media images for a user, according to some embodiments of the present disclosure.

[0031] [Figure 15] FIG. 15 is a flowchart of an illustrative process for selecting a message to add to an object in an augmented reality view, according to some embodiments of the disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Thus, systems and methods are described herein for enhancing a user's augmented reality experience. In particular, systems and methods are presented herein for adding messages, such as messages from a user's social network, to objects within a field of view in an augmented reality device.

[0033] Augmented reality devices such as augmented reality headsets or smart glasses, or portable devices with augmented reality applications such as mobile phones, smartphones, or tablets, may be used to present the augmented reality view. The view may be presented with the augmented reality device or application so that the user can see a particular view with additional information about the view. For example, an augmented reality environment at a cosmetics counter may show the user with various types of applied makeup. In another example, augmented reality can be used at home to show how furniture might fit into a room. In another example, an augmented reality device can be used while grocery shopping to make price and nutritional comparisons. While these applications are very useful to users, users may also want to have them tailored for information to be added to the augmented reality.

[0034] One way this may be accomplished is by using the media guidance application to analyze the view within the augmented reality environment to detect objects within the view. The objects can be analyzed to determine references within the object that may have participants or events associated with them. The media guidance application may search the user's social network to identify content related to the object, participant, or event. For example, the media guidance application may search for posts by the user on social media that name or describe the object, participant, or event. If such a message is identified by the media guidance application, the media guidance application may add that message or some version of the message to the augmented reality view of the object. For example, the media guidance application may generate an overlay of the message to be displayed over an image in the augmented reality environment, and in particular over or near the associated object. This may be useful to the user because the message was originally provided by the user in another context and is now being added to the user's augmented reality view.

[0035] See Fig. 1, which depicts a street scene 100 including a user 110 going out to stroll the street wearing an augmented reality headset 120 so that the environment can be viewed with additional information provided by the augmented reality device. In an embodiment, the user may walk around with the augmented reality device and view direction or map information while walking, and the augmented reality device 120 may also capture images from the user's current environment. For example, the augmented reality device 120 may capture an individual 150 on the sidewalk wearing a T-shirt with the New England Patriots logo.

[0036] The augmented reality device may also be a smartphone that may capture a view of a street, such as that shown in FIG. 2, in field of view 200, including an individual 210 wearing a T-shirt with a New England Patriots logo. Any of the augmented reality environments shown in FIGS. 1 and 2 may be suitable for a user to access the improved augmented reality features described herein. The view seen in the augmented reality environment may be captured, for example, as an image, by the media guidance application using camera circuitry in the augmented reality device. The user may focus on a particular aspect of the view, which may inform the media guidance application of an item of interest in the view. Alternatively, the media guidance application may analyze the image and identify the object using object recognition techniques. The media guidance application may also request external databases and information, and user information, to provide useful information to include in the augmented reality view.

[0037] With reference to FIG. 3, the media guidance application may, for example, search the user's social network to find information about the user that may be interesting to view within the augmented reality environment. As shown in FIG. 3, the user may have a record 300 of comments or posts on a social network that includes date information and messages, for example, 302 and 310, and 305 and 315, respectively. As may be seen by record 300, the user may have some interest in football. The media guidance application may analyze such comments to determine the interest. In addition, the media guidance application may search a sports schedule or sports media database for metadata that may define the sporting events referenced in messages 305 and 315, for example.

[0038] Using social network information about the user from record 300, the media guidance application may find messages related to references or objects in a view captured by the augmented reality device, such as view 200 (FIG. 2). For example, the media guidance application may identify individual 210 (FIG. 2) and their T-shirt logo and detect that the logo is related to the New England Patriots and football. The media guidance application may analyze the user's social media record 300 and detect messages about football. Such messages may be included within the augmented reality view. For example, message 305 may be added to augmented reality view 200 (FIG. 2) as shown in FIG. 4.

[0039] FIG. 4 depicts an augmented reality view 400 that may be visible to a user using an augmented reality device. In view 400, the media guidance application has captured an object in view 400, namely, pedestrian 405 wearing a T-shirt with a New England Patriots logo. The media guidance application may search the user's social media network information to find a suitable message to include in the view. As shown in FIG. 4, message 410 has been added to view 400 that was taken from the user's social media record (e.g., message 305 (FIG. 3)). The message selected for inclusion in augmented reality view 400 may be based on a variety of factors, such as time, relevance, user interest, or another criterion. Message 410 selected for view 400 may be deemed relevant, for example, due to timeliness. Because message 410 is a temporally recent message posted by the user (see, e.g., record 300 (FIG. 3)), the media guidance application may determine that it is suitable for inclusion in view 400.

[0040] In another example, a message may be selected to be added to the augmented reality view by the media guidance application based on the social network record and additional information. As shown in FIG. 5, augmented reality view 500 may include an individual 505 wearing a T-shirt with a New England Patriots logo. The user may focus on the T-shirt, or the media guidance application may capture the logo as an interesting object in the image. The media guidance application may search for suitable content to include in augmented reality view 500. In searching the user's social network record (e.g., record 300 (FIG. 3)), the media guidance application may detect an interest in football. Further, the media guidance application may select a recent post, such as message 305 (FIG. 3), and determine whether there are any associated events for the object related to the message. For example, the media guidance application may search a media database or a sports database and identify sporting events related to the sport or team that are around the time of the post. Thus, for message 305 (FIG. 3), the media guidance application may search for football events around the date of the message. In addition, the media guidance application may search for football events for a team associated with a logo or reference object from the augmented reality view. A search of the sports database may show New England Patriots games after November 21, such as the Dolphins vs. Patriots game on November 26, 2017, as shown in message 510.

[0041] In another example, the media guidance application may obtain information about a reference object in the augmented reality view and select a message that provides a link to access content related to the reference object, which may also be based on the user's history, e.g., preference profile or social network. As shown in FIG. 6, augmented reality view 600 may include an individual 605 wearing a T-shirt with the logo of the New England Patriots football team. The media guidance application may detect the logo as a reference object in the image of view 600 and search for a suitable message to overlay in view 600 that is relevant to the object. The media guidance application may use image analysis tools to identify the logo as being for the New England Patriots. The media guidance application may also search a sports database for metadata about the team. The media guidance application may also search the user's history, e.g., social network records 300 (FIG. 3), for messages that may be relevant to the team or certain metadata identified in the search of the sports database. The media guidance application may determine that a team, i.e., the New England Patriots, was a participant in the Super Bowl referenced in message 315 (FIG. 3). Thus, the media guidance application may select clips related to teams playing in the Super Bowl to provide in links overlaid in view 600.

[0042] In another example, the media guidance application may select a reference object identified within the augmented reality view for sharing. The reference object may be shared on social media for the user or shared with another individual. For example, as shown in FIG. 7, the augmented reality view 700 may include an individual 705 wearing a New England Patriots T-shirt, and the media guidance application may detect the T-shirt as a reference object within the view. Detection of the T-shirt and its logo may be based on image analysis and recognition. The logo may be stored in a user preference profile, for example, based on the user's viewing history or social network history. The media guidance application may capture the reference object, e.g., the T-shirt and logo, and provide a message 720 within the augmented reality view 700 that includes the reference object and a comment. The comment "sleep after eating turkey and can't wait for football" may be obtained by the media guidance application, for example, using the user's social network history. For example, the comment may be based on recent posts 305 (FIG. 3) that the media guidance application considers to be timely and related to the sport associated with the logo. Message 720 may also include selectable options 730 for the user to post the image to a social network or send it to a recipient, or other manner of sharing the image. In some examples, the recipient may be identified in message 720 and options 730 as an individual 705 who may have access to the user's contacts and image history, and who may be identified by the media guidance application through facial / image recognition techniques.

[0043] Although the embodiments described with reference to Figures 1-7 involved only sporting events, other types of image objects may also be used in the same manner. For example, faces, brand names, flags, television programs, landmarks, or other objects may be identified using image recognition techniques, and a database search on metadata associated with the images may be performed to find events related to such items. For example, a billboard advertisement showing Lady Gaga may be recognized, and a database search on her name may include information about her appearance at the 2017 Super Bowl halftime show, which may also be added to the augmented reality view and related to the user's comments 315 (Figure 3) about watching the Super Bowl. In another embodiment, a billboard showing Lady Gaga may carry a message about her upcoming concert.

[0044] The amount of content available to users in any given content delivery system can be staggering. As a result, many users desire a form of media guidance through an interface that allows them to efficiently navigate content selections and easily identify content that may be desired. Applications that provide such guidance are referred to herein as interactive media guidance applications, but may also sometimes be referred to as media guidance applications or guidance applications.

[0045] Interactive media guidance applications may take a variety of forms depending on the content for which they provide guidance. One typical type of media guidance application is an interactive television program guide. Interactive television program guides (sometimes referred to as electronic program guides) are well-known guidance applications that, among other things, allow users to navigate among and identify many types of content or media assets. Interactive media guidance applications may generate graphical user interface screens that allow users to navigate among, identify and select content. As referred to herein, the terms "media assets" and "content" should be understood to mean television programs and electronically consumable user assets such as pay programs, on-demand programs (as in video-on-demand (VOD) systems), Internet content (e.g., streaming content, downloadable content, webcasts, etc.), video clips, audio, content information, photos, rotating images, documents, playlists, websites, articles, books, e-books, blogs, chat sessions, social media, applications, games, and / or any other media or multimedia, and / or combinations thereof. Guidance applications also allow users to navigate among and identify content. As referred to herein, the term "multimedia" should be understood to mean content that utilizes at least two of the different content forms described above, e.g., text, audio, image, video, or interactive content forms. The content may be recorded, played, displayed, or accessed by a user equipment device, but may also be part of a live performance.

[0046] The media guidance application and / or any instructions for implementing any of the embodiments discussed herein may be encoded on a computer-readable medium. A computer-readable medium includes any medium capable of storing data. A computer-readable medium may be transient, including, but not limited to, the propagation of electrical or electromagnetic signals, or non-transient, including, but not limited to, volatile and non-volatile computer memory or storage devices, such as hard disks, floppy disks, USB drives, DVDs, CDs, media cards, register memory, processor cache, random access memory ("RAM"), and the like.

[0047] With the advent of the Internet, mobile computing, and high-speed wireless networks, users are accessing media on user equipment devices that they had not traditionally used. As referred to herein, the terms “user equipment device”, “user equipment”, “user device”, “electronic device”, “electronic device”, “media equipment device”, or “media device” should be understood to mean any device for accessing the content described above, such as a television, a smart TV, a set-top box, an integrated receiver decoder (IRD) for supporting satellite television, a digital storage device, a digital media receiver (DMR), a digital media adapter (DMA), a streaming media device, a DVD player, a DVD recorder, a connected DVD, a local media server, a BLU-RAY® player, a BLU-RAY® recorder, a personal computer (PC), a laptop computer, a tablet computer, a Web TV box, a personal computer television (PC / TV), a PC media server, a PC media center, a handheld computer, a landline phone, a personal digital assistant (PDA), a mobile phone, a portable video player, a portable music player, a portable gaming console, a smartphone, or any other television equipment, computing equipment, or wireless device, and / or combinations thereof. In some embodiments, the user equipment devices may have front and rear screens, multiple front screens, or multiple angled screens. In some embodiments, the user equipment devices may have front and / or rear cameras. On these user equipment devices, users may be able to navigate among and locate the same content available through a television. As a result, media guides may also be available on these devices. The guides provided may be for content available only through a television, for content available only through one or more of the other types of user equipment devices, or for content available both through a television and one or more of the other types of user equipment devices.The media guidance application may be provided on a user equipment device, as an online application (i.e., provided on a website), or as a stand-alone application or client. Various devices and platforms on which the media guidance application may be implemented are described in more detail below.

[0048] One of the functions of the media guidance application is to provide media guidance data to a user. As referred to herein, the phrases "media guidance data" or "guide data" should be understood to mean any data related to content or data used in operating the guidance application. For example, the guide data may include program information, guide application settings, user preferences, user profile information, media listings, media-related information (e.g., broadcast time, broadcast channel, title, content, rating information (e.g., parental control rating, critics' rating, etc.), genre or category information, actor information, logo data of a broadcaster or provider logo, etc.), media format (e.g., standard definition, high definition, 3D, etc.), on-demand information, blogs, websites, and any other type of guide data that helps a user navigate among and identify desired content selections.

[0049] FIGS. 8-9 show illustrative display screens that may be used to provide media guidance data. The display screens shown in FIGS. 8-9 may be implemented on any suitable user equipment device or platform. Although the displays in FIGS. 8-9 are illustrated as full-screen displays, they may also be fully or partially overlaid on the displayed content. A user may indicate a desire to access content information by selecting a selectable option provided on the display screen (e.g., a menu option, a list option, an icon, a hyperlink, etc.) or by pressing a dedicated button (e.g., a “Guide” button) on a remote control or other user input interface or device. In response to a user indication, the media guidance application may provide the display screen with media guidance data organized in one of several ways, such as by time and channel in a grid, by time, by channel, by source, by content type, by category (e.g., movies, sports, news, kids, or other programming categories), or other predetermined, user-defined, or other organization criteria.

[0050] FIG. 8 shows an illustrative grid of program listings display 800 arranged by time and channel that also allows access to different types of content within a single display. Display 800 may include a grid 802 with: (1) columns of channel / content type identifiers 804, where each channel / content type identifier (cell in the column) identifies a different channel or type of content available; and (2) rows of time identifiers 806, where each time identifier (cell in the row) identifies a time slot for a program. Grid 802 also includes cells of program listings, such as program listing 808, where each listing provides the title of the program provided above the listing's associated channel and time. Using a user input device, a user can select a program listing by moving highlight region 810. Information about the program listing selected by highlight region 810 may be provided in program information region 812. Region 812 may include, for example, the program title, program content, the time the program is offered (if applicable), the channel the program is broadcast on (if applicable), the program's rating, and other desired information.

[0051] In addition to providing access to linear programming (e.g., content scheduled for transmission to multiple user equipment devices at predetermined times and provided according to a schedule), the media guide application also provides access to nonlinear programming (e.g., content that is accessible to user equipment devices at any time and is not provided according to a schedule). Nonlinear programming may include content from different content sources, including on-demand content (e.g., VOD), Internet content (e.g., streaming media, downloadable media, etc.), locally stored content (e.g., content stored on any of the user equipment devices described above or other storage devices), or other content that is not time-constrained. On-demand content may include movies or any other content provided by a particular content provider (e.g., HBO On Demand, which provides "The Sopranos" and "Curb Your Enthusiasm"). HBO ON DEMAND is a service mark owned by Time Warner Company LP et al., and THE SOPRANOS and CURB YOUR ENTHUSIASM are trademarks owned by Home Box Office, Inc. Internet content may include web events such as chat sessions or webcasts, or content available on demand as streaming or downloadable content through an Internet website or other Internet access (eg, FTP).

[0052] Grid 802 may provide media guide data for non-linear programming, including on-demand listings 814, recorded content listings 816, and Internet content listings 818. Displays that combine media guidance data for content from different types of content sources are sometimes referred to as "mixed media" displays. Various permutations of the types of media guidance data that may be displayed, other than display 800, may be based on user selection or guide application definition (e.g., displaying only recorded and broadcast listings, displaying only on-demand and broadcast listings, etc.). As illustrated, listings 814, 816, and 818 are shown as spanning the entire time period displayed in grid 802 to indicate that selection of these listings may provide access to a dedicated view of on-demand listings, recorded listings, or Internet listings, respectively. In some embodiments, listings for these content types may be included directly in grid 802. In response to a user selecting one of navigation icons 820, additional media guidance data may be displayed. (Pressing an arrow key on a user input device may affect the display in a similar manner to selecting navigation icon 820.)

[0053] Display 800 may also include a video region 822 and an options region 826. Video region 822 may allow a user to watch and / or preview programs that are currently available, will be available, or have been available to the user. The content of video region 822 may correspond to or be independent of one of the listings displayed in grid 802. Grid displays that include a video region are sometimes referred to as picture-in-guide (PIG) ​​displays. PIG displays and their functionality are described in more detail in U.S. Pat. No. 6,564,378 to Satterfield et al., issued May 13, 2003, and U.S. Pat. No. 6,239,794 to Yuen et al., issued May 29, 2001, which are incorporated herein by reference in their entireties. PIG displays may be included in other media guidance application display screens of the embodiments described herein.

[0054] Options region 826 may allow a user to access different types of content, media guidance application views, and / or media guidance application features. Options region 826 may be part of display 800 (and other display screens described herein) or may be invoked by a user by selecting an option on the screen or by pressing a dedicated or assignable button on a user input device. The selectable options in options region 826 may relate to features related to program listings in grid 802 or may include options available from a main menu display. Features related to program listings may include finding other air times or methods of receiving a program, recording a program, enabling continuous recording of a program, setting a program and / or channel as a favorite, purchasing a program, or other features. Options available from a main menu display may include search options, VOD options, parental control options, Internet options, cloud-based options, device sync options, second screen device options, options to access various types of media guidance data displays, options to subscribe to premium services, options to edit a user's profile, options to access a browser overlay, or other options.

[0055] The media guidance application may be personalized based on user preferences. A personalized media guidance application allows a user to customize displays and features to create a personalized “experience” with the media guidance application. This personalized experience may be created by allowing the user to input these customizations and / or by the media guidance application monitoring user activity to determine various user preferences. Users may access their personalized guidance application by logging in or otherwise identifying themselves to the guidance application. Customizations of the media guidance application may be created according to a user profile. Customizations may include changing the presentation scheme (e.g., color scheme of the display, font size of text, etc.), aspects of the content listings displayed (e.g., only HDTV programs or only 3D programs, user-specified broadcast channels based on favorite channel selections, sorting of channel display, recommended content, etc.), desired recording characteristics (e.g., recording or continuous recording for a particular user, recording quality, etc.), parental control settings, customized presentation of Internet content (e.g., presentation of social media content, email, electronically distributed articles, etc.), and other desired customizations.

[0056] The media guidance application may allow a user to provide user profile information or may automatically compile user profile information. The media guidance application may, for example, monitor the content the user accesses and / or other interactions the user may have with the guidance application. In addition, the media guidance application may obtain other user profiles, in whole or in part, associated with a particular user (e.g., from other websites on the Internet the user accesses, such as www.Tivo.com, from other media guidance applications the user accesses, from other interactive applications the user accesses, from the user's other user equipment devices, etc.) and / or obtain information about the user from other sources to which the media guidance application may have access. As a result, a user can be provided with a unified guidance application experience across the user's different user equipment devices. This type of user experience is described in more detail below in connection with FIG. 11. Additional personalized media guidance application features are described in U.S. Patent Application Publication No. 2005 / 0251827 to Ellis et al., filed July 11, 2005; U.S. Patent Application Publication No. 2005 / 0251827 to Boyer et al., published January 16, 2007; and U.S. Patent Application Publication No. 2005 / 0251827 to Boyer et al., published January 16, 2007. No. 7,165,098 to Ellis et al., and U.S. Patent Application Publication No. 2002 / 0174430, filed February 21, 2002, which are incorporated by reference in their entireties.

[0057] Another display arrangement for providing a media guide is shown in FIG. 9. Video mosaic display 900 includes selectable options 902 for content information organized based on content type, genre, and / or other organization criteria. In display 900, television listings option 904 is selected, thus providing listings 906, 908, 910, and 912 as broadcast program listings. In display 900, the listings may provide graphic images including cover art, still images from the content, previews of video clips, live video from the content, or other types of content that indicate to the user the content described by the media guide data in the listing. The graphic listings may also be accompanied by text, respectively, to provide further information about the content associated with the listing. For example, listing 908 may include more than one portion, including media portion 914 and text portion 916. Media portion 914 and / or text portion 916 may be selectable to watch the content full screen or to view information related to the content displayed in media portion 914 (e.g., to view listings for the channel on which the video is displayed).

[0058] The listings in display 900 are of different sizes (i.e., listing 906 is larger than listings 908, 910, and 912), although all listings could be the same size, as desired. The listings may be differently sized or graphically highlighted to indicate a degree of user interest or to highlight certain content, as desired by the content provider or based on user preferences. Various systems and methods for graphically highlighting content listings are discussed, for example, in U.S. Patent Application Publication No. 2010 / 0153885 to Yates, filed Nov. 12, 2009, which is incorporated herein by reference in its entirety.

[0059] A user may access content and media guidance applications (and their display screens, described above and below) from one or more of their user equipment devices. FIG. 10 shows a generalized embodiment of an illustrative user equipment device 1000. A more specific implementation of a user equipment device is discussed below in connection with FIG. 11. User equipment device 1000 may receive content and data via input / output (hereinafter “I / O”) path 1002. I / O path 1002 may provide content (e.g., broadcast programs, on-demand programs, Internet content, content available over a local area network (LAN) or wide area network (WAN), and / or other content) and data to control circuitry 1004, which includes processing circuitry 1006 and storage device 1008. Control circuitry 1004 may be used to send and receive commands, requests, and other suitable data using I / O path 1002. I / O path 1002 may connect control circuitry 1004 (specifically, processing circuitry 1006) to one or more communication paths (described below). An I / O function may be provided by one or more of these communication paths, but is shown as a single path in FIG. 10 to avoid overcomplicating the drawing.

[0060] Control circuitry 1004 may be based on any suitable processing circuitry, such as processing circuitry 1006. As referred to herein, processing circuitry is understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc., and may include multi-core processors (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputers. In some embodiments, processing circuitry may be distributed across multiple separate processors or processing units, such as multiple processing units of the same type (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry 1004 executes instructions for a media guidance application stored in memory (i.e., storage device 1008). In particular, control circuitry 1004 may be instructed by the media guidance application to perform the functions discussed above and below. For example, the media guidance application may provide control circuitry 1004 with instructions to generate a media guidance display. In some implementations, any actions performed by control circuitry 1004 may be based on instructions received from a media guidance application.

[0061] In a client-server based embodiment, the control circuitry 1004 may include suitable communications circuitry for communicating with a guide application server or other network or server. Instructions for implementing the functionality described above may be stored on the guide application server. The communications circuitry may include a cable modem, an Integrated Services Digital Network (ISDN) modem, a Digital Subscriber Line (DSL) modem, a telephone modem, an Ethernet card, or a wireless modem for communication with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communications network or path (described in more detail in connection with FIG. 11). In addition, the communications circuitry may include circuitry to enable peer-to-peer communication of user equipment devices, or communication of user equipment devices at locations remote from each other (described in more detail below).

[0062] The memory may be an electronic storage device provided as storage device 1008 that is part of control circuitry 1004. As referred to herein, the phrase "electronic storage device" or "storage device" should be understood to mean any device for storing electronic data, computer software, or firmware, such as random access memory, read-only memory, hard drive, optical drive, digital video disc (DVD) recorder, compact disc (CD) recorder, BLU-RAY® Disc (BD) recorder, BLU-RAY® 3D Disc recorder, digital video recorder (DVR, or sometimes called personal video recorder or PVR), solid state device, quantum storage device, game console, game media, or any other suitable fixed or removable storage device, and / or any combination thereof. Storage device 1008 may be used to store various types of content described herein, as well as media guidance data described above. Non-volatile memory may also be used (e.g., to initiate boot-up routines and other instructions). Cloud-based storage, as described in connection with FIG. 11, may be used to supplement storage 1008 or in place of storage 1008.

[0063] The control circuitry 1004 may include video generation and tuning circuitry, such as one or more analog tuners, one or more MPEG-2 decoders or other digital decoding circuitry, high definition tuners, or any other suitable tuning or video circuitry, or a combination of such circuits. Encoding circuitry (e.g., for converting over-the-air, analog, or digital signals to MPEG signals for storage) may also be provided. The control circuitry 1004 may also include scaling circuitry for up-converting and down-converting content to a preferred output format of the user equipment 1000. The circuitry 1004 may also include digital-to-analog and analog-to-digital conversion circuitry for converting between digital and analog signals. The tuning and encoding circuitry may be used by the user equipment device to receive and display, play, or record content. The tuning and encoding circuitry may also be used to receive guide data. For example, the circuitry described herein, including tuning, video generation, encoding, decoding, encryption, decryption, scaling, and analog / digital circuitry, may be implemented using software running on one or more general-purpose or specialized processors. Multiple tuners may be provided to accommodate simultaneous tuning functions (e.g., watch and record functions, picture-in-picture (PIP) functions, multi-tuner recording functions, etc.) If storage 1008 is provided as a device separate from user equipment 1000, the tuning and encoding circuitry (including multiple tuners) may be associated with storage 1008.

[0064] A user may send instructions to the control circuitry 1004 using the user input interface 1010. The user input interface 1010 may be any suitable user interface, such as a remote control, a mouse, a trackball, a keypad, a keyboard, a touch screen, a touch pad, a stylus input, a joystick, a voice recognition interface, or other user input interface. The display 1012 may be provided as a stand-alone device or may be integrated with other elements of the user equipment device 1000. For example, the display 1012 may be a touch screen or a touch-sensitive display. In such situations, the user input interface 1010 may be integrated or combined with the display 1012. The display 1012 may be one or more of a monitor, a television, a liquid crystal display (LCD) for a mobile device, an amorphous silicon display, a low temperature polysilicon display, an electronic ink display, an electrophoretic display, an active matrix display, an electrowetting display, an electrofluidic display, a cathode ray tube display, a light emitting diode display, an electroluminescent display, a plasma display panel, a high performance addressing display, a thin film transistor display, an organic light emitting diode display, a surface conduction electron emitter display (SED), a laser television, a carbon nanotube, a quantum dot display, an interferometric modulator display, or any other suitable device for displaying visual images. In some embodiments, the display 1012 may be HDTV compatible. In some embodiments, the display 1012 may be a 3D display, and the interactive media guide application and any suitable content may be displayed in 3D. A video card or graphics card may generate output to the display 1012. The video card may provide various functions such as accelerated rendering of 3D scenes and 2D graphics, MPEG-2 / MPEG-4 decoding, a TV output, or the ability to connect multiple monitors.The video card may be any of the processing circuitry described above in association with control circuitry 1004. The video card may be integrated with control circuitry 1004. Speakers 1014 may be provided as integrated with other elements of user equipment device 1000 or may be a stand-alone unit. Audio components of video and other content displayed on display 1012 may be played through speakers 1014. In some embodiments, the audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers 1014.

[0065] The guide application may be implemented using any suitable architecture. For example, it may be a standalone application implemented entirely on user equipment device 1000. In such an approach, the application's instructions are stored locally (e.g., in storage 1008) and data for use by the application is periodically downloaded (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach). Control circuitry 1004 may read the application's instructions from storage 1008 and process the instructions to generate any of the displays discussed herein. Based on the processed instructions, control circuitry 1004 may determine an action to perform when input is received from input interface 1010. For example, moving a cursor up / down on a display may be indicated by the processed instructions when input interface 1010 indicates that an up / down button has been selected.

[0066] In some embodiments, the media guidance application is a client-server based application. Data for use by a thick or thin client implemented on user equipment device 1000 is retrieved on demand by issuing requests to a server remote from user equipment device 1000. In one example of a client-server based guidance application, control circuitry 1004 launches a web browser that interprets web pages provided by a remote server. For example, the remote server may store instructions for the application in a storage device. The remote server may use circuitry (e.g., control circuitry 1004) to process the stored instructions and generate the displays discussed above and below. The client device may receive the displays generated by the remote server and display the contents of the displays locally on equipment device 1000. In this manner, the processing of instructions is performed remotely by a server while the resulting displays are provided locally on equipment device 1000. Equipment device 1000 may receive inputs from a user via input interface 1010 and transmit those inputs to the remote server for processing and generating corresponding displays. For example, the appliance device 1000 may transmit a communication to a remote server indicating that an up / down button was selected via the input interface 1010. The remote server may process instructions according to the input and generate a display of the application corresponding to the input (e.g., a display that moves a cursor up / down). The generated display is then transmitted to the appliance device 1000 for presentation to the user.

[0067] In some embodiments, the media guidance application is downloaded and interpreted or otherwise invoked by an interpreter or virtual machine (invoked by control circuitry 1004). In some embodiments, the guidance application may be encoded in ETV Binary Interchange Format (EBIF) and received by control circuitry 1004 as part of a suitable feed and interpreted by a user agent running on control circuitry 1004. For example, the guidance application may be an EBIF application. In some embodiments, the guidance application may be defined by a series of JAVA-based files that are received and invoked by a local virtual machine or other suitable middleware executed by control circuitry 1004. In some such embodiments (e.g., those employing MPEG-2 or other digital media encoding schemes), the guidance application may be encoded and transmitted, for example, in an MPEG-2 object carousel with the program's MPEG audio and video packets.

[0068] User equipment devices 1000 of FIG. 10 may be implemented in system 1100 of FIG. 11 as user television equipment 1102, user computer equipment 1104, wireless user communications devices 1106, or any other type of user equipment suitable for accessing content, such as non-portable gaming consoles. For simplicity, these devices may be referred to collectively herein as user equipment or user equipment devices and may be substantially similar to the user equipment devices described above. The user equipment devices on which the media guidance application may be implemented may function as stand-alone devices or may be part of a network of devices. Various network configurations of devices may be implemented and are discussed in more detail below.

[0069] User equipment devices utilizing at least some of the system features described above in connection with FIG. 10 may not simply be classified as user television equipment 1102, user computer equipment 1104, or wireless user communications device 1106. For example, user television equipment 1102, like some user computer equipment 1104, may be Internet-enabled allowing access to Internet content, while user computer equipment 1104, like some television equipment 1102, may include a tuner allowing access to television programs. The media guidance application may also have the same layout on various different types of user equipment or may be tailored to the display capabilities of the user equipment. For example, on user computer equipment 1104, the guidance application may be provided as a website accessed by a web browser. In another example, the guidance application may be scaled down for wireless user communications device 1106.

[0070] In system 1100, there will typically be more than one of each type of user equipment device, but to avoid overcomplicating the drawing, only one of each is shown in Figure 11. In addition, each user may utilize more than one type of user equipment device and also more than one of each type of user equipment device.

[0071] In some embodiments, a user equipment device (e.g., user television equipment 1102, user computer equipment 1104, wireless user communications device 1106) may be referred to as a "second screen device." For example, the second screen device may complement the content presented on the first user equipment device. The content presented on the second screen device may be any suitable content that complements the content presented on the first device. In some embodiments, the second screen device provides an interface for adjusting settings and display preferences of the first device. In some embodiments, the second screen device is configured to interact with other second screen devices or to interact with a social network. The second screen device may be located in the same room as the first device, in a different room from the first device but in the same home or building, or in a different building from the first device.

[0072] A user may also set various settings to maintain consistent media guidance application settings across in-home and remote devices. Settings include those described herein as well as favorite channels and programs, program preferences that the guidance application utilizes to recommend programs, display preferences, and other desired guide settings. For example, if a user sets a channel as a favorite on his or her office personal computer, e.g., on the website www.Tivo.com, the same channel will be displayed as a favorite on the user's in-home devices (e.g., user television equipment and user computer equipment) and, if desired, on the user's mobile devices. Thus, changes made on one user equipment device can change the guide experience on another user equipment device, whether the same or a different type of user equipment device. In addition, changes made may be based on settings entered by the user and user activity monitored by the guidance application.

[0073] User equipment devices may be coupled to communications network 1114. That is, user television equipment 1102, user computer equipment 1104, and wireless user communications device 1106 are coupled to communications network 1114 via communications paths 1108, 1110, and 1112, respectively. Communications network 1114 may be one or more networks including the Internet, a cellular network, a mobile voice or data network (e.g., a 4G or LTE network), a cable network, a public switched telephone network, or other types of communications networks, or combinations of communications networks. Paths 1108, 1110, and 1112, separately or together, may include one or more communications paths, such as satellite paths, fiber optic paths, cable paths, paths supporting Internet communications (e.g., IPTV), free space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications paths, or combinations of such paths. Path 1112 is depicted in the exemplary embodiment shown in Figure 11 as a dashed line to indicate that it is a wireless path, and paths 1108 and 1110 are depicted as solid lines to indicate that they are wired paths (although these paths may also be wireless paths, if desired). Communication with a user equipment device may be provided by one or more of these communication paths, but are shown in Figure 11 as a single path to avoid overcomplicating the drawing.

[0074] Although communication paths are not depicted between the user equipment devices, these devices may communicate directly with each other via communication paths such as those described above in connection with paths 1108, 1110, and 1112, and other short-range point-to-point communication paths such as a USB cable, an IEEE 1394 cable, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or other short-range communications via wired or wireless paths. BLUETOOTH is a certification mark owned by Bluetooth SIG, INC. User equipment devices may also communicate directly with each other through indirect paths via communications network 1114.

[0075] System 1100 includes a content source 1116 and a media guidance data source 1118 coupled to communications network 1114 via communications paths 1120 and 1122, respectively. Paths 1120 and 1122 may include any of the communications paths described above in connection with paths 1108, 1110, and 1112. Communications with content source 1116 and media guidance data source 1118 may be exchanged via one or more communications paths, but are shown in FIG. 11 as a single path to avoid overcomplicating the drawing. In addition, there may be more than one of each of content source 1116 and media guidance data source 1118, but only one of each is shown in FIG. 11 to avoid overcomplicating the drawing. (Each of these different types of sources is discussed below.) If desired, content source 1116 and media guidance data source 1118 may be integrated as one source device. Although communications between sources 1116 and 1118 and user equipment devices 1102, 1104, and 1106 are shown as being through communications network 1114, in some embodiments sources 1116 and 1118 may communicate directly with user equipment devices 1102, 1104, and 1106 via communications paths (not shown) such as those described above in connection with paths 1108, 1110, and 1112.

[0076] The content sources 1116 may include one or more types of content distribution equipment, including television distribution facilities, cable system head ends, satellite distribution facilities, program sources (e.g., television broadcasting companies, such as NBC, ABC, HBO, etc.), intermediate distribution facilities and / or servers, Internet providers, on-demand media servers, and other content providers. NBC is a trademark owned by National Broadcasting Company, Inc., ABC is a trademark owned by American Broadcasting Company, INC., and HBO is a trademark owned by Home Box Office, Inc. The content sources 1116 may be content originators (e.g., television broadcasting companies, webcast providers, etc.) or non-content originators (e.g., on-demand content providers, Internet providers of broadcast program content for download, etc.). The content sources 1116 may include cable sources, satellite providers, on-demand providers, Internet providers, over-the-top content providers, or other content providers. Content sources 1116 may also include remote media servers used to store different types of content (including video content selected by the user) at locations remote from any of the user equipment devices. Systems and methods for remote storage of content and for providing remotely stored content to user equipment are discussed in more detail in connection with U.S. Patent No. 7,761,892, issued July 20, 2010 to Ellis et al., which is incorporated herein by reference in its entirety.

[0077] Media guidance data source 1118 may provide media guidance data, such as the media guidance data described above. The media guidance data may be provided to user equipment devices using any suitable approach. In some embodiments, the guide application may be a stand-alone interactive television program guide that receives program guide data via a data feed (e.g., a continuous feed or a trickle feed). Program schedule data and other guide data may be provided to user equipment on a sideband of a television channel, using an in-band digital signal, using an out-of-band digital signal, or by any other suitable data transmission technique. Program schedule data and other media guidance data may be provided to user equipment over multiple analog or digital television channels.

[0078] In some embodiments, guidance data from media guidance data source 1118 may be provided to the user's equipment using a client-server approach. For example, the user equipment device may pull the media guidance data from a server, or the server may push the media guidance data to the user equipment device. In some embodiments, a guidance application client resident on the user's equipment may initiate a session with source 1118 and obtain the guidance data as needed, for example, when the guide data is out of date or when the user equipment device receives a request from the user to receive the data. Media guidance may be provided to the user equipment at any suitable frequency (e.g., continuously, daily, at user-defined time periods, at system-defined time periods, in response to a request from the user equipment, etc.). Media guidance data source 1118 may provide the user equipment devices 1102, 1104, and 1106 with the media guidance application itself, or software updates for the media guidance application.

[0079] In some embodiments, the media guidance data may include viewer data. For example, the viewer data may include current and / or historical user activity information (e.g., the content a user typically watches, the time of day the user watches content, whether the user interacts with social networks, the time the user interacts with social networks and posts information, the type of content the user typically watches (e.g., pay TV or free TV), mood, brain activity information, etc.). The media guidance data may also include subscription data. For example, the subscription data may identify sources or services to which a given user subscribes and / or sources or services to which a given user previously subscribed but subsequently discontinued access (e.g., whether the user subscribed to a premium channel, whether the user added a premium level of service, whether the user accelerated their Internet speed). In some embodiments, the viewer data and / or subscription data may identify a given user's patterns over a period of more than one year. The media guidance data may include a model (e.g., a survivor model) that is used to generate a score indicative of the likelihood that a given user will discontinue access to a service / source. For example, the media guidance application may process the viewer data along with the subscription data using a model to generate a value or score that indicates the likelihood that a given user will terminate access to a particular service or source. In particular, a higher score may indicate a higher level of confidence that the user will terminate access to a particular service or source. Based on the score, the media guidance application may generate promotions that encourage the user to maintain the particular service or source indicated by the score as one that the user will likely terminate access to.

[0080] The media guidance application may be, for example, a stand-alone application implemented on a user equipment device. For example, the media guidance application may be implemented as software or a set of executable instructions that may be stored in storage 1008 and executed by control circuitry 1004 of user equipment device 1000. In some embodiments, the media guidance application may be a client-server application, where only the client application resides on the user equipment device and the server application resides on a remote server. For example, the media guidance application may be implemented partially as a client application on control circuitry 1004 of user equipment device 1000 and partially on the remote server as a server application (e.g., media guidance data source 1118) running on control circuitry of the remote server. When executed by control circuitry of the remote server (such as media guidance data source 1118), the media guidance application may instruct the control circuitry to generate a guidance application display and transmit the generated display to the user equipment device. The server application may instruct control circuitry of media guidance data source 1118 to transmit data for storage on the user equipment. The client application may instruct control circuitry of the receiving user equipment to generate a guide application display.

[0081] The content and / or media guidance data delivered to user equipment devices 1102, 1104, and 1106 may be over-the-top (OTT) content. OTT content delivery allows Internet-enabled user devices, including any of the user equipment devices described above, to receive content transmitted over the Internet, including any of the content described above, in addition to content received over a cable or satellite connection. OTT content is delivered over an Internet connection provided by an Internet Service Provider (ISP), although third parties also distribute content. ISPs may not be responsible for viewing capabilities, copyright, or redistribution of content, and may only transmit IP packets provided by OTT content providers. Examples of OTT content providers include YOUTUBE®, NETFLIX, and HULU, which provide audio and video over IP packets. Youtube® is a trademark owned by Google Inc., Netflix is ​​a trademark owned by Netflix, Inc., and Hulu is a trademark owned by Hulu, LLC. An OTT content provider may additionally or alternatively provide the media guidance data described above. In addition to the content and / or media guidance data, the provider of the OTT content may distribute a media guidance application (e.g., a web-based application or a cloud-based application) or the content may be displayed by a media guidance application stored on the user equipment device.

[0082] Media guidance system 1100 is intended to illustrate several approaches or network configurations whereby user equipment devices and sources of content and guidance data may communicate with each other for purposes of accessing content and providing media guidance. The embodiments described herein may be applied in any one or some of these approaches, or in systems employing other approaches for delivering content and providing media guidance. The following four approaches provide specific illustrations of the generalized example of FIG. 11:

[0083] In one approach, user equipment devices may communicate with each other within a home network. The user equipment devices may communicate with each other directly via the short-range point-to-point communication scheme described above, via an indirect path through a hub or other similar device provided on the home network, or via communications network 1114. Multiple individuals within a household may each operate different user equipment devices on the home network. As a result, it may be desirable for various media guidance information or settings to be communicated between the different user equipment devices. For example, as described in more detail in U.S. Patent Publication No. 2005 / 0251827, filed July 11, 2005, to Ellis et al., it may be desirable for a user to maintain consistent media guidance application settings on different user equipment devices in a home network. Different types of user equipment devices in a home network may also communicate with each other and transfer content. For example, a user may transfer content from a user computer equipment to a portable video player or a portable music player.

[0084] In a second approach, a user may have multiple types of user equipment through which they access content and obtain media guidance. For example, some users may have a home network that is accessed by in-home and mobile devices. The user may control the in-home devices through a media guidance application implemented on a remote device. For example, the user may access an online media guidance application on a website through a personal computer in their office or a mobile device such as a PDA or web-enabled cell phone. The user may set various settings (e.g., recording, reminders, or other settings) on the online guide application to control the user's in-home devices. The online guide may control the user's equipment directly or by communicating with the media guide application on the user's in-home devices. Various systems and methods for communication of user equipment devices, where the user equipment devices are located remotely from one another, are discussed, for example, in U.S. Pat. No. 8,046,801, issued Oct. 25, 2011 to Ellis et al., which is incorporated herein by reference in its entirety.

[0085] In a third approach, users of user equipment devices inside and outside the home may use their media guidance application to communicate directly with content sources 1116 and access content. Specifically, within the home, users of user television equipment 1102 and user computer equipment 1104 may access the media guidance application to navigate among and locate desired content. Users may also use wireless user communications devices 1106 to access the media guidance application outside the home to navigate among and locate desired content.

[0086] In a fourth approach, user equipment devices may operate within a cloud computing environment and access cloud services. In a cloud computing environment, various types of computing services for content sharing, storage, or distribution (e.g., video sharing sites or social networking sites) are provided by a collection of network-accessible computing and storage resources, referred to as a "cloud." For example, a cloud may include a collection of server computing devices, which may be located in central or distributed locations, that provide cloud-based services to various types of users and devices, connected through a network, such as the Internet via communications network 1114. These cloud resources may include one or more content sources 1116 and one or more media guidance data sources 1118. Additionally or alternatively, the remote computing sites may include other user equipment devices, such as user television equipment 1102, user computer equipment 1104, and wireless user communications devices 1106. For example, the other user equipment devices may provide access to stored copies of videos or streamed videos. In such an embodiment, the user equipment devices may operate in a peer-to-peer manner without communicating with a central server.

[0087] The cloud provides access to services such as content storage, content sharing, or social networking services, among other examples, for user equipment devices, and access to any of the content described above. Services can be provided in the cloud, through a cloud computing service provider, or through other providers of online services. For example, cloud-based services can include content storage services, content sharing sites, social networking sites, or other services through which user-supplied content is distributed for viewing by others on connected devices. These cloud-based services may enable user equipment devices to store content in the cloud and receive content from the cloud, rather than storing content locally and accessing the locally stored content.

[0088] Users may record content using a variety of content capture devices, such as camcorders, digital cameras with video modes, audio recorders, mobile phones, and handheld computing devices. Users can upload content to a content storage service on the cloud, for example, directly, either from user computing equipment 1104 or from a wireless user communication device 1106 with content capture features. Alternatively, users can first transfer content to a user equipment device, such as user computing equipment 1104. The user equipment device that stores the content uploads the content to the cloud using a data transmission service on the communications network 1114. In some embodiments, the user equipment device itself is a cloud resource, and other user equipment devices can access the content directly from the user equipment device where the user stored the content.

[0089] Cloud resources may be accessed by a user equipment device using, for example, a web browser, a media guidance application, a desktop application, a mobile application, and / or any combination of those access applications. A user equipment device may be a cloud client that relies on cloud computing for application delivery, or the user equipment device may have some functionality without accessing cloud resources. For example, some applications that run on a user equipment device may be cloud applications, e.g., applications that are delivered as a service over the Internet, while other applications may be stored and run on the user equipment device. In some embodiments, a user device may receive content from multiple cloud resources simultaneously. For example, a user device may stream audio from one cloud resource while downloading content from a second cloud resource. Or, a user device may download content from multiple cloud resources for more efficient downloading. In some embodiments, a user equipment device may use cloud resources for processing operations, such as the processing operations performed by the processing circuitry described in connection with FIG. 10.

[0090] As referenced herein, the term "in response to" refers to "initiated as a result of." For example, a first action being performed in response to a second action may include an intervening step between the first and second action. As referenced herein, the term "in direct response to" refers to "caused by." For example, a first action being performed in direct response to a second action may not include an intervening step between the first and second action.

[0091] FIG. 12 is a flowchart of illustrative steps of a process 1200 for adding information to an augmented reality view according to some embodiments of the disclosure. Note that process 1200, or any steps thereof, may be performed on or provided by any of the devices shown in FIGS. 10-11. For example, process 1200 may be executed by control circuitry 1004 (FIG. 10) as instructed by a media guidance application implemented on a user device (e.g., user equipment devices 1102, 1104, and / or 1106 (FIG. 11)) to present the added information within an augmented reality environment. In addition, one or more steps of process 1200 may be incorporated into or combined with one or more steps of any other process or embodiment described herein.

[0092] FIG. 12 depicts an illustrative process 1200 for providing information within an augmented reality field of view. In step 1210, control circuitry 1004 for the media guidance application may identify objects within the field of view of the augmented reality device. For example, a user using an augmented reality device may have a field of view (e.g., field of view 200, 400, 500, 600, or 700) that includes various objects. The media guidance application may use the camera circuitry to capture images within the field of view. The media guidance application may also determine whether the user is focused on a particular object or direction within the field of view.

[0093] In step 1220, control circuitry 1004 for the media guidance application may process the image of the object. The media guidance application may use camera circuitry to capture and process the image with control circuitry 1004 to detect references on the object that are associated with participants of the event. The media guidance application may use image analysis tools to analyze the image with control circuitry 1004 to identify the object. The media guidance application may then search a media or other database to obtain additional information about the object and determine the references, participants, and events associated with the object.

[0094] In step 1230, control circuitry 1004 for the media guidance application may search the user's profile to find content that may be relevant to the object. In particular, control circuitry 1004 for the media guidance application may search the user profile to identify messages about participants previously transmitted by the user to a social network. Thus, the media guidance application may use participant or event information associated with the object to find messages from the user's social network that are relevant to the object in the augmented reality view. For example, the media guidance application may search the user's social network record 300 (FIG. 3) to identify messages related to the participants of the object. Building on the examples provided previously, the media guidance application may detect a sports team logo in the augmented reality device display and determine participants, e.g., players on the team, associated with the logo, sport, or other stakeholder metadata for the logo. The media guidance application may then search the user's history to find related content, e.g., social network comments, media associated with the user history, etc.

[0095] In step 1240, control circuitry 1004 for the media guidance application may overlay messages identified in the user's history within a display on the augmented reality device. For example, the media guidance application may use comments from the user's social network history that are associated with the object references to overlay within the augmented reality view. Control circuitry 1004 for the media guidance application may cause the overlay to be generated in the vicinity of the object or in another suitable location such that the user may see an association between the message and the object. The message overlay may be a combination of the content of the message and the reference object from the user's social network history. Thus, in an example, a social network message about football may be displayed within the augmented reality view as an overlay of the sports logo reference object within the augmented reality view.

[0096] It is contemplated that the steps or descriptions of FIG. 12 may be used in conjunction with any other embodiment of the present disclosure. In addition, the steps and descriptions described in connection with FIG. 12 may be performed in alternate orders or in parallel to further the objectives of the present disclosure. For example, each of these steps may be performed in any order, in parallel, or substantially simultaneously to reduce delays or increase speed of the system or method. Any of these steps may also be skipped or omitted from the process. Additionally, it is noted that any of the devices or equipment discussed in connection with FIGS. 10-11 may be used to perform one or more of the steps of FIG. 12.

[0097] FIG. 13 is a flowchart of illustrative steps of a process 1300 for adding information to an augmented reality view according to some embodiments of the disclosure. Note that process 1300, or any steps thereof, may be performed on or provided by any of the devices shown in FIGS. 10-11. For example, process 1300 may be executed by control circuitry 1004 (FIG. 10) as instructed by a media guidance application implemented on a user device (e.g., user equipment devices 1102, 1104, and / or 1106 (FIG. 11)) to present the added information within an augmented reality environment. In addition, one or more steps of process 1300 may be incorporated into or combined with one or more steps of any other process or embodiment described herein.

[0098] FIG. 13 is an illustrative flowchart depicting a process 1300 that may be followed to provide information in an augmented reality view including a message with a reference object in the view. In step 1310, control circuitry 1004 for the media guidance application may be used to position the first user's augmented reality device over an object in the field of view of the augmented reality device. The media guidance application may focus a view of the display in the augmented reality device over the object in the field of view. For example, in field of view 200 shown in FIG. 2, the media guidance application may focus on one or more objects in field of view 200. For example, the media guidance application may focus on an advertisement in a billboard, a nearby store, or one of the individuals in field of view 200, such as individual 210. Focusing on an object in the field of view may be accomplished, for example, by detecting the direction or orientation of the augmented reality device, a user history of object interest, the user's calendar to determine a time focus, or another criterion. In another example, the object focus may be based on a user's actions in the augmented reality environment.

[0099] In step 1315, control circuitry 1004 for the media guidance application may identify an object within the field of view of the first user's augmented reality device. The object may be identified by control circuitry 1004 by narrowing the focus from step 1310 within the field of view. The object may be identified using object recognition tools and techniques.

[0100] When an object is identified, control circuitry 1004 for the media guidance application may capture an image of the object in step 1320. The image may be captured by control circuitry 1004 for the media guidance application using photographic or video image capture, which may include capturing an image using a camera circuit and using pixels or another image component.

[0101] In step 1325, control circuitry 1004 for the media guidance application may search a database to identify content that matches at least a portion of the image of the object. The database search may be performed by the media guidance application using image search tools or techniques to determine content and context for the object image. The image search tools may provide metadata about the object image. Such metadata may be searched by the media guidance application in the database to find content that matches the object image. For example, an image of individual 210 ( FIG. 2 ) may be captured in step 1320, and the image may be analyzed using image search techniques to detect, for example, that the image is of an individual, that individual 210 may be recognized as one known by the user based on a comparison of face or image recognition tools and the user's touches, and in addition, the image search may detect that individual 210 is wearing shoes that were purchased by the user (based on the user's online shopping history), that individual 210 is wearing a T-shirt with a New England Patriots logo (which may be known to the media guidance application as relating to the user's favorite or opposing team), or other details about the image.

[0102] In step 1330, in response to identifying content that matches at least a portion of the image of the object, control circuitry 1004 for the media guidance application will retrieve metadata associated with the content and detect references on the object related to participants of the event. Thus, in step 1330, the media guidance application will use the information about identifying matching content for the image obtained in step 1325 and retrieve metadata about the matching content. For example, if the media guidance application detects, e.g., using a facial recognition tool, that individual 210 (FIG. 2) is known to the user, the media guidance application may retrieve metadata about the individual or contact. In another example, if the media guidance application identifies that the object image is a logo of the New England Patriots, the media guidance application will retrieve metadata about the New England Patriots, e.g., event and participant information. Such metadata and information may be obtained from a sports database or another suitable database. In another example, the media guidance application may identify that the object image includes clothing similar to something the user previously purchased or liked on social media. In this example, the media guidance application may retrieve metadata about store sales, spokespeople, pop-up shops, retailers, or other events or participants that are related to the content.

[0103] In step 1335, control circuitry 1004 for the media guidance application may search a list of messages previously transmitted by the first user to the social network and identify a given message that includes content that matches the participant. To perform step 1335, the media guidance application may access the user's social network and search for messages such as comments, tags, links, or other content posted by the user or posted by another contact and associated with or tagged for the user, as shown in record 300 (FIG. 3). The social media messages may be searched to find content related to the participant identified in step 1330. In an example, in response to detecting a New England Patriots logo on a T-shirt, the media guidance application may retrieve metadata for the New England Patriots team and search the user's social network for messages about the team. In this example, the media guidance application may detect message 310 (FIG. 3) as related to the New England Patriots, who played in the Super Bowl in February 2017. In another example, the media guidance application may detect another message 302 (FIG. 3) as relating to the New England Patriots because the team will be playing shortly after the date on which the message is posted.

[0104] In step 1340, control circuitry 1004 for the media guidance application may identify a location on the object of reference associated with the participant by determining an area of ​​the object in the field of view that includes a portion of the image that matches the identified content. To perform step 1340, the media guidance application may compare the context of the augmented reality view to the object image information to determine the location of the object in the field of view.

[0105] Control circuitry 1004 for the media guidance application may then use the augmented reality device to combine the content of the given message with the reference detected on the object at a location relative to the identified location, in step 1350. Thus, the social media message identified in step 1340 may be added to a view in the augmented reality device at a location near the object, in step 1350 to provide a view such as that shown in FIGS.

[0106] It is contemplated that the steps or descriptions of FIG. 13 may be used in conjunction with any other embodiment of the present disclosure. Additionally, the steps and descriptions described in connection with FIG. 13 may be performed in alternate orders or in parallel to further the objectives of the present disclosure. For example, each of these steps may be performed in any order, in parallel, or substantially simultaneously to reduce delays or increase speed of the system or method. Any of these steps may also be skipped or omitted from the process. Additionally, it is noted that any of the devices or equipment discussed in connection with FIGS. 10-11 may be used to perform one or more of the steps of FIG. 13.

[0107] FIG. 14 is a flowchart of illustrative steps of a process 1400 for generating options for a user to use aspects depicted in an augmented reality view, according to some embodiments of the disclosure. Note that process 1400, or any steps thereof, may be implemented on or provided by any of the devices shown in FIGS. 10-11. For example, process 1400 may be executed by control circuitry 1004 (FIG. 10) as instructed by a media guidance application implemented on a user device (e.g., user equipment devices 1102, 1104, and / or 1106 (FIG. 11)) to present an option to share added information within an augmented reality environment. In addition, one or more steps of process 1400 may be incorporated into or combined with one or more steps of any other process or embodiment described herein.

[0108] 14 depicts a process 1400 that may be used to share an augmented reality view. In particular, this may be used to share a portion of an augmented reality view that includes information added to the view by a media guidance application. Process 1400 may follow process 1300. For example, process 1400 may follow step 1350 of process 1300, where messages and objects are combined and presented in an augmented reality environment. In another example, process 1400 may be used as a stand-alone process that uses photographs obtained by other techniques.

[0109] Process 1400 begins at step 1410, where control circuitry 1004 for the media guidance application may store a photograph representing a view through the augmented reality device of the content of a given message combined with a reference detected on the object. The photograph may be captured by the media guidance application from a view in the augmented reality device that includes the message and the reference object using the camera circuitry. The photograph may be a view such as that shown in FIGS. 4-6 that includes an augmented reality view with a message based on content obtained from the user's social network. In some examples, the photograph may be a portion of the view that includes the message and the object. In other examples, the photograph may be a message combined with a portion of the reference object from the view. Other versions of the photograph may include various combinations of the message and the object. The photograph may be stored in a cache or database by the media guidance application.

[0110] At step 1420, control circuitry 1004 for the media guidance application may generate to display options for the user to transmit the photo to a social network. To implement step 1420, the media guidance application may generate a display within the augmented reality environment that uses the previously stored photo and includes an option to share the photo based on an enhanced view within the augmented reality environment. An example of such an option is depicted in FIG. 7 , where window 720 shows a T-shirt with a logo from individual 705 and includes option 730 to post or send an image and a message. Other types of interactions with the photo may also be provided in the options for the user to use.

[0111] It is contemplated that the steps or descriptions of FIG. 14 may be used in conjunction with any other embodiment of the present disclosure. Additionally, the steps and descriptions described in connection with FIG. 14 may be performed in alternate orders or in parallel to further the objectives of the present disclosure. For example, each of these steps may be performed in any order, or in parallel, or substantially simultaneously to reduce delays or increase speed of the system or method. Any of these steps may also be skipped or omitted from the process. Additionally, it is noted that any of the devices or equipment discussed in connection with FIGS. 10-11 may be used to perform one or more of the steps of FIG. 14.

[0112] FIG. 15 is a flowchart of illustrative steps of a process 1500 for selecting a message to combine with an object in an augmented reality field of view according to some embodiments of the disclosure. Note that process 1500, or any steps thereof, may be implemented on or provided by any of the devices shown in FIGS. 10-11. For example, process 1500 may be executed by control circuitry 1004 (FIG. 10) as instructed by a media guidance application implemented on a user device (e.g., user equipment devices 1102, 1104, and / or 1106 (FIG. 11)) to select a message to add to an augmented reality environment. In addition, one or more steps of process 1500 may be incorporated into or combined with one or more steps of any other process or embodiment described herein.

[0113] FIG. 15 depicts a process 1500 for selecting a message to combine with an image in an augmented reality environment. Process 1500 may be used to select a message to add to the augmented reality view, for example, the message in step 1335 (FIG. 13), or to select a message to add to a portion of the augmented reality view that has an object that has a relationship to the message in the user's social network. In an embodiment, the media guidance application may search the user's social network history, user profile, Internet cache history, or other history to identify a message or comment left by the user and match it to an object in the augmented reality view. Adding a social network message to a matched object in the augmented reality view may be interesting or relevant to the user because it is a comment or thought the user had in the past that the user felt was compelling enough to share with the user's social network. To facilitate matching messages and images within the augmented reality environment, in some embodiments, the media guidance application may build a database for a particular user or set of users of images and image objects that may be associated with metadata and links that may help facilitate searching for complementary content to add to the augmented reality environment, and messages from the user's social network that may be related to the images and image objects.

[0114] In step 1510, control circuitry 1004 for the media guidance application may receive an image of an object captured within a field of view of the augmented reality device. The image may be received from the augmented reality device using camera circuitry. In an embodiment, the image may be a field of view, such as the augmented reality field of view from FIG. 1-2 or 4-7.

[0115] In step 1520, control circuitry 1004 for the media guidance application may capture the received image and store the image in a database in association with a number of entries associating the image with content. For example, the image may be stored in a relational database that includes links and metadata for the image. The image may be analyzed and broken down into components or elements, and links and metadata for each component or element of the image may also be stored in the database. For example, the media guidance application may store an image taken from an augmented reality device and analyze the image for objects that may be identified using image recognition or image processing tools to detect and then define elements in the image.

[0116] In step 1530, control circuitry 1004 for the media guidance application may detect multiple distinct items in the image of the object. The distinct items may be detected using image processing tools that may focus on elements in the image and match them with images in the image database. In some embodiments, image processing may include comparing pixels in the image and identifying boundaries for the object that may be matched with objects in the image database.

[0117] At step 1540, the media guidance application may identify one of a plurality of entries that includes a given image that substantially corresponds to a given one of the distinct items. To perform step 1540, the media guidance application may search a media database, an image database, or another database to find an entry for an image that is substantially identical to some element or component of the received image. The search may be performed using image processing and recognition tools such that the image element may be compared to images in the database. The database may have metadata and links for image items in the database. For example, an image database may have associated source information, descriptive data, titles, keywords, or other metadata associated with the images to facilitate retrieving the content.

[0118] At step 1550, the media guidance application may retrieve content associated with the given image matched from the database at step 1540. The content may be obtained, for example, by accessing metadata for the image and using the link information to retrieve the content.

[0119] Metadata for the content may be analyzed in step 1560 by control circuitry 1004 for the media guidance application to determine whether the content is associated with an entity. For example, the media guidance application may determine whether the content is related to a group, an organization, a team, or another entity. The entity information may be useful to obtain messages to include with content that is relevant to the entity. For example, if the media guidance application retrieves an image that includes a logo for the New England Patriots, the matching image may be associated with content for football games, player news, team articles, team merchandise, NFL statistics, or other content. The media guidance application may detect that the content is associated with an entity, such as the New England Patriots, football, NFL, or another entity. If the media guidance application determines in step 1560 that the content is associated with an entity, the media guidance application may search for messages in the user's social network that are associated with the entity in step 1565.

[0120] In step 1565, control circuitry 1004 for the media guidance application may select a message based on the entity. The message may be selected by the media guidance application from the user's social network history, such as record 300 (FIG. 3), or another user history or user profile.

[0121] However, if the media guidance application does not find an entity associated with the content, the media guidance application may continue with process 1500 at step 1570 and determine whether the content is associated with a time. For example, the media guidance application may find an event schedule for a matching image. For example, the image may be of a celebrity who may have a concert schedule, or a part of a movie with a specific release schedule and airtime in a theater, or an NFL logo of a team with a certain game schedule, etc. The event information and associated time can be used as criteria to find a suitable message from the user's social network to include within the augmented reality environment. In some scenarios, step 1570 may also be performed if the media guidance application is unable to find a suitable message based on the entity in step 1565.

[0122] If a time-based factor is determined in step 1570, the media guidance application may select messages from the user's social network record, such as record 300 (FIG. 3), and find messages that are relevant to the time.

[0123] If a time-based element is not found by the media guidance application in step 1570, control circuitry 1004 for the media guidance application may not select a message using the time element in step 1580. In such a scenario, the media guidance application may select a message from the social network using other criteria, such as keyword matches based on metadata for matching images, or some combination of the techniques described herein.

[0124] It is contemplated that the steps or descriptions of FIG. 15 may be used in conjunction with any other embodiment of the present disclosure. In addition, the steps and descriptions described in connection with FIG. 15 may be performed in alternate orders or in parallel to further the objectives of the present disclosure. For example, each of these steps may be performed in any order, in parallel, or substantially simultaneously to reduce delays or increase speed of the system or method. Any of these steps may also be skipped or omitted from the process. Additionally, it is noted that any of the devices or equipment discussed in connection with FIGS. 10-11 may be used to perform one or more of the steps of FIG. 15.

[0125] The processes discussed above are intended to be illustrative, not limiting. Those skilled in the art will understand that the steps of the processes discussed herein may be omitted, modified, combined, and / or rearranged, and any additional steps may be implemented, without departing from the scope of the invention. More generally, the above disclosure is illustrative and not intended to be limiting. Only the following claims are intended to set boundaries as to what the invention encompasses. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and that the flow charts or examples of one embodiment may be combined with any other embodiment in a suitable manner, performed in a different order, or performed in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and / or methods described above may be applied to or used in accordance with other systems and / or methods.

Claims

1. A method of operating a system, the system comprising a circuit, the method comprising: generating an augmented reality view for display on a display device associated with the user account, the circuitry detecting an object within the augmented reality field of view; the circuitry identifying social network content associated with metadata of the detected object posted in a social network in association with the user account; the circuitry modifying the display of the augmented reality view to include the identified social network content; A method comprising:

2. The method of claim 1, wherein the social network content posted on the social network in connection with the user account corresponds to a posting on the social network by a device associated with the user account.

3. The method of claim 1, wherein the social network content posted on the social network in connection with the user account corresponds to a post in a social network feed by a contact associated with the user account.

4. The method of claim 1, wherein the identified social network content includes at least one of messages related to the object, video clips related to the object, television programs related to the object, images related to the object, books related to the object, other media content related to the object, information about an event related to the object, or information about a participant related to the object obtained from the social network content.

5. The method of claim 1, wherein the modifying includes the circuitry overlaying the identified social network content on a portion of the augmented reality field of view and / or the circuitry combining the identified social network content with a portion of the augmented reality field of view.

6. The method of claim 1, comprising: the circuit determining an area of ​​focus in the augmented reality field of view; and the circuit performing the generating, detecting, identifying, and modifying of the object located within the area of ​​focus.

7. The method of claim 1, wherein the circuitry determines a location for placement of the identified social network content, and the circuitry places the identified social network content based on the location.

8. The method of claim 1, further comprising: the circuit determining an event associated with the object; and the circuit determining a specific time or a specific order for the modifications based on the determined event.

9. The method of claim 1, further comprising, after the modifying, the circuitry generating for display selectable options for at least one of capturing the modified augmented reality view including the identified social network content, consuming the identified social network content, or accessing content related to the object.

10. A system comprising a circuit, The circuit comprises: generating an augmented reality view for display on a display device associated with the user account; Detecting an object within the augmented reality field of view; and identifying social network content associated with metadata of the detected object that has been posted on a social network in association with the user account; modifying the display of the augmented reality view to include the identified social network content; A system configured to:

Images