Content Set Navigation and Automatic Forwarding

Abstract

Describes embodiments for communicating and displaying a collection of image and video clip content. In one example embodiment, the device receives interface information about a set of content collections from a server computer system. When the user inputs a selection of a first content collection, the device displays the images and video clips in the order defined by the content collection. Each content segment (e.g., an image or a video clip) is displayed for less than a threshold display time. When the device finishes playing the first content collection, the device automatically starts playing the next content collection. Additional content collections generated from content submitted by other client devices can be received from the server computer system and played continuously and automatically indefinitely. Some embodiments include content collections generated by the server computer system, as well as advertising elements or other system images presented between the content collections.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201780021118.4. The filing date of the original application is March 29, 2017, the priority date is March 29, 2016, the date of entry into the Chinese national phase is September 28, 2018, and the invention title is "Content Collection Navigation and Automatic Forwarding".

[0002] Claim for Priority

[0003] This patent application claims priority to U.S. Patent Application No. 15 / 083,976, filed on March 29, 2016, which is hereby incorporated by reference in its entirety. Background Art

[0004] Content such as news stories or other collections of real-time or recent content has traditionally been presented to consumers in a tightly controlled and curated format. Early news reporting formats included newspapers and magazines. Later formats included radio and television news. Traditional media and news sources for time-sensitive content have generally been closely associated with companies or well-known individuals that collect and present information about current events and happenings. In the modern Internet era, many of these news sources have fragmented, but the core aspects of information collection and presentation generally still involve professionals collecting and sharing information in a manner related to their personal identities. While these practices have been able to support some news structures with valuable analysis, the story generation process of having professionals filter information and generate stories is very time-consuming and introduces a significant delay between the occurrence of an event and the presentation of information to news consumers. Brief Description of the Drawings

[0005] The various drawings in the figures only illustrate example embodiments of the present disclosure and should not be considered as limiting its scope.

[0006] Figure 1 is a block diagram showing a networked system that can be used to implement some example embodiments.

[0007] Figure 2A shows aspects of a server system operation for receiving content from different geographical regions according to certain example embodiments.

[0008] Figure 2B shows aspects of a server system operation for distributing different content collections according to certain example embodiments.

[0009] Figure 3A shows an example user interface for presenting a content collection on a mobile device according to some example embodiments.

[0010] Figure 3B shows aspects of the automatic forwarding of a content collection according to some example embodiments.

[0011] Figure 3C Illustrates aspects of content set presentation and navigation on a mobile device according to some example embodiments.

[0012] Figure 3D Illustrates aspects of content set presentation and navigation on a mobile device according to some example embodiments.

[0013] Figure 4 Illustrates aspects of content messages and associated content according to some example embodiments.

[0014] Figure 5 Illustrates aspects of a server system for communication of content messages, generation of content sets, and selection of content sets for automatic forwarding according to some example embodiments.

[0015] Figure 6 Illustrates aspects of a communication system according to some example embodiments, the communication system including a device communicating with a server system to communicate content messages, generate content sets, and select content sets for automatic forwarding.

[0016] Figure 7 Illustrates a method according to some example embodiments.

[0017] Figure 8 Illustrates a method according to some example embodiments.

[0018] Figure 9 Is a block diagram illustrating an example of a software architecture that may be installed on a machine according to some example embodiments.

[0019] Figure 10 Illustrates a graphical representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methods discussed herein. Detailed Description

[0020] The systems and methods described herein relate to automatic presentation of content sets, including processes for generating, selecting, and continuously presenting content sets to a user.

[0021] As described herein, "content" refers to one or more images or video clips collected by an electronic device, as well as any associated metadata descriptions and graphics or animations added to the image or video clip. This includes metadata generated by the electronic device that captures the image or video, as well as metadata that can later be associated by other devices. A "piece of content" refers to a single image or video segment collected by a client device, where any changes are made to the image or video clip (e.g., transformation, filter, adding text, etc.). Each piece of content can have multimedia elements, including graphics, text, animations, emojis, or other such elements added along with the image or video clip elements. The content collected by the image sensor of the client device can be sent via a network to other client devices that are part of a social sharing network, along with any added multimedia elements from the user. Each piece of content can have a time limit or an associated display time, which is within a display threshold set by the system. For example, an embodiment system can limit a video clip to less than 10 seconds and can allow the user to select a display time of less than 10 seconds for image content.

[0022] The "content message" mentioned herein refers to the communication of content between one or more users via the system. Content can also be sent from a client device to a server system for general sharing with other system users. Some embodiments limit the content message to an image or video clip captured using an interface, which does not allow the content to be stored and sent later, but rather uses an associated content message with a single piece of content and any added multimedia that is sent before any other operations are performed on the device. The embodiments described herein relate to methods of grouping such content into content collections (e.g., stories). In various systems, a content message can be sent from a single user to another single user, as, for example, an ephemeral message, and can also be sent to a server computer system for inclusion in various content collections.

[0023] A "content collection" as described herein is an ordered group of content. The individual content segments that make up a particular content collection can be related in a variety of different ways. For example, in some embodiments, a content collection includes all content segments marked as public that have been sent to a server system from a particular user within a specific time range (e.g., within the past 24 hours). Access to such a content collection may be limited to certain other users (e.g., friends) identified by the user who generated the content of the collection. In some other embodiments, a content collection includes content segments from different users that are related by time, location, content, or other metadata. In some embodiments, a content collection is referred to as a story. As described in more detail in this document, a story or content collection can be generated from content segments that are related in a variety of different ways.

[0024] For example, two user devices can each be associated with an account, and the accounts can be identified as connected in a communication system. Throughout a given day, one user captures images or video clips (e.g., content segments) and sends a content message for each content segment to the communication system. Some of these content segments are identified as being included in a semi-private content collection associated with the sending device. Some of the content segments are identified as being available for a public content collection.

[0025] Because the two users are connected, the second user is able to access the semi-private content collection associated with the first user's account, which contains only content segments generated by the first user's device. When the second user's device is connected to the communication system, the second user's device receives interface information indicating that the first user has a content collection of updated content segments. The interface information sent to the second user's device also includes an indication of content collections of other users that have been updated, as long as the second user is associated with or otherwise authorized to view the content collections. Additionally, the interface information can include details of content collections created from content messages sent by any other system user. These real-time or location-based content collections can be curated or automatically generated by the communication system.

[0026] When the second user selects any one of the content sets described in the interface information, the device of the second user begins to sequentially display each content segment from the selected set. Each content segment is associated with a display time that is less than a threshold (e.g., less than 10 seconds). When the device reaches the end of the selected content set, the device automatically forwards to the next content set instead of stopping the display of content. The communication system can determine which content set to play next in a variety of different ways. In some embodiments, a content set is first selected from the associated user content sets that have the most recently unseen updates. After all content sets from these friend accounts have been displayed, public content sets can be selected based on proximity, time, attention, or other such metrics. In other embodiments, other user selections or preferences can be used to select the next content set.

[0027] In some embodiments, the communication system receives enough content from enough users such that the system can generate and display content sets indefinitely. Additionally, some systems can allow for the automatic insertion of advertising elements between some or all content sets. Further, some systems allow users to jump to the next or previous content segment or the next or previous content set, and also display content segments and content sets in a progressive manner, automatically advancing to the next content segment and then over time to the next content set. Additional embodiments and details are described below.

[0028] Figure 1is a block diagram showing a networking system 100 according to some example embodiments. The system 100 includes client devices 110, client device 120, a server system 150, and a network 140 for communicating communications between client devices 110 and 120 and the server system 150. The client devices 110 and 120 can be any smart phone, tablet computer, phablet, laptop computer, network-enabled camera, or any other such network-enabled device. The client devices 110, 120 can include a camera device for capturing content, or can be coupled to a separate camera device for capturing content before being sent to other client devices 110, 120 for storage. Thus, some embodiments can include wearable devices, such as a pendant having an integrated camera coupled to the client devices 110, 120. Other embodiments can include other associated devices having an integrated camera that can be worn, such as a watch, glasses, clothing such as a hat or coat having integrated electronics, a clip-on electronic device, or any other such device that can communicate with or be integrated with the client devices 110, 120. The client devices 110 and 120 are connected to the server system 150 via the network 140. The network 140 can include any combination of wired and wireless connections. This can include a cellular access network, an access point interface to the Internet, or any other such network 140 or network element. For example, the client device 110 can interface with the network 140 using a Long Term Evolution (LTE) cellular network to communicate with the server system 150, while the client device 120 can use a Wi-Fi access point to interface with the network 140 and communicate with the server system 150. The server system 150 can be one or more computing devices, as part of a service or network computing system. In certain embodiments, particularly those having a large number of client devices 110, 120 interfacing with the server system 150 from widely different locations around the globe, the server system 150 can be a distributed network 140 of server computers that are similarly widely distributed and communicate with each other via the network 140. In some embodiments, any elements of the client devices 110 and 120, and the network 140 and the server system 150 can use Figure 9 and Figure 10 the elements of the software architecture 702 or the machine 1000 described in

[0029] Then, the networked system 100 can be used for the communication of content messages from the client devices 110, 120 to the system 150 and the communication of content collections from the system 150 to the client devices 110, 120. As shown in FIG. 1, the client device 110 communicates a content message 112 to the server system 150, and the client device 110 receives a content collection 114 from the server system 150. In addition to this function for the embodiments described herein, the client device 110 can additionally receive private content segments and communications from other users and can communicate a personal content collection to the server system 150, where the personal content collection includes images and / or videos from the content message 112 generated by the client device 110 or another device coupled to the client device 110. Similarly, the client device 120 sends a content message 122 and receives a content collection 124 and can additionally perform other actions.

[0030] Figure 2A Shows aspects of a server system 250 receiving content messages from different geographical regions according to certain example embodiments. Figure 2B Shows aspects of a server system 250 sending different real-time content collections to different geographical regions. Contrasted with showing two client devices 110 and 120 Figure 1 Instead, Figure 2A -B shows an abstract form of the client side of a system in which thousands or millions of client devices 110, 120 in different regions can interact with the server system 250.

[0031] Instead of separate client devices 110, 120, Figure 2A and Figure 2B Shows a simple user segment representation having two local geographical regions 204 and 206 (which are the lowest level regions in this example). In certain embodiments, geofencing is used to define local areas. Such geofencing can be tracked by aspects of the networked system 100, which includes a positioning system within client devices (such as client devices 110 and 120), a network-based positioning system as part of the network 140, a separate positioning system such as the Global Positioning System (GPS), or any combination of these or other positioning systems. In various embodiments, real-time or public content collections (e.g., a first content collection group 292 or a second content collection group 294) can be made available to a particular device based on the location of the device and the location associated with a particular content collection.

[0032] In other embodiments, the system can generate content collections for each client device 110, 120 individually, rather than considering setting up geofences or user groups. In such embodiments, whenever a user navigates to a content collection interface within an application operating on client devices 110, 120 (e.g., as shown in Figure 3A ), client devices 110, 120 communicate their current location to the server system 250. The location of the device providing the information at this time, or the location of other devices, can be used to generate a list of content collections for the device.

[0033] In Figure 2A the example shown, client devices 110, 120 within the first local geographic area 204 are grouped together and communicate 1000 content messages 260 to the server system 250 over a first time period. The content associated with these content messages 260 is shown as SF1 to SF1000. During the same time period, 10,000 content messages 262 containing individual video clips or images are sent by client devices 110, 120 within the second local geographic area 206 to the server system 250, as shown by content LA1 to LA10000. These are public content communications, the content of which can be used to communicate with any user. The system can additionally include content for content collections restricted to "friend" accounts, as well as content sent as a message to a specific user, which will be deleted as a short message and not shared with any other user other than the intended recipient. The above-mentioned volume of public content is sufficient to overwhelm individual users. Therefore, the server system 250 operates as a curator to filter content messages 260, 262 and provide a selected set of pictures and videos from content messages 260, 262 as one or more content collections.

[0034] In various embodiments described below, the curation can be performed by a system operator using curation tools or can be performed by automated analysis and selection performed by the communication system. An example embodiment divides users by local region. The content sets of client devices 110, 120 are generated from the most recent content messages 260, 262 generated within the current local region of client devices 110, 120. Such local content messages 260, 262 for the content sets can be further filtered based on image quality and image content. Image content can be used to prevent excessive content duplication, provide a variety of different content, provide content identified as newsworthy (e.g., images associated with celebrities), or be selected based on any other such content filtering. Image content can also be analyzed to identify content duplication and avoid placing extremely similar content (e.g., videos of the same event from similar angles) in a single content set. Additionally, the server system 250 can analyze trends associated with incoming content messages 260, 262 from other local regions to generate content sets based on trends identified by the system. Additional details regarding curation and content set generation will be discussed below with reference to Figure 5 be discussed.

[0035] Figure 2B Then it is shown that the first content set group 292 is available to all client devices 110, 120 within the first local geographic region 204. Similarly, the second content set group 294 includes content sets visible to all client devices 110, 120 within the second local geographic region 206. The second content set group 294 is shown as including three content sets, where all three content sets are generated from content messages 260, 262 originating from the second local geographic region 206. These content sets of the second content set group 294 include LA content sets 291 - 293. The first content set group 292 is shown as including two content sets generated from content messages 260, 262 originating from the local geographic region 204, SF content set 281 and SF content set 282. The first content set group 292 also includes a content set generated from content messages 260, 262 originating from the local geographic region 206, LA content set 291. As described above, the LA content set 291 can be identified by the server system 250 analyzing system trends, where a greater than normal number of content set views, screenshots, incoming additional content messages 260, 262, or other system trends identify the LA content set 291 as a content set to be made visible to a larger user segment.

[0036] Figure 3AAn embodiment of the user interface of the client device 300 is shown. The client device 300 shows user-selectable interface areas for each content collection. This includes an optional common real-time content collection in section 303 and a semi-private content collection from a friend's account in section 305. Thus, the content collection in section 305 may include content from a device associated with a single other account. The content in these content collections 330, 331 may include content within a most recent time frame (such as the past 24 hours or a week), or may simply include previously unseen content. In other embodiments, the content may be restricted for the account-based "My Story" or account-based content collections in any other manner described herein. In Figure 3A the example, the client device 300 is associated with two other accounts having semi-private content collections 330 and 331 (with unseen recent updates), and the client device 300 is in a first local geographic region 204 and can thus access content collections in a first content collection group 292, the content collections in the first content collection group 292 including content collection 281, content collection 282, and content collection 291. Additional content collections may be provided by scrolling left or right or up and down. Each interface area may provide basic details or sample images associated with each content collection. These details are received at the client device 300 as interface information for the available content collections. In some embodiments, a content collection or a portion of a content collection may be provided to the client device 300 prior to selection of the interface area. In other embodiments, after selection of a particular interface area, an image of the content collection is communicated from a server system such as server system 250.

[0037] The user may physically touch the client device 300, and in response to the touch, the client device 300 may determine tactile data such as touch location, touch force, or gesture movement. In various example embodiments, the client device 300 displays a home screen that is operable to launch an application in a content viewing area 397 or otherwise manage various aspects of the client device 300. In some example embodiments, the home screen provides status information such as battery life, connectivity, or other hardware status. The user may activate a user interface element by touching the area occupied by the corresponding user interface element. In this way, the user interacts with applications of the client device 300, including applications that implement content collection display and automatic forwarding as described herein.

[0038] In some embodiments, the content may be presented within the touch screen of the client device 300 operating as client devices 110, 120, and as described herein, touch input may be used to initiate communication to a server system 150 to request content collections and automatically forward a set of consecutive content collections.

[0039] Many types of applications 910 (also referred to as “application software”) can be executed on the machine 1000, such as native applications (e.g., applications programmed in Objective-C, Swift, or other suitable languages running on IOS TM or applications programmed in Java running on ANDROID TM ), mobile web applications (e.g., applications written in HyperText Markup Language - 5 (HTML5)), or hybrid applications (e.g., native shell applications that initiate HTML5 sessions). For example, the client device 300 includes messaging application software, audio recording application software, camera application software, e - book reader application software, media application software, fitness application software, file management application software, location application software, browser application software, settings application software, contacts application software, phone call application software, or other application software (e.g., game application software, social network application software, biometric monitoring application software). In another example, the client device 300 includes social messaging application software, such as which, in accordance with some embodiments, allows a user to send public content to the server system 250 and receive a set of content from the server system 250. Such a SNAPCHAT application can also implement the exchange of private short - lived image and / or video content messages, in addition to the communication of content messages 260, 262, to be included in a story and for display on the client device 300 in the form of a received set of content.

[0040] Figure 3C and Figure 3D illustrates an embodiment of an interface for viewing a set of content (such as Figure 3B the set of content shown in) after a touch selection of the set of content 291 in the interface of the client device 300. For example, as Figure 3B shown, when the user selects a portion of the segment 303 associated with the set of content 291, the individual content segments within the set are displayed in sequence, as shown by content segments LA7, LA55, and LA 986 - 989. Each content segment has an associated display time that is below a display threshold time, and each content segment is presented in sequence within the content viewing area 397 up to the associated display time of that content segment. In Figure 3AIn the example of -B, the content set 310 with the most recent content update is displayed at the top of the content set list in segment 305 and is the first automatically selected content set after the display of the content set 291 selected by the user input ends. After the display of the content set 291 ends, each content segment 50-57 of the content set 310 is played in sequence, and the system automatically selects the next content set 320 in the list based on the user. After the content segments 80-84 of the content set 320 are displayed, the system selects another content set. If there are no other content sets presented by the user selection left, the client device 300 will receive a common content set from the server computer system. This content set can be a curated set by the system operator or a content set automatically generated from all the content available to the server system 250. When the display of the newly received content set is completed, the client device 300 continues to receive new content sets until the user input stops the command to present new content.

[0041] In some embodiments, additional elements are inserted between the end of one content set and the beginning of the next content set. For example, in some embodiments, a set end graphic is presented after the content LA989 has been displayed for the associated display time and before the content 50 is displayed. In some embodiments, the element is an advertisement element, which can include an image, text, video, or multiple instances of any such element presented as an advertisement. In other embodiments, such advertisement elements are presented between each story. In some embodiments, the advertisement element is presented between stories after a threshold time period, after a set number of content sets, or based on any combination of such elements. As described herein, multiple advertisement elements or multiple sets of advertisement elements can be presented between content sets on the user device. In some embodiments, a "sponsored" content set is content from an advertising account. The system can periodically select these sponsored content sets based on advertising metrics within the system. In some embodiments, such sponsored content sets are selected in a manner similar to the selection of advertisement elements discussed above, and such sponsored content sets can include content segments from the device, as well as graphic, video, or image content generated using an advertising tool and submitted to the content database as sponsored content.

[0042] Thus, as described herein, a user accesses a content collection via an interface of a mobile device. The interface can include information about the available content collections, and the interface can sort the content collections based on details of the content within the collections. For example, in one embodiment, a user can have an account associated with a plurality of other accounts (e.g., friends). Each friend account can have an associated content collection that includes content segments sent to the server system 250. In one embodiment, the content collection of each friend account includes content segments selected for a personal story and sent to the server computer system within the past 24 hours, and the older content segments are deleted from the content collection. Additionally, in some embodiments, the available content collections include content collections associated with a public event or location. Each of these content collections can be presented and sorted in a user interface on the user's mobile device. Further, a "real-time" content collection can be available, which presents a curated or automatically generated content collection filtered from a large amount of content sent to the system.

[0043] In the embodiments described herein, selecting an individual content collection from a set of presented content collections can then result in the sequential and continuous presentation of additional content collections or stories. This sequential presentation occurs as part of an automatic forwarding operation on the mobile device that displays subsequent content collections after the display of the initially selected content collection by the user has ended.

[0044] For example, in one embodiment, a user can select an interface associated with a first content collection from a first friend. The exemplary content collection can have six content segments, each ten seconds in length. After sixty seconds, assuming no further input from the user, the device will automatically forward to another content collection. In some embodiments, the selection of the next content collection can be based on the content collection source. For example, some embodiments automatically forward the next content collection from a friend account based on which friend account has a story with a recently added content segment that has not been viewed. After all content collections with previously unviewed content have been displayed, content collections from local events can be automatically played similarly based on the most recently unviewed updates. After all local content collections with unviewed content segments have been displayed, other content collections are automatically displayed based on system settings. The content collections can be selected based on distance, time, interest, user preferences, or any other such system information. In a system that receives a sufficient number of content messages 260, 262 available for a story, the mobile device can continue to automatically forward to present additional stories indefinitely, where time, location, quality, attention, or other filtering factors are relaxed as needed to provide additional content collections.

[0045] Additionally, in some embodiments, advertisements or other segmented elements may be displayed between some or all of the content collections. For example, in one embodiment, after the last content segment of each content collection is displayed, a standard collection end graphic may be displayed for a set period of time. In other embodiments, one or more advertisement images or video clips (e.g., advertisement elements) may be displayed for a set period of time before automatically forwarding to display the next content collection. In some embodiments, both the collection end graphic and the advertisement elements are used.

[0046] As described above, then, the system can receive content messages 260, 262 from a variety of users and use the content segments from these messages to generate different types of content collections or stories. Users can access these stories via the device, and the system can automatically forward through an ongoing sequence of content collections, where the next content collection is selected in a variety of different ways.

[0047] In some embodiments, in addition to automatically presenting content collections at preselected times, the user interface also allows the user to navigate content segments within and between content collections. Figure 3C A content viewing area 397 is shown, as well as inputs 398 and 399, which are selectable areas on the touchscreen of the client device 300. In Figure 3C In the interface shown, the content viewing area 397 is displaying a content segment 82 from the content collection 320. The content collections 291 and 310 have been viewed previously. Input 398 provides a selectable interface to return to display the content of the content collection 291, and input 399 provides a selectable interface to return to display the content of the content collection 310. When the system automatically forwards to present content from the next content collection, the interface data for the previously viewed content collections can move to inputs 398, 399. In one embodiment, for example, after the content collection 320 is completed, input 399 will show the interface information for the content collection 320, the interface information for the content collection 310 will move to the input area 398, and the content collection 291 will no longer have a user-selectable interface for directly jumping to the content of that content collection.

[0048] Figure 3D Another example embodiment showing aspects of user input for navigating content collections. In Figure 3DIn an embodiment, before the end of the content display period, tapping the right side of the touchscreen display advances to the next content segment. Tapping the left side of the display causes the content segment that was just before the currently displayed content segment to be displayed again. Thus, this tapping can allow the user to navigate forward and backward through the respective content segments. Similarly, swiping input 393 from left to right can move to the first content segment of the content set that was presented just before the current content set, and swiping input 391 from right to left can cause the beginning of the next content set to be displayed. When a content segment is displayed after a user navigation input, the display time of each content segment is used to automatically advance between content segments and then advance to a new content set after the last content segment has been displayed. Swiping input 392 upward can return to Figure 3A the content set selection interface, and swiping input 392 downward can provide a navigation interface to view similar content segments or receive additional details related to the current content segment or element displayed in the content viewing area 397.

[0049] Figure 4 Then an example of content messages 260, 262 is described. In some embodiments, the content messages 260, 262 are used to send content from the client device 300 to the server computer system, such as communicating content message 112 from the client device 110 to the server system 150. In some embodiments, the content of a particular message 400 is used to populate a message table stored in a database that is accessible by the server computer system. Similarly, the content of message 400 is stored in memory as data "in transit" or "in flight" of the client device 110 or the server computer system.

[0050] Message 400 is shown as including the following components:

[0051] · Message identifier 402: A unique identifier that identifies message 400.

[0052] · Message text payload 404: Text generated by the user via the user interface of the client device 110 and included in message 400.

[0053] · Message image payload 406: Image data acquired by the camera component of the client device 110 or obtained from the memory of the client device 110 and included in message 400.

[0054] · Message video payload 408: Video data acquired by the camera component or obtained from the memory component of the client device 110 and included in message 400.

[0055] · Message audio payload 410: Audio data that is collected by a microphone or obtained from a memory component of the client device 110 and included in the message 400.

[0056] · Message annotation 412: Annotation data (e.g., filters, tags, or other enhancements) representing an annotation to be applied to the message image payload 406, message video payload 408, or message audio payload 410 of the message 400.

[0057] · Message duration parameter 414: A parameter value indicating the amount of time, in seconds, that the message content (e.g., message image payload 406, message video payload 408, message audio payload 410) will be presented or accessible to a user via a message client application operating on any of the devices described herein.

[0058] · Message geolocation parameter 416: Geolocation data (e.g., latitude and longitude coordinates) associated with the content payload of the message. Multiple message geolocation parameter 416 values may be included in the payload, each of these parameter values being associated with a content item included in the content (e.g., a specific image within the message image payload 406, or a specific video within the message video payload 408).

[0059] · Message story identifier 418: An identifier value that identifies one or more content collections (e.g., “stories”) to which a specific content item within the message image payload 406 of the message 400 is associated. For example, multiple images within the message image payload 406 may each be associated with multiple content collections using the identifier value.

[0060] · Message tag 420: Each message 400 may be tagged with multiple tags, each tag indicating a theme of the content included in the message payload. For example, in the case where a specific image included in the message image payload 406 depicts an animal (e.g., a lion), a tag value may be included within the message tag 420 indicating the relevant animal. Tag values may be generated manually based on user input, or may be generated automatically using, for example, image recognition. Tag values may additionally be used to identify whether content from the message is to be included in a content collection associated with the account of the sending client device 110, which content collection has other content from that account. Tag values may also identify that the content of the message is available for public dissemination, including inclusion in a public content collection having content from devices associated with other accounts.

[0061] · Message sender identifier 422: An identifier that indicates a user of the client device 110 (e.g., a messaging system identifier, an email address, or a device identifier) on which the message 400 is generated and from which the message 400 is sent.

[0062] · Message recipient identifier 424: An identifier that indicates a user of the client device 110 to which the message 400 is addressed (e.g., a messaging system identifier, an email address, or a device identifier).

[0063] The content (e.g., values) of the various components of the message 400 can be pointers to locations in a table in which the content data values are stored. For example, the image value in the message image payload 406 can be a pointer to a location (or its address) within an image table. Similarly, the values within the message video payload 408 can point to data stored within a video table, the values stored within the message annotation 412 can point to data stored within an annotation table, the values stored within the message story identifier 418 can point to data stored within a content collection table, and the values stored within the message sender identifier 422 and the message recipient identifier 424 can point to user records stored within an entity table. In some embodiments, the message recipient identifier 424 can be used to associate the content of a message with a particular content collection or to make the content available for curating a story. For example, one message recipient identifier 424 can be associated with a user's personal content collection that is only viewable by friends. Another message recipient identifier 424 can be for a content pool presented to system curators who use the submitted content messages 260, 262 to generate a public content collection.

[0064] In some embodiments, data from such content messages 260, 262 and other sources (e.g., curation tools, advertising tools, etc.) are stored in a database of the server computer system. While the content of the database is discussed as multiple tables, it can be understood that the data can be stored in other types of data structures (e.g., as an object-oriented database).

[0065] An example database includes message data stored within a message table. An entity table stores entity data, including entity diagrams. Entities for which records are maintained within the entity table can include individuals, corporate entities, organizations, objects, locations, events, etc. Regardless of type, any entity for which the messaging server system 250 stores data related to can be an identified entity. A unique identifier and an entity type identifier are provided for each entity.

[0066] The entity graph further stores information about the relationships and associations between entities. By way of example only, the relationships can be social, professional (e.g., working in a common company or organization), interest-based, or activity-based relationships. For example, these relationships can provide access to semi-private content collections 330, 331, such that interface information for the content collections is sent to the user device and, if a relationship is established between users, the user device is allowed to select and view the associated content of another user.

[0067] In some embodiments, the database also stores annotation data in the form of examples of filters in an annotation table. The filters stored within the annotation table are associated with videos (whose data is stored in a video table) and / or images (whose data is stored in an image table) and are applied to the video or image. In one example, a filter is an overlay that is displayed over an image or video during presentation to a recipient user. Filters can be of various types, including filters selected by a user from a filter library that is presented to the sending user by a messaging client application when the sending user is composing a message. Other types of filters include geolocation filters (also known as geo-filters), which can be presented to the sending user based on a geographical location. For example, based on geolocation information determined by the GPS unit of client device 110, the messaging client application can present a geolocation filter specific to a neighborhood or a particular location within the user interface. Another type of filter is a data filter, which can be selectively presented to the sending user by the messaging client application based on other inputs 398 or information collected by client device 110 during the message creation process. Examples of data filters include the current temperature at a particular location, the current speed at which the sending user is traveling, the battery life of client device 110, or the current time.

[0068] Other annotation data that can be stored within the image table is so-called "lens" data. A "lens" can be a real-time special effect and sound that can be added to an image or video.

[0069] As described above, the video table stores video data that, in one embodiment, is associated with a message that maintains a record within the message table. Similarly, the image table stores image data associated with a message whose data is stored in the entity table. The entity table can associate various annotations from the annotation table with the various images and videos stored in the image table and the video table.

[0070] The content collection table stores data regarding a collection of messages and associated image, video, or audio data, which is compiled into a collection (e.g., a SNAPCHAT story or a gallery). The creation of a particular collection can be initiated by a particular user (e.g., each user maintaining a record in an entity table). A user can create a "personal story" or content collection in the form of a content collection created and sent / broadcast by that user. To this end, the user interface of the messaging client application can include user-selectable icons to enable the sending user to add specific content to his or her personal story.

[0071] The collection can also constitute a "Live Story", which is a collection of content from multiple users created manually, automatically, or using a combination of manual and automatic techniques. For example, such a live content collection can constitute a curated set of user submissions from various locations and events. A user whose client device 110 has location services enabled and is at a common location event at a particular time can, for example, via the user interface presentation options of the messaging client application, contribute content to a particular Live Story. The messaging client application 104 can identify the Live Story to the user based on his or her location. The end result is a "Live Story" told from a community perspective.

[0072] Another type of content collection is called a "Location Story", which enables a user (whose client device 110 is located at a specific geographical location (e.g., on a college or university campus)) to contribute to a particular collection. In some embodiments, contributions to a Location Story can use a second degree of authentication to verify that the end user belongs to a particular organization or other entity (e.g., is a student on a university campus). In various other embodiments, as described herein, other types of content collections can be identified and constructed within the database using system operator curation tools or automated computer-based analysis.

[0073] Figure 5 Aspects of a server system 550 for automatic local content collection generation and curation are shown in accordance with some example embodiments. In various embodiments, the server system 550 can be used as an implementation of the server system 150 or the server system 250. The example server system 550 includes an input and output (I / O) module 552, a content feature analysis module 554, a machine vision module 556, a content database 558, an account management module 562, an automatic content collection module 560, and a curation tool 564.

[0074] The I / O module 552 can include any hardware, firmware, or software elements required to send and receive content and content collections to and from client devices 110, 120 via network 140. The content feature analysis module 554 can include devices, processors, and software to analyze images of pictures and frames from video clips and then determine content features, including details about when and where pictures or videos were generated. In some embodiments, the content feature analysis module 554 can be implemented as multiple different modules, each analyzing different content features, including any of the content features described herein.

[0075] The machine vision module 556 describes a specific module that can be used to identify content features based on the content of one or more images in a video. The machine vision module 556 includes hardware, firmware, and / or software for analyzing and understanding content. In one embodiment, the machine vision module 556 is associated with a dictionary that includes image and video content values. In this embodiment, the machine vision module 556 can use the objects identified in the images of the content segments and the arrangement of the objects identified therein to select one or more content values from the dictionary as content features. For example, such a simple machine vision module 556 can identify a ball in an image and select the values of ball and game as content features. A more complex module can identify the type of ball as a basketball and include "basketball" as a feature value. A more complex machine vision module 556 can identify a basketball, a crowd, the color of the court, and an elevated view of the court to identify "professional basketball game" and "basketball arena" as content values of the content. The same complex machine vision module 556 can identify a basketball, a park background, and a concrete court surface and associate "amateur basketball game" and "playground basketball" as content values of the content.

[0076] These content values generated by the machine vision module 556 can then be stored in the content database 558 together with other feature values. The feature values can include: one or more content values (i.e., identifiers of the content in the content); generation time; generation period; generation location; generation area; one or more quality values, any metadata values associated with the content, identifiers of specific content segments, or any other such values. In some embodiments, a copy of the content can be stored in the content database 558 together with location information, acquisition time information, and any other such information about the content segment. In certain embodiments, the content database 558 can anonymously store details about content usage. For example, the client devices 110, 120 can communicate details about the presentation of the content and about taking screenshots of the content on the screens of the devices. Anonymous metrics about how often a content segment that is part of a content collection is viewed, how long the content is viewed, and the frequency of taking screenshots can then be measured by the server system 550 as part of the analysis by the content feature analysis module 554, where the resulting data is stored in the content database 558. In some embodiments, the content database 558 can include any content or content message information discussed above regarding Figure 4 any of the content or content message information discussed above or this content information in any database or table structure discussed above.

[0077] The account management module 562 includes application or interface functionality to enable a user to manage entity / account relationships via communication between the user device and the server system 550. The account management module 562 can also manage the content collections of individual users as described herein.

[0078] The curation tool 564 includes tools available to system operators or advertisers to generate and present content collections from the large amount of content received at the server system 550 and to be made available for inclusion in public content collections (e.g., real-time content collections, location-based content collections, content-based content collections, etc.) through user selection. Similarly, the automatic content collection generation module 560 can filter the large number of received content segments to generate content collections grouped by location, time, topic, or any other such basis. In some embodiments, elements of the automatic content collection generation module 560 are used to filter the number of content segments provided to the curation tool 564 to a smaller number (e.g., filtering 10,000 received content segments to provide 500 content segments to the curation tool 564 for review by the system operator).

[0079] In some embodiments, the automatic content set generation module 560 can then use information about content segments from the content database 558 to select specific pictures or videos for the automatically generated content set. In various embodiments, the automatic content set generation module 560 can use sophisticated scoring, weighting, and other rules in generating the content set. For example, some embodiments can operate such that all content segments meet a quality threshold, unless a trend with certain threshold characteristics is identified and all content associated with the trend is below the quality threshold. Another embodiment can weight content set generation based on the plurality of content sets currently available in the local geographical regions 204, 206. In further embodiments, any number of sophisticated rules can be applied together as part of content set generation to filter the images and videos of the content set based on time, location, content, and quality.

[0080] The content set selection and automatic forwarding module 566 is used to select one or more next content sets to be sent to the client device 110 after receiving a request input or an automatic forwarding message to select the next content set and following the selection of the initial content set displayed on the device. In some embodiments, the initial content set group is cached on the client device 110, and the module 560 only interacts with the device after presenting the locally cached content set. In other embodiments, different sets of rules are applied to different content set groups. For example, the content set selection and automatic forwarding module 566 can apply the most recent update order to select the next entity relationship content set, and apply a scoring based on the user's interests to the selection order for all other content sets. In other embodiments, other sets of selection rules are applied to the device to determine which content set will be selected as the next content set to be automatically displayed (e.g., automatically forwarded). In some embodiments, the content set selection and automatic forwarding module 566 also manages the communication of elements to be displayed between content sets (including advertisement elements or set end graphics).

[0081] In some embodiments, the quality scoring within the automatic content set generation module 560 and / or the content set selection and automatic forwarding module 566 can be used to filter or select content segments of a specific content set and to filter different content sets for presentation to the user. In some embodiments, the quality score is based on a detailed exposure analysis of sample frames in an image or video clip. For example, a histogram of luminance can be calculated, and a quality can be assigned to the image or video based on the correlation of the histogram with the quality score. The correlation can be based on a table or function associating certain histogram patterns with selected quality scores, or can be generated in any other such matter. For a video analyzing multiple sample frames, the average score of each frame can be used to select a score, the worst score of a single frame of all analyzed frames can be used, or any such combination or function of multiple scores or a selection of scores can be used.

[0082] In some embodiments, motion blur estimation of an image or a selected video clip is used as part of a quality score. For example, the motion blur estimation can be based on the calculation of energy gradients on detected edges, or other such motion estimations. For a video clip, identifying video frames with a motion blur above a threshold amount can trigger the analysis of additional sample frames to determine the extent to which the video is affected by motion blur, or to identify when camera sensor jitter affects the entire video. In certain embodiments, the system can use a threshold for video motion or “jitter” to filter out videos with camera motion or jitter above the threshold. In other embodiments, the jitter or motion score can simply modify the overall quality score. In other embodiments, both a hard threshold and an input to the overall quality score can be used.

[0083] In some embodiments, image or sample video frames can be analyzed for compression artifacts or other image processing artifacts that indicate lower image quality or errors introduced within the image due to various compression or communication issues. These artifacts can include image ringing, image contouring, staircase noise along curved edges, color separation artifacts, or block boundary artifacts. Video can be analyzed for compression artifacts based on additional video, such as block boundary artifacts associated with motion compensation or mosquito noise, which can be identified by analyzing selected frames of the video. The presence of such compression artifacts and the intensity of any identified compression artifacts can be used to modify or select the quality score of an image or video clip. In addition to such information loss associated with compression or lossy transmission, images and video frames can be analyzed for other types of noise. For example, variations in smooth or uniform regions of an image can be analyzed for noise artifacts, such as noise associated with a low-quality or malfunctioning camera sensor, a camera with low-quality or dirty optics, or any other such noise source that can degrade, corrupt, or modify the data in the image.

[0084] In some embodiments, audio data is also used for quality scoring of video clips. In this embodiment, various audio metrics, such as dynamic range, noise level, speech intelligibility, or speech recognition data, or any other such audio-based information can be used to select an audio quality score or influence the overall quality score. In some embodiments, different audio data metrics are used based on the determined audio environment. For example, a video clip with speech can be evaluated differently from a clip with music, or video clips with different music genres can be evaluated differently. Additionally, in some embodiments, audio recognition for identifying offensive audio content (e.g., taboo spoken words or explicit music lyrics) can be used for quality scoring or quality threshold flagging.

[0085] In addition to quality scores based on image quality, some scores can be based on image content. For example, as described above, image processing can be used to identify offensive content within an image or video clip, such as nudity or taboo language. In some embodiments, a preferred orientation (e.g., landscape or portrait) can be used for quality scoring. Some systems may additionally use image recognition to identify desired content. For example, in some systems, an image of an animal or an object associated with a party environment is recognized as desired. The presence of such images within a video frame or picture can be used to increase an overall quality score, or to generate a content score.

[0086] In certain embodiments, feedback or machine learning is used to select or set quality scores. Such systems can use neural networks to extract features that are identified as preferred or of interest to the system user. For example, in some embodiments, images selected by the system user to be included in one or more stories can be selected for a learning group. Some or all of the images and video frames from the learning group can have features that are extracted and analyzed using a feedforward artificial neural network such as a convolutional neural network to identify desired elements of the image, and to automatically assign an attention score to future images received based on the neural network generated using the learning group. The feature maps used within the neural network can be based on any of the analysis metrics described herein, including image quality features and image content features. In some embodiments, learnable filters can be selected and automatically updated based on an image database from an image processing service for content analysis of an image or video frame. In other embodiments, any other such source can be used for the learnable filters. The analysis can be applied to both the image elements of the content and the audio elements of the video.

[0087] In various embodiments, other feedback mechanisms can be used. For example, in some embodiments, a content source, user, or account associated with generating an image or video clip can have associated historical data. In some embodiments, the association of the content source with the history of content selected by the system user or the association of the content source with the history of content associated with a high quality rating can be used as an input to the quality score, or can also be used as a quality flag. Various content source metrics can be used, such as quality history, the number of images sent, the number of system followers or interconnections, or other such metrics.

[0088] In some embodiments, multiple different quality scores may be associated with each respective media content segment such that an image may have an exposure quality score, a noise quality score, a motion quality score, a compression quality score, a resolution quality score, an audio quality score, a content score, or any other such individual quality scores. In this embodiment, an overall quality score based on any combination of such individual quality scores may also be provided. Additionally, as described above, some or all of these quality scores may be used individually to automatically reject certain media content segments, and only images or videos that exceed all thresholds are presented to the system user. The system may have any number of thresholds based on individual quality scores or different combinations of different quality scores. In some embodiments, such thresholds may be variable to present a target number of images and / or videos to the system user. Similarly, different types of images or video clips may be evaluated differently such that different weights may be applied to different images based on content, location, time, proximity to a holiday or news event in terms of location or time, overall environment, or other such information. In some embodiments, any of the above metrics and weights are applied differently to a selfie taken indoors and a concert shot taken outdoors at night. Additionally, the total attention and quality scores of an entire set of content (e.g., a balanced or weighted rating of the content segments within the content set) are used to classify and select content sets for presentation to the user.

[0089] Figure 6 Aspects of an embodiment of system operation for automatically forwarding the presentation of a content set that is part of a communication between a user device 610 and a server system 650 are then shown. In the example embodiment of FIG. 6, in operation 662, the server system 650 receives and stores content. As described above, the content is received from a variety of different devices. The server system 650 processes the content to generate content sets. Based on various factors discussed above, such as entity or account relationships, location, system usage history, or other factors that may be used in different embodiments, the server system 650 selects certain content sets to be made available to the user of device 610 (e.g., via an interface, such as Figure 3A the interface shown). In operation 664, interface information is sent to device 610, which may include collection updates and details of image frames of the content from the collection. Then, in operation 666, device 610 presents an interface including the interface information from operation 664 and receives a user selection of a content set. In Figure 6 the example, device 610 sends a request for the result of the selected content (e.g., a story request) and receives the content as a response. In various other embodiments, some or all of the content in one or more content sets may be cached at device 610. In Figure 6In an embodiment, in operation 668, the server system 650 sends the content of the selected set in response to the user's selection of operation 666 and as a response to the story request.

[0090] Then, in operation 669, the device 610 begins to display the content of the content set. When the display of the first content set is completed or expected (e.g., within a threshold time at the end of the last content segment or based on some other content set end trigger), the device 610 sends an automatic forward communication requesting additional content to the server system 650 in operation 670 and receives the content from the next content set in operation 672. In embodiments having a cache, the initial automatic forward communication may occur based on a trigger associated with the end of the cached content. Then, proceeding with operations 674A-N, the device 610 requests additional content based on automatic forward operations or user navigation inputs. For example, in some embodiments, if the user approaches the end of a content set and enters a "next content set" command (e.g., input 393), a request is sent from the device 610 to the server system 650 and in turn to the next content set. This continues with the content displayed in continuous display operation 669 until an input to end content display is received at the device 610 as part of operation 676.

[0091] Figure 7 Then a method 700 for content set curation and automatic forward operations according to some embodiments is shown. In some embodiments, the method 700 may be executed by a server computer system or server system 250 (such as server systems 150, 250). In other embodiments, the method 700 may be implemented as computer-readable instructions that, when executed by a processor of a computer, cause the computer to execute the method 700. The various embodiments may be implemented in any of the ways described herein. Additionally, it is understood that certain operations may be repeated, reordered, or other operations may be performed between the operations described while still working according to the content set curation and automatic forward embodiments described herein.

[0092] Method 700 begins with operation 702 of storing, at a database of a server computer system, a plurality of content messages 260, 262 from a plurality of client devices 110, 120, where corresponding content segments from each of the content messages 260, 262 are associated with a display time that is less than a threshold display time. Examples of the threshold display time are 10 seconds, 30 seconds, and 60 seconds. Other thresholds may be used in other embodiments. The actual display time of a content segment (e.g., an image that does not have a fixed presentation time) may be less than the threshold. The actual display time may be set by the originator of the image or by system operation. For image content, the threshold may be enforced by simply cutting the video clip display time to the threshold maximum and deleting any data of the image that exceeds the threshold display time limit.

[0093] Operation 704 then involves generating, by the server computer system, a first plurality of content sets from the plurality of content messages 260, 262, where each content set in the first plurality of content sets includes associated content from one or more of the plurality of content messages 260, 262. In operation 706, the server computer system sends interface information for each content set in the first plurality of content sets to a first mobile device. In operation 708, the server computer receives a first story request from the first mobile device, the first story request being associated with a first selection of a first content set in the first plurality of content sets at the first mobile device. In some embodiments, the first story request is merely a notification that content has been selected for display. In some embodiments, the story request is a request for the server to send story content. In some embodiments, the request identifies previously sent content that is being cached at the device for presentation. Then, operation 710 involves receiving, from the first mobile device, an automatic forward communication associated with completion of the presentation of the first content set at the first mobile device. The association between the automatic forward communication and completion may be based on the actual display of the last content segment in the content set, or on a threshold or other indicator for the expected display of the next content set. Any such threshold or operation for initiating communication of the next content set may be used for the automatic forward communication. Then, operation 712 includes automatically transmitting, in response to receiving the story request and the automatic forward communication, a second content set from the server computer system to the first mobile device for automatic display on the first mobile device, where the second content set is automatically selected by the server computer system.

[0094] In various embodiments, operations of the method can continue where an automated forward communication requests the display of a next content set on the device. In some embodiments, when all content sets associated with the interface information provided to the device have been displayed, the device can send a real-time automated forward communication to the server. This real-time automated forward communication can simply indicate a request for current or real-time content in content set formation, or in addition can request that the server take certain actions to select additional content sets other than the content sets identified in the initial interface information. In some embodiments, the real-time automated forward communication can indicate that the device is requesting a change from one set of rules for selecting a next content set to a different set of rules for selecting a next content set.

[0095] Figure 8 Method 800 is an example embodiment of operations that can be implemented as operations of a corresponding device in communication with a server computer executing Method 700. In other embodiments, methods other than Method 800 and Method 700 can be executed by the device and server system 250 to implement curation and automated forwarding as described herein. In different embodiments, the operations of Method 800 can be implemented in any manner described for similar operations discussed herein.

[0096] Method 800 begins at operation 802, receiving interface information for a first plurality of content sets from a server computer system, the interface information including a plurality of content images associated with each content set in the first plurality of content sets. Then, operation 804 involves receiving, by the first mobile device, user input initiating a story request associated with selecting a first content set in the first plurality of content sets, and operation 806 continues by receiving from the server computer system a first ordered group of content segments associated with the first content set, each content segment in the ordered group of content segments having an associated display time that is less than a threshold display time.

[0097] Then, in operation 808, the first mobile device displays at least a portion of the first ordered content segment group. In operation 810, which follows operation 808, a story end trigger associated with the display of the last content segment in the first ordered content segment group is identified, and in response to the identification of the story end trigger, the device automatically initiates, in operation 812, the display of a second ordered content segment group associated with a second content set among the first plurality of content sets. In various embodiments, the story end trigger can be the actual display end of the last content segment from a content set. In other embodiments, this can be a trigger based on an expected time to fetch content from the server before the display of the last content segment of the content set ends. In some embodiments, this can be a trigger indicating a request to cache the content of the next content set. In still further embodiments, other triggers can also be used to request the selection and / or transmission of the next content set to be automatically presented at the device. Additionally, as described herein, in various embodiments, multimedia advertising elements (e.g., images, videos, graphics, etc.) can be provided to the device to be displayed between any or all content sets. Different advertising thresholds can be used, such as the time since the last presentation of an advertising element, the advertising-to-content ratio, or any other such threshold.

[0098] Figure 9 is a block diagram 900 showing the architecture of software 902 that can be installed on any one or more of the above devices. For example, in various embodiments, client devices 110 and 120 and server systems 150, 250, and 550 can implement some or all elements of software 902. Figure 9 is merely a non-limiting example of a software architecture, and it will be understood that many other architectures can be implemented to facilitate the functions described herein. In various embodiments, software 902 is implemented by hardware of a machine 1000 such as Figure 10 and the machine 1000 includes a processor 1010, a memory 1030, and I / O components 1050. In this example architecture, software 902 can be conceptualized as a stack of layers, where each layer can provide a specific function. For example, software 902 includes layers such as an operating system 904, libraries 906, frameworks 908, and applications 910. Operationally, consistent with some embodiments, application 910 makes application programming interface (API) calls 912 through the software stack and receives messages 914 in response to the API calls 912.

[0099] In various embodiments, the operating system 904 manages hardware resources and provides common services. The operating system 904 includes, for example, a kernel 920, services 922, and drivers 924. Consistent with some embodiments, the kernel 920 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 920 provides functions such as memory management, processor management (e.g., scheduling), component management, network connectivity, and security settings. The services 922 may provide other common services to other software layers. According to some embodiments, the drivers 924 are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 924 may include a display driver, a camera driver, or Bluetooth Low Energy a flash drive, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a driver, an audio driver, a power management driver, etc.

[0100] In some embodiments, the libraries 906 provide a low-level common infrastructure utilized by the applications 910. The libraries 906 may include system libraries 930 (e.g., the C standard library), which may provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, the libraries 906 may include API libraries 932, such as media libraries (e.g., libraries that support the presentation and manipulation of various media formats such as Moving Picture Experts Group - 4 (MPEG4), High Efficiency Video Coding (H.264 or AVC), Moving Picture Experts Group - 3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., the OpenGL framework for presenting two-dimensional (2D) and three-dimensional (3D) graphics content on a display), database libraries (e.g., SQLite that provides various relational database functions), web libraries (e.g., WebKit that provides web browsing functionality), etc. The libraries 906 may also include a variety of other libraries 934 to provide many other APIs to the applications 910.

[0101] According to some embodiments, the frameworks 908 provide a high-level common architecture that can be utilized by the applications 910. For example, the frameworks 908 provide various Graphical User Interface (GUI) functions, high-level resource management, high-level location nodes, etc. The frameworks 908 may provide a wide range of other APIs that can be utilized by the applications 910, some of which may be specific to a particular operating system 904 or platform.

[0102] In an example embodiment, the applications 910 include a home page application 950, a contacts application 952, a browser application 954, a book reader application 956, a location application 958, a media application 960, a messaging application 962, a gaming application 964, and other widely classified applications such as third-party applications 966. According to some embodiments, the applications 910 are programs that execute functions defined in the program. One or more applications 910 can be created using various programming languages and structured in various ways, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, the third-party application 966 (e.g., an application developed using an ANDROID TM or IOS TM software development kit (SDK) by an entity other than the vendor of a particular platform) can be mobile software that runs on a mobile operating system such as IOS TM 、ANDROID TM 、 Phone or other mobile operating systems). In this example, the third-party application 966 can call the API calls 912 provided by the operating system 904 to facilitate the execution of the functions described herein.

[0103] Some embodiments can particularly include a content collection application 967. In certain embodiments, this can be a stand-alone application that operates to manage communication with a server system such as the server system 150. In other embodiments, this function can be integrated with another application, such as the media application 960 or another such application. The content collection application 967 can use the camera device of the machine 1000 to manage content collection, communicate with the server system 550 via the I / O component 1050, and receive and store the received content collection in the memory 1030. The presentation of the content and the user input associated with the content can be managed by the story application 967 using different frameworks 908, library 906 elements, or elements of the operating system 904 operating on the machine 1000.

[0104] Figure 10 is a block diagram showing the components of a machine 1000 that can read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and execute any one or more of the methods discussed herein. Specifically, Figure 10FIG. 0 shows a schematic diagram of a machine 1000 in the example form of a computer system within which instructions 1016 (e.g., software, program, application 910, applet, application program, or other executable code) can be executed to cause the machine 1000 to perform any one or more of the methods discussed herein. In alternative embodiments, the machine 1000 operates as a stand-alone device or can be coupled (e.g., network-connected) to other machines. In a networked deployment, the machine 1000 can operate in the capacity of a server machine 550 or a client device 110, 120 in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 can include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smartphone, a mobile device 800, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a network device, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1016 that specify actions to be taken by the machine 1000, either sequentially or otherwise. Further, although only a single machine 1000 is shown, the term "machine" can also be considered to include a collection of machines 1000 that individually or jointly execute the instructions 1016 to perform any one or more of the methods discussed herein.

[0105] In various embodiments, the machine 1000 includes a processor 1010, a memory 1030, and I / O components 1050 that can be configured to communicate with each other via a bus 1002. In an example embodiment, the processor 1010 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) includes, for example, a processor 1012 and a processor 1014 that can execute the instructions 1016. The term "processor" is intended to include a multi-core processor 1010 that can include more than two independent processors 1012, 1014 (also referred to as "cores") that can execute the instructions 1016 simultaneously. Although Figure 10 multiple processors 1010 are shown, the machine 1000 can include a single processor 1010 with a single core, a single processor 1010 with multiple cores (e.g., a multi-core processor 1010), multiple processors 1012, 1014 with a single core, multiple processors 1010, 1012 with multiple cores, or any combination thereof.

[0106] According to some embodiments, the memory 1030 includes a main memory 1032, a static memory 1034, and storage unit 1036 that can be accessed by the processor 1010 via the bus 1002. The storage unit 1036 may include a machine-readable medium 1038 having instructions 1016 embodied thereon that implement any one or more of the methods or functions described herein. The instructions 1016 may also reside, completely or at least partially, within at least one of the main memory 1032, the static memory 1034, the processor 1010 (e.g., within a cache of the processor) or any suitable combination during execution thereof by the machine 1000. Accordingly, in various embodiments, the main memory 1032, the static memory 1034, and the processor 1010 are considered to be machine-readable media 1038.

[0107] As used herein, the term "memory" refers to a machine-readable medium 1038 that can store data temporarily or permanently and may be considered to include, without limitation, random access memory (RAM), read-only memory (ROM), cache memory, flash memory, and buffer memory. Although the machine-readable medium 1038 is shown as a single medium in the exemplary embodiments, the term "machine-readable medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) that can store the instructions 1016. The term "machine-readable medium" may also be regarded as including any medium or combination of media that can store instructions (e.g., instructions 1016) for execution by a machine (e.g., machine 1000) such that the instructions 1016, when executed by one or more processors (e.g., processor 1010) of the machine 1000, cause the machine 1000 to perform any one or more of the methods described herein. Thus, the term "machine-readable medium" refers to a single storage device or apparatus and to "cloud-based" storage systems or storage networks that include multiple storage devices or apparatuses. Accordingly, the term "machine-readable medium" may be considered to include, without limitation, one or more data repositories in the form of solid-state memory (e.g., flash memory), optical media, magnetic media, other non-volatile memory (e.g., erasable programmable read-only memory (EPROM)), or any suitable combination thereof. The term "machine-readable medium" specifically excludes non-statutory signals per se.

[0108] The I / O components 1050 include a variety of components for receiving input, providing output, generating output, sending information, exchanging information, taking measurements, and the like. Generally, it will be appreciated that the I / O components 1050 may include Figure 10Many other components not shown. The I / O components 1050 are grouped according to function only for the sake of simplicity in the following discussion, and the grouping is in no way restrictive. In various example embodiments, the I / O components 1050 include output components 1052 and input components 1054. The output components 1052 include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), auditory components (e.g., speakers), tactile components (e.g., a vibration motor), other signal generators, and the like. The input components 1054 include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), haptic input components (e.g., physical buttons, a touch screen that provides the location and force of a touch or touch gesture, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0109] In some additional example embodiments, the I / O components 1050 include biometric components 1056, motion components 1058, environmental components 1060, or location components 1062 among various other components. For example, the biometric components 1056 include components that detect expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measure biometric signals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identify people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), and the like. The motion components 1058 include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), and the like. The environmental components 1060 include, for example, lighting sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensor components (e.g., machine olfactory detection sensors, gas detection sensors for detecting the concentration of hazardous gases for safety or measuring pollutants in the atmosphere), or other components that may provide an indication, measurement, or signal corresponding to the surrounding physical environment. The location components 1062 include location sensor components (e.g., global positioning system (GPS) receiver components), altitude sensor components (e.g., altimeters or barometers that can detect the air pressure from which the altitude can be derived), orientation sensor components (e.g., magnetometers), and the like.

[0110] Communication can be implemented using a variety of techniques. The I / O component 1050 can include a communication component 1064 that is operable to couple the machine 1000 to the network 1080 or the device 1070 via the coupler 1082 and the coupler 1072, respectively. For example, the communication component 1064 includes a network interface component or another suitable device that interfaces with the network 1080. In another example, the communication component 1064 includes a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, components (e.g., low power ), components and other communication components that provide communication via other modes. The device 1070 can be any one of another machine 1000 or a variety of peripheral devices (e.g., peripheral devices coupled via a universal serial bus (USB)).

[0111] In addition, in some embodiments, the communication component 1064 detects an identifier or includes a component operable to detect an identifier. For example, the communication component 1064 includes a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as a universal product code (UPC) barcode, multi-dimensional barcodes such as a quick response (QR) code, Aztec code, data matrix, digital graphic, maxi code, PDF417, hypercode, unified commerce code reduced space symbol (UCC RSS)-2D barcode, and other optical codes), an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag), or any suitable combination thereof. In addition, various information such as a location via Internet protocol (IP) geolocation, a location via signal triangulation, a location via detecting or an NFC beacon signal, etc. can be derived via the communication component 1064 that can indicate a specific location.

[0112] Transmission medium

[0113] In various example embodiments, one or more portions of the network 1080 can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, another type of network, or a combination of two or more such networks. For example, network 1080 or a portion of network 1080 can include a wireless or cellular network, and the coupling 1082 can be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling 1082 can implement any one of a variety of types of data transmission technologies, such as Single-Carrier Radio Transmission Technology (1xRTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, GSM Enhanced Data Rates for GSM Evolution (EDGE) technology, 3rd Generation Partnership Project (3GPP) including 3G, 4th Generation Wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standard, other standards defined by various standards organizations, other remote protocols, or other data transmission technologies.

[0114] In an example embodiment, instructions 1016 are sent or received via network 1080 using a transmission medium (e.g., a network interface component included in communication component 1064), and any one of a plurality of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)) is utilized. Similarly, in other example embodiments, instructions 1016 are sent or received to or from device 1070 via coupling 1072 (e.g., a peer-to-peer coupling) using a transmission medium. The term "transmission medium" can be regarded as including any non-transitory medium that is capable of storing, encoding, or carrying instructions 1016 executable by machine 1000, and includes digital or analog communication signals or other non-transitory media for facilitating the communication implementation of such software.

[0115] Furthermore, since machine-readable medium 1038 does not embody a propagated signal, machine-readable medium 1038 is non-transitory (in other words, does not have any transient signals). However, labeling machine-readable medium 1038 as "non-transitory" should not be construed to mean that the medium cannot be moved. Medium 1038 should be considered to be transportable from one physical location to another. Additionally, since machine-readable medium 1038 is tangible, medium 1038 can be considered a machine-readable device.

[0116] Language

[0117] Throughout the specification, multiple instances may implement components, operations, or structures described as a single instance. Although the separate operations of one or more methods are shown and described as separate operations, one or more of the separate operations may be performed concurrently, and the operations need not be performed in the order shown. Structures and functions presented as separate components in an example configuration may be implemented as a combined structure or component. Similarly, structures and functions presented as a single component may be implemented as separate multiple components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0118] Although the subject matter of the present invention has been described with reference to specific example embodiments, various modifications and changes can be made to these embodiments without departing from the broader scope of the disclosed embodiments. Such embodiments of the subject matter of the present invention may herein be referred to individually or collectively by the term "invention," solely for convenience, and are not intended to limit the scope of the present application to any single disclosure or inventive concept if more than one is in fact disclosed.

[0119] The embodiments shown herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes can be made without departing from the scope of the present disclosure. Accordingly, the detailed description should not be considered limiting, and the scope of various embodiments is defined only by the appended claims and the full scope of equivalents given by those claims.

[0120] As used herein, the term "or" may be interpreted in an inclusive or exclusive manner. Additionally, multiple instances may be provided for resources, operations, or structures described herein as a single instance. Further, the boundaries between various resources, operations, modules, engines, and data stores are to some extent arbitrary, and particular operations are shown in the context of a particular illustrative configuration. Other allocations of functionality are conceivable, and these other allocations may fall within the scope of various embodiments of the present disclosure. In general, structures and functions presented as separate resources in an example configuration may be implemented as a combined structure or resource. Similarly, structures and functions presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of the embodiments of the present disclosure as represented by the appended claims. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. A method for network communication of an image-based content collection, the method comprising: Storing, at a database of a server computer system, a plurality of content messages from a plurality of client devices, wherein a respective content segment from each content message is associated with a display time that is less than a threshold display time; Generating, by the server computer system, a plurality of content collections from the plurality of content messages, each content collection of the plurality of content collections including associated content from one or more of the plurality of content messages; Transmitting, from the server computer system, interface information for each content collection of the plurality of content collections to a first mobile device; Receiving, from the first mobile device, a first story request associated with a first selection of a first content collection of the plurality of content collections at the first mobile device; Receiving, from the first mobile device, an auto-forward communication associated with completion of a presentation of the first content collection at the first mobile device; and In response to receiving the first story request and the auto-forward communication, automatically transmitting a second content collection from the server computer system to the first mobile device for automatic display on the first mobile device, wherein, The second content collection is automatically selected by the server computer system based on a predefined order for presenting the plurality of content collections, the predefined order being at least partially based on a total quality score assigned to each content collection, For each content collection of the plurality of content collections, the total quality score corresponds to the sum of individual quality scores assigned to respective content segments within the content collection, each individual quality score being based on a set of image quality metrics applied to the respective content segment, and Selecting each respective content segment within the content collection for presentation based on the individual quality score and weighting based on at least one of content, location, or time associated with the respective content segment.

2. The method according to claim 1, wherein The first content collection includes a first set of content from a first set of content messages of the plurality of content messages, wherein the first set of content messages is received from a second mobile device associated with a second user account of the server computer system; wherein the first mobile device is associated with a first user account of the server computer system; and wherein the first mobile device and the second mobile device are associated with the server computer system.

3. The method according to claim 2, wherein, The first content collection is selected by the server computer system for the plurality of content collections based on an association between the first user account and the second user account.

4. The method according to claim 2, wherein The first set of content messages includes all content messages received at the server computer system from the second mobile device within a story threshold time before receiving the first story request at the server computer system.

5. The method according to claim 4, wherein The story threshold time is 24 hours.

6. The method according to claim 1, wherein The plurality of content collections includes content collections from a plurality of friend accounts associated with the first mobile device.

7. The method according to claim 6, further comprising: Receiving, at the server computer system, a real-time automatic forwarding communication associated with completion of an end presentation of a last first content set of the plurality of content sets at the first mobile device; And In response to receiving the real-time automatic forwarding communication, automatically transmitting a first real-time content set from the server computer system to the first mobile device for automatic display on the first mobile device.

8. The method according to claim 7, further comprising: Communicating one or more advertisement elements for presentation on the first mobile device prior to automatic display of the second content set.

9. The method according to claim 7, further comprising: After presentation of the first real-time content set or a trigger content set of the plurality of content sets, communicating one or more advertisement elements for presentation on the first mobile device, wherein the trigger content set is selected from content sets of the first real-time content set or the plurality of content sets based on an advertisement threshold.

10. The method according to claim 7, further comprising: Receiving, at the server computer system, a second plurality of content messages; Receiving, at the server computer system via a curation tool, a selection of real-time media content for the first real-time content set from the second plurality of content messages; And Periodically updating the real-time media content of the first real-time content set using the curation tool.

11. The method according to claim 1, further comprising: Receiving, at the first mobile device, interface information of the plurality of content sets from the server computer system; Receiving, at the first mobile device, a user input that initiates a story request associated with a selection of the first content set among the plurality of content sets; Receiving, at the first mobile device, a first ordered content segment group associated with the first content set from the server computer system, each content segment of the first ordered content segment group having an associated display time that is less than a threshold display time.

12. The method according to claim 11, further comprising: Displaying, at the first mobile device, at least a portion of the first ordered content segment group; Identifying, at the first mobile device, a story end trigger associated with display of a last content segment of the first ordered content segment group; and In response to identification of the story end trigger, automatically initiating display of a second ordered content segment group associated with a second content set among the plurality of content sets.

13. The method according to claim 1, wherein The interface information includes at least one frame of image data of each content segment from each content set of the plurality of content sets.

14. The method according to claim 7, wherein, The plurality of content sets includes a third real-time content set; Wherein the interface information further includes one or more frames of image data of each content segment from the third real-time content set, and each frame of the one or more frames of image data of each content segment from the third real-time content set is selected using a curation tool.

15. The method according to claim 1, wherein The threshold display time includes a time value less than or equal to ten seconds.

16. A method for communication and presentation of an image-based content set at a first mobile device, the method comprising: Receiving interface information of multiple content sets from a server computer system, the interface information including multiple content images associated with each of the multiple content sets; Receiving, by the first mobile device, a user input that initiates a story request associated with a selection of a first content set among the multiple content sets; Receiving, from the server computer system, a first ordered group of content segments associated with the first content set, each content segment in the first ordered group of content segments having an associated display time that is less than a threshold display time; Displaying, by the first mobile device, at least a portion of the first ordered group of content segments; Identifying, by the first mobile device, a story end trigger associated with the display of the last content segment of the first ordered group of content segments; and Automatically initiating, in response to the identification of the story end trigger, the display of a second ordered group of content segments associated with a second content set among the multiple content sets, wherein: The second content set is automatically selected based on a predefined order for presenting the multiple content sets, the predefined order being at least partially based on a total quality score assigned to each content set, For each of the multiple content sets, the total quality score corresponds to the sum of individual quality scores assigned to the corresponding content segments within the content set, each individual quality score being based on a set of image quality metrics applied to the corresponding content segment, and Each corresponding content segment within the content set is selected for presentation based on the individual quality score and weighted based on at least one of the content, position, or time associated with the corresponding content segment.

17. The method according to claim 16, further comprising: Automatically presenting each of the multiple content sets in the predefined order, wherein the predefined order is further based on the time associated with the most recently received content message for each of the multiple content sets.

18. The method according to claim 17, further comprising: Receiving, at the first mobile device, a first click input on the touch screen of the first mobile device; In response to the first click input, transitioning to a previous media content segment of the currently displayed content set; Receiving, at the first mobile device, a first swipe input on the touch screen of the first mobile device; And In response to the first swipe input, transitioning to the display of a previous content set and continuing the display from the beginning of the previous content set based on the predefined order.

19. The method according to claim 17, further comprising: After the end of each of the multiple content sets, before automatically displaying the next content set in the predefined order, displaying a standardized media display indicating the end of the content set; and Before the start of the next content set, displaying one or more advertisement elements after at least a portion of the multiple content sets.

20. A non-transitory computer-readable medium includes instructions that, when executed by one or more processors in a first mobile device, cause the first mobile device to perform a method for automatic forwarding during the display of multiple content sets on the first mobile device. The method includes: Receiving, from a server computer system, interface information for multiple content sets, the interface information including multiple content images associated with each content set of the multiple content sets; Receiving, by the first mobile device, user input that initiates a story request associated with the selection of a first content set of the multiple content sets; Receiving, from the server computer system, a first ordered content segment group associated with the first content set, each content segment in the first ordered content segment group having an associated display time that is less than a threshold display time; Displaying, by the first mobile device, at least a portion of the first ordered content segment group; Identifying, by the first mobile device, a story end trigger associated with the display of the first ordered content segment group; Transmitting an automatic forwarding communication associated with the story end trigger to the server computer system; and Automatically initiating, in response to the identification of the story end trigger, the display of a second ordered content segment group associated with a second content set of the multiple content sets, Wherein: The second content set is automatically selected based on a predefined order for presenting the multiple content sets, the predefined order being at least partially based on a total quality score assigned to each content set, For each content set of the multiple content sets, the total quality score corresponds to the sum of individual quality scores assigned to the corresponding content segments within the content set, each individual quality score being based on a set of image quality metrics applied to the corresponding content segment, and Each corresponding content segment within the content set is selected for presentation based on the individual quality score and weighted based on at least one of the content, location, or time associated with the corresponding content segment.

Images