Content collections linked to elementary items

Abstract

Content sharing among the first user, the second user and the third user is facilitated. The first common content set includes a first user input content item of a first user. In response to receiving an indication of a combined user input from a second user, a combination function is invoked to allow the second user to combine the first user input content item with the second user input content item to create a first combined user input content item. The first combined user input content item is stored in a second common content set in association with the first user input content item. A third user provides a combined set of presentation user inputs related to the first user input content items. In response, a combined set presentation function is invoked to enable the third user to navigate the second common set of content.

Description

[0001] Priority declaration

[0002] This application claims the benefit of priority to U.S. Provisional Application Serial No. 63 / 365,167 filed on May 23, 2022, U.S. Provisional Application Serial No. 63 / 365,648 filed on June 1, 2022, U.S. Provisional Application Serial No. 63 / 353,818 filed on June 20, 2022, U.S. Provisional Application Serial No. 63 / 369,363 filed on July 25, 2022, and U.S. Application Serial No. 18 / 147,405 filed on December 28, 2022, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] The subject matter disclosed herein generally relates to, but is not limited to, techniques for modifying, enhancing, or combining content items in the context of a media application to create a combined content item. Background Art

[0004] The popularity of computer-implemented programs that allow users to access content online and interact with other users continues to grow. There are various computer-implemented applications that allow users to share content with other users through a messaging client. Some of such computer-implemented applications may be designed to run on mobile devices (such as mobile phones, tablets, or wearable devices) with back-end services provided on a server computer system to perform operations that may require more resources than are reasonably required to be performed at the client device.

[0005] A media application such as a messaging client executed at a user device may provide a user interface that allows the user to capture photos or videos and share the captured or viewed content to other devices. The media application may also provide a user interface that allows the user to modify or enhance the content before sharing it to other devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the accompanying drawings (which are not necessarily drawn to scale), the same reference numerals may describe similar components in different views. To easily identify the discussion of any particular element or action, the highest digit or digits in the reference numeral refer to the figure number in which the element is first introduced. Some non-limiting examples are shown in the accompanying drawings, in which:

[0007] Figure 1 is a diagrammatic representation of a networked environment in which the present disclosure may be deployed, according to some examples.

[0008] Figure 2 is a diagrammatic representation of an interactive system having both client-side and server-side functionality, according to some examples.

[0009] Figure 3 is a diagrammatic representation of data structures as maintained in a database according to some examples.

[0010] Figure 4 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0011] Figure 5 is a flow chart illustrating a method including aspects related to combining functions according to some examples.

[0012] Figure 6 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0013] Figure 7 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0014] Figure 8 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0015] Fig. 9 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0016] Fig.10 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0017] Fig.11 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0018] Fig.12 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0019] Fig.13 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0020] Fig.14 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0021] Fig.15 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0022] Fig.16 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0023] Fig.17 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0024] Fig.18 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0025] Fig.19 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0026] Fig. 20 is a flow chart illustrating a method including aspects related to enhancing user input content items according to some examples.

[0027] Fig.21 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0028] Fig. 22 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0029] Fig.23 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0030] Fig.24 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0031] Fig.25 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0032] Fig.26 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0033] Fig. 27 is a user interface diagram illustrating aspects related to enhancing user input of content items according to some examples.

[0034] Fig.28 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0035] Fig.29 is a flow chart illustrating a method including aspects related to combining functionality of media applications according to some examples.

[0036] Fig.30 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0037] Fig.31 is a flow chart illustrating a method including aspects related to combining functionality of media applications according to some examples.

[0038] Fig.32 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0039] Fig.33 is a flow chart illustrating a method including aspects related to combining functions according to some examples.

[0040] Fig.34 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0041] Fig.35 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0042] Fig.36 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0043] Fig.37 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0044] Fig.38 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0045] Fig.39 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0046] Fig.40 is a user interface diagram illustrating aspects related to combining user input content items according to some examples.

[0047] Fig.41 is a user interface sequence diagram illustrating aspects related to combining user input of a content item and unlocking a content sharing destination according to some examples.

[0048] Fig.42 is a flow chart illustrating a method including aspects related to combining functions and unlocking content sharing destinations according to some examples.

[0049] Fig.43 is a flow chart illustrating a method including aspects related to combining functions and unlocking content sharing destinations according to some examples.

[0050] Fig.44is a user interface diagram illustrating aspects related to combining user input content items and unlocking content sharing targets according to some examples.

[0051] Fig.45 is a user interface diagram illustrating aspects related to combining user input content items and unlocking content sharing targets according to some examples.

[0052] Fig.46 is a user interface diagram illustrating aspects related to combining user input content items and unlocking content sharing targets according to some examples.

[0053] Fig.47 is a user interface diagram illustrating aspects related to combining user input content items and unlocking content sharing targets according to some examples.

[0054] Fig.48 is an interaction diagram illustrating a process including aspects related to sharing a combined user input content item to a content collection, according to some examples.

[0055] Fig.49 is a flow chart illustrating a method including aspects related to sharing a combined user input content item to a content collection according to some examples.

[0056] Fig.50 is a flow chart illustrating a method including aspects related to sharing a combined user input content item to a content collection according to some examples.

[0057] Fig.51 is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0058] Fig.52 is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0059] Fig.53 is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0060] Fig.54 is a user interface diagram illustrating a portion of a user interface showing aspects related to sharing a combined user input content item to a content collection, according to some examples.

[0061] Fig.55 is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0062] Fig.56is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0063] Fig.57 is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0064] Fig.58 is a user interface diagram illustrating aspects related to sharing a combined user input content item to a content collection according to some examples.

[0065] Fig.59 is a user interface diagram illustrating a portion of a user interface showing aspects related to sharing a combined user input content item to a content collection, according to some examples.

[0066] Fig.60 is a graphical representation of messages according to some examples.

[0067] Fig.61 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein.

[0068] Fig.62 A system including a head wearable device according to some examples is shown.

[0069] Fig.63 is a block diagram illustrating a software architecture in which examples may be implemented. DETAILED DESCRIPTION

[0070] Examples of the present disclosure improve the functionality of electronic software and systems by enabling users to modify, enhance, or combine content items to create combined content items in media applications. In examples of the present disclosure, users are able to combine multiple user input content items (e.g., multiple user generated content items, referred to as UGC) into combined user input content items, thereby providing enhanced software and system functionality and providing users with a powerful storytelling mechanism.

[0071] Networked computing environment

[0072] Figure 11 is a block diagram illustrating an example interactive system 100 for facilitating interactions over a network (e.g., exchanging text messages, conducting text audio and video calls, or playing games). The interactive system 100 includes a plurality of user systems 104, each of which hosts a plurality of applications including an interactive client 106 and other applications 108. Each interactive client 106 is communicatively coupled to other instances of the interactive client 106 (e.g., hosted on respective other user systems 104, interactive server systems 102, and third-party servers 112) via one or more communication networks including a network 110 (e.g., the Internet). The interactive client 106 may also communicate with the locally hosted application 108 using an application program interface (API).

[0073] Each user system 104 may include multiple user devices, such as a mobile device 114 , a head wearable device 116 , and a computer device 118 .

[0074] The interactive clients 106 interact with other interactive clients 106 and with the interactive server system 102 via the network 110. Data exchanged between the interactive clients 106 (e.g., interaction 120) and between the interactive clients 106 and the interactive server system 102 includes functions (e.g., commands for activating functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0075] The interactive server system 102 provides server-side functions to the interactive clients 106 via the network 110. Although certain functions of the interactive system 100 are described herein as being performed by the interactive clients 106 or by the interactive server system 102, it may be a design choice whether certain functions are located within the interactive clients 106 or within the interactive server system 102. For example, it may be technically preferred to initially deploy certain technologies and functions within the interactive server system 102, but later migrate the technologies and functions to the interactive clients 106 where the user systems 104 have sufficient processing power.

[0076] The interactive system 100 supports various services and operations provided to the interactive clients 106. Such operations include sending data to the interactive clients 106, receiving data from the interactive clients 106, and processing data generated by the interactive clients 106. The data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. The data exchange within the interactive server system 102 is activated and controlled by functions available through the user interface of the interactive client 106.

[0077] Turning now specifically to the interaction server system 102, the API server 122 is coupled to and provides a programming interface to the interaction server 124, making the functionality of the interaction server 124 accessible to the interaction clients 106, other applications 108, and third-party servers 112. The interaction server 124 is communicatively coupled to a database server 126, thereby facilitating access to a database 128 that stores data associated with interactions processed by the interaction server 124. Similarly, a web server 130 is coupled to the interaction server 124 and provides a web-based interface to the interaction server 124. To this end, the web server 130 handles incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0078] The API server 122 receives and sends interaction data (e.g., commands and message payloads) between the interaction server 124 and the user system 104 (and, for example, the interaction client 106 and other applications 108) and the third-party server 112. Specifically, the API server 122 provides a set of interfaces (e.g., routines and protocols) that the interaction client 106 and other applications 108 can call or query to activate the functions of the interaction server 124. The API server 122 exposes various functions supported by the interaction server 124, including account registration; login functionality; sending interaction data from a particular interaction client 106 to another interaction client 106 via the interaction server 124; transferring media files (e.g., images or videos) from the interaction client 106 to the interaction server 124; setting up a collection of media data (e.g., a story); retrieving a friend list of a user of the user system 104; retrieving information and content; adding and removing entities (e.g., friends) from an entity graph (e.g., a social graph); locating friends within a social graph; and opening application events (e.g., related to the interaction client 106).

[0079] The interactive server 124 hosts multiple systems and subsystems, as shown below. Figure 2 Give a description.

[0080] Linked Applications

[0081] Returning to the interactive client 106, the features and functions of the external resource (e.g., the linked application 108 or applet) are available to the user via the interface of the interactive client 106. In this context, "external" refers to the fact that the application 108 or applet is outside the interactive client 106. External resources are usually provided by a third party, but can also be provided by the creator or provider of the interactive client 106. The interactive client 106 receives a user selection of an option for launching or accessing the features of such an external resource. The external resource can be an application 108 (e.g., a "local app") installed on the user system 104, or a small-scale version of an application (e.g., a "applet") hosted on the user system 104 or located at the user system 104 remote end (e.g., on a third-party server 112). The small-scale version of the application includes a subset of the features and functions of the application (e.g., a full-scale local version of the application) and is implemented using a markup language document. In some examples, the small-scale version of the application (e.g., a "applet") is a web-based markup language version of the application and is embedded in the interactive client 106. In addition to using markup language documents (eg, .*ml files), applets may include scripting languages ​​(eg, .*js files or .json files) and style sheets (eg, .*ss files).

[0082] In response to receiving a user selection of an option for launching or accessing a feature of an external resource, the interactive client 106 determines whether the selected external resource is a web-based external resource or a locally installed application 108. In some cases, the application 108 locally installed on the user system 104 can be independent of the interactive client 106 and launched separately from the interactive client 106, such as by selecting an icon corresponding to the application 108 on a home screen of the user system 104. A small-scale version of such an application can be launched or accessed via the interactive client 106, and in some examples, no portion of the small-scale application can be accessed outside of the interactive client 106 or only a limited portion of the small-scale application can be accessed outside of the interactive client 106. The small-scale application can be launched by the interactive client 106 receiving, for example, a markup language document associated with the small-scale application from the third-party server 112 and processing such a document.

[0083] In response to determining that the external resource is a locally installed application 108, the interactive client 106 instructs the user system 104 to launch the external resource by executing the locally stored code corresponding to the external resource. In response to determining that the external resource is a web-based resource, the interactive client 106 communicates with the third-party server 112 (for example) to obtain a markup language document corresponding to the selected external resource. The interactive client 106 then processes the obtained markup language document to present the web-based external resource within the user interface of the interactive client 106.

[0084] The interactive client 106 can notify the user of the user system 104 or other users (e.g., "friends") related to such users of activities occurring in one or more external resources. For example, the interactive client 106 can provide notifications related to external resources currently or recently used by one or more members of a group of users to participants in a conversation (e.g., a chat session) in the interactive client 106. One or more users can be invited to join an active external resource or start an external resource that was recently used but is currently inactive (in the group of friends). The external resource can provide the participants in the conversation who each use the corresponding interactive client 106 with the ability to share items, conditions, states, or locations in the external resource with one or more members of a group of users in the chat session. The shared item can be an interactive chat card, and the members of the chat can interact with the interactive chat card to, for example, start the corresponding external resource, view specific information in the external resource, or bring the members of the chat to a specific location or state in the external resource. In a given external resource, a response message can be sent to the user on the interactive client 106. The external resource can selectively include different media items in the response based on the current context of the external resource.

[0085] The interactive client 106 can present a list of available external resources (e.g., applications 108 or applets) to the user to launch or access a given external resource. The list can be presented in the form of a context-sensitive menu. For example, icons representing different applications (or applets) of the application 108 (or applets) can change based on how the user launches the menu (e.g., from a conversational interface or from a non-conversational interface).

[0086] System Architecture

[0087] Figure 2 1 is a block diagram showing additional details about the interactive system 100 according to some examples. Specifically, the interactive system 100 is shown to include an interactive client 106 and an interactive server 124. The interactive system 100 contains a plurality of subsystems that are supported on the client side by the interactive client 106 and on the server side by the interactive server 124. An exemplary system will be discussed below.

[0088] Image processing system 202 provides various functions that enable a user to capture and enhance (eg, annotate or otherwise modify or edit) media content associated with a message.

[0089] The camera system 204 includes control software (e.g., in a camera application) that interacts with and controls the hardware camera hardware of the user system 104 (e.g., directly or via operating system control) to modify and enhance real-time images captured and displayed via the interactive client 106. The camera system 204 includes multiple subsystems, including a remix system 206 that enables users of the interactive system 100 to "remix" multiple content items to create a combined content item. For example, using the remix system 206, a user can combine a user input content item with a selected additional user input content item (e.g., an image or video captured by the user, or an image or video selected by the user from a media library or file storage) to create a combined user input content item while viewing a collection of user input content items of one or more users of the interactive system 100. The remix system 206 provides various tools to enable users to creatively combine content items using such tools, including layout tools. Using the layout tools, the relative layout of a first user input content item (or multiple user input content items) and a second user input content item can be defined and specified.

[0090] The enhancement system 208 provides functionality related to the generation and publication of enhancements (e.g., media overlays) to images captured in real time by the camera of the user system 104 or retrieved from the memory of the user system 104. For example, the enhancement system 208 is operable to select, present, and display media overlays (e.g., image filters or image lenses) to the interactive client 106 for enhancing real-time images received via the camera system 204 or stored images retrieved from the memory of the user system 104. These enhancements are selected and presented to the user of the interactive client 106 by the enhancement system 208 based on some input and data, such as:

[0091] The geolocation of the user system 104; and

[0092] Social network information of users of the user system 104 .

[0093] Enhancement can include audio and visual content and visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlays. Audio and visual content or visual effects can be applied to media content items (e.g., photos or videos) at user systems 104 for transmission in messages, or applied to video content such as video content streams or feeds sent from interactive clients 106. Therefore, image processing system 202 can interact with various subsystems of communication system 210, and supports various subsystems of communication system 210, such as messaging system 212 and video communication system 214.

[0094] Media overlays may include text or image data that can be superimposed on a photo taken by the user system 104 or a video stream produced by the user system 104. In some examples, the media overlay may be a location overlay (e.g., Venice Beach), the name of a live event, or a business name overlay (e.g., Beach Cafe). In other examples, the image processing system 202 uses the geolocation of the user system 104 to identify a media overlay that includes the name of a business at the geolocation of the user system 104. The media overlay may include other tags associated with the business. The media overlay may be stored in the database 128 and accessed by the database server 126.

[0095] The image processing system 202 provides a user-based publishing platform that enables a user to select a geographic location on a map and upload content associated with the selected geographic location. The user can also specify situations in which a particular media overlay should be provided to other users. The image processing system 202 generates a media overlay that includes the uploaded content and associates the uploaded content with the selected geographic location.

[0096] The augmented creation system 216 supports the augmented reality developer platform and includes applications for content creators (e.g., artists and developers) to create and publish augmentations (e.g., augmented reality experiences) for the interactive clients 106. The augmented creation system 216 provides content creators with a library of built-in features and tools, including, for example, custom shaders, tracking techniques, and templates.

[0097] In some examples, the enhancement creation system 216 provides a merchant-based publishing platform that enables merchants to select specific enhancements associated with a geographic location via a bidding process. For example, the enhancement creation system 216 associates the media overlay of the highest bidding merchant with the corresponding geographic location for a predefined amount of time.

[0098] The communication system 210 is responsible for enabling and processing various forms of communication and interaction within the interactive system 100, and includes a messaging system 212, an audio communication system 218, and a video communication system 214. The messaging system 212 is responsible for implementing temporary or time-limited access to content by the interactive client 106. The messaging system 212 includes multiple timers (e.g., within a transient timer system) that selectively enable access (e.g., for presentation and display) of messages and associated content via the interactive client 106 based on duration and display parameters associated with a message or a collection of messages (e.g., a story). Additional details on the operation of the transient timer system 220 are provided below. The audio communication system 218 enables and supports audio communication (e.g., real-time audio chat) between multiple interactive clients 106. Similarly, the video communication system 214 enables and supports video communication (e.g., real-time video chat) between multiple interactive clients 106.

[0099] The user management system 222 is operationally responsible for managing user data and profiles, and includes a social networking system 224 that maintains information about relationships between users of the interactive system 100 .

[0100] The collection management system 226 is responsible for managing collections or collections of media (e.g., collections of text, image video, and audio data) in operation. Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into "event galleries" or "event stories." Such collections can be made available within a specified time period (e.g., the duration of the event to which the content relates). For example, content related to a concert can be available as a "story" for the duration of the concert. The collection management system 226 can also be responsible for publishing an icon that provides notifications of a specific collection to the user interface of the interactive client 106. The collection management system 226 includes a curation function that enables collection managers to manage and curate specific content collections. For example, the curation interface enables event organizers to curate collections of content related to a specific event (e.g., deleting inappropriate content or redundant messages). In addition, the collection management system 226 uses machine vision (or image recognition technology) and content rules to automatically curate content collections. In some examples, compensation can be paid to users for including user-generated content in a collection. In such a case, the collection management system 226 operates to automatically pay such users to use their content.

[0101] The mapping system 228 provides various geolocation functions and supports the presentation of map-based media content and messages by the interactive client 106. For example, the mapping system 228 enables the display of a user icon or avatar (e.g., stored in the profile data 302) on a map to indicate the current or past locations of the user's "friends" within the context of the map, as well as media content (e.g., a collection of messages including photos and videos) generated by these friends. For example, on the map interface of the interactive client 106, messages posted by the user to the interactive system 100 from a particular geographic location can be displayed to the "friends" of the particular user within the context of that particular location on the map. The user can also share his or her location and status information with other users of the interactive system 100 via the interactive client 106 (e.g., using an appropriate status avatar), where the location and status information is similarly displayed to the selected user within the context of the map interface of the interactive client 106.

[0102] The gaming system 230 provides various gaming functions within the context of the interactive client 106. The interactive client 106 provides a gaming interface that provides a list of available games that can be launched by a user within the context of the interactive client 106 and played with other users of the interactive system 100. The interactive system 100 also enables a particular user to invite other users to participate in playing a particular game by sending an invitation to such other users from the interactive client 106. The interactive client 106 also supports audio, video, and text messaging (e.g., chatting) within the context of game play, provides leaderboards for games, and also supports the provision of in-game rewards (e.g., game coins and items).

[0103] The external resource system 232 provides an interface for the interactive client 106 to communicate with a remote server (e.g., a third-party server 112) to launch or access an external resource (i.e., an application or applet). Each third-party server 112 hosts, for example, an application or a small-scale version of an application based on a markup language (e.g., HTML5) (e.g., a game application, a utility application, a payment application, or a ride-sharing application). The interactive client 106 can launch a web-based resource (e.g., an application) by accessing an HTML5 file from a third-party server 112 associated with a web-based resource. The application hosted by the third-party server 112 is programmed in JavaScript using a software development kit (SDK) provided by the interactive server 124. The SDK includes an API with functions that can be called or activated by a web-based application. The interactive server 124 hosts a JavasScript library that provides access to a given external resource to specific user data of the interactive client 106. HTML5 is an example of a technology for programming games, but applications and resources programmed based on other technologies can be used.

[0104] In order to integrate the functionality of the SDK into the web-based resource, the SDK is downloaded from the interactive server 124 by the third-party server 112, or the SDK is received by the third-party server 112 in other ways. Once downloaded or received, the SDK is included as part of the application code of the web-based external resource. The code of the web-based resource can then call or activate certain functions of the SDK to integrate the features of the interactive client 106 into the web-based resource.

[0105] The SDK stored on the interactive system 100 effectively provides a bridge between external resources (e.g., applications 108 or applets) and the interactive client 106. This gives the user a seamless experience of communicating with other users on the interactive client 106 while also preserving the appearance of the interactive client 106. In order to bridge the communication between the external resources and the interactive client 106, the SDK facilitates the communication between the third-party server 112 and the interactive client 106. The WebViewJavaScriptBridge running on the user system 104 establishes two one-way communication channels between the external resources and the interactive client 106. Messages are sent asynchronously between the external resources and the interactive client 106 via these communication channels. Each SDK function activation is sent as a message and a callback. Each SDK function is implemented by constructing a unique callback identifier and sending a message with the callback identifier.

[0106] By using the SDK, not all information from the interactive client 106 is shared with the third-party server 112. The SDK limits which information is shared based on the needs of the external resource. Each third-party server 112 provides an HTML5 file corresponding to a web-based external resource to the interactive server 124. The interactive server 124 can add a visual representation (e.g., a box design or other graphics) of the web-based external resource in the interactive client 106. Once the user selects the visual representation or instructs the interactive client 106 to access the features of the web-based external resource through the graphical user interface of the interactive client 106, the interactive client 106 obtains the HTML5 file and instantiates the resource for accessing the features of the web-based external resource.

[0107] The interactive client 106 presents a graphical user interface (e.g., a login page or title screen) for an external resource. During, before, or after presenting the login page or title screen, the interactive client 106 determines whether the launched external resource has been previously authorized to access the user data of the interactive client 106. In response to determining that the launched external resource has been previously authorized to access the user data of the interactive client 106, the interactive client 106 presents another graphical user interface of the external resource including the functions and features of the external resource. In response to determining that the launched external resource has not been previously authorized to access the user data of the interactive client 106, after displaying the login page or title screen of the external resource for a threshold period of time (e.g., 3 seconds), the interactive client 106 slides up a menu (e.g., animating the menu to emerge from the bottom of the screen to the middle or other part of the screen) for authorizing the external resource to access the user data. The menu identifies the type of user data that the external resource will be authorized to use. In response to receiving a user selection of the accept option, the interactive client 106 adds the external resource to the list of authorized external resources and allows the external resource to access the user data from the interactive client 106. External resources are authorized by the interactive client 106 to access user data under the OAuth 2 framework.

[0108] The interaction client 106 controls the type of user data shared with the external resource based on the type of external resource that is authorized. For example, an external resource including a full-scale application (e.g., application 108) is provided with access to a first type of user data (e.g., a two-dimensional avatar of a user with or without different avatar characteristics). As another example, an external resource including a small-scale version of an application (e.g., a web-based version of the application) is provided with access to a second type of user data (e.g., payment information, a two-dimensional avatar of a user, a three-dimensional avatar of a user, and an avatar with various avatar characteristics). Avatar characteristics include different ways to customize the appearance of an avatar (e.g., different poses, facial features, clothing, etc.).

[0109] The advertising system 234 operatively enables third parties to purchase advertisements for presentation to end users via the interactive clients 106, and also handles the delivery and presentation of these advertisements.

[0110] Data Architecture

[0111] Figure 3 is a schematic diagram illustrating a data structure 300 that may be stored in a database 304 according to some examples. The database 304 may form part of the interactive system 100. Although the contents of the database 304 are shown as including a plurality of tables, it will be appreciated that the data may be stored in other types of data structures (e.g., as an object-oriented database).

[0112] Database 304 includes message data stored in message table 306. For any particular message, the message data includes at least message sender data, message recipient (or receiver) data, and payload. Fig.60 Additional details regarding information that may be included in a message and included within the message data stored in message table 306 are described.

[0113] The entity table 308 stores entity data and is linked (e.g., referenced) to the entity graph 310 and the profile data 302. The entities for which records are maintained within the entity table 308 may include individuals, corporate entities, organizations, objects, places, events, etc. Regardless of the entity type, any entity about which the interactive system 100 stores data may be an identified entity. Each entity is provided with a unique identifier and an entity type identifier (not shown).

[0114] The entity graph 310 stores information about the relationships and associations between entities. As examples only, such relationships may be social relationships, professional relationships (e.g., working in a common company or organization) based on interests or activities.

[0115] The profile data 302 stores multiple types of profile data about a particular entity. Based on the privacy settings specified by the particular entity, the profile data 302 can be selectively used and presented to other users of the interactive system 100. In the case where the entity is a person, the profile data 302 includes, for example, the user name, phone number, address, settings (e.g., notification and privacy settings), and an avatar representation (or a collection of such avatar representations) selected by the user. The particular user can then selectively include one or more of these avatar representations within the content of messages transmitted via the interactive system 100 and on a map interface displayed to other users by the interactive client 106. The collection of avatar representations can include a "status avatar" that presents a graphical representation of a status or activity that the user can choose to transmit at a particular time.

[0116] Where the entity is a group, profile data 302 for the group may similarly include one or more avatar representations associated with the group, in addition to the group name, members, and various settings for the relevant group (eg, notifications).

[0117] Database 304 also stores enhancement data, such as overlays or filters, in enhancement table 312. The enhancement data is associated with and applied to user input content such as videos (data for videos is stored in video table 314) and images (data for images is stored in image table 316).

[0118] In some examples, a filter is an overlay displayed as an overlay on an image or video during presentation to a recipient user. Filters can be of various types, including filters selected by a user from a set of filters presented to the recipient user by the interactive client 106 when the recipient user is composing a message. Other types of filters include geo-location filters (also referred to as geo-filters), which can be presented to the recipient user based on geolocation. For example, a geo-location filter specific to a nearby or special location can be presented by the interactive client 106 within a user interface based on geolocation information determined by a global positioning system (GPS) unit of the user system 104.

[0119] Another type of filter is a data filter, which may be selectively presented to the sending user by the interaction client 106 based on other input or information collected during the message creation process by the user system 104. Examples of data filters include the current temperature at a particular location, the current speed at which the sending user is traveling, the battery life of the user system 104, or the current time.

[0120] Other augmented data that may be stored in the image table 316 include augmented reality content items (eg, corresponding to application Lenses or augmented reality experiences). Augmented reality content items may be real-time special effects and sounds that may be added to an image or video.

[0121] As described above, augmented data includes augmented reality (AR), virtual reality (VR) and mixed reality (MR) content items, overlays, image transformations, images and modifications that can be applied to image data (e.g., videos or images). This includes real-time modifications that modify images when they are captured using the device sensor (e.g., one or more cameras) of the user system 104 and then display the modified images on the screen of the user system 104. This also includes modifications to stored content (e.g., video clips in a collection or group that can be modified). For example, in a user system 104 that can access multiple augmented reality content items, a user can use a single video clip with multiple augmented reality content items to see how different augmented reality content items will modify the stored clips. Similarly, real-time video capture can use modifications to show how the video image currently captured by the sensor of the user system 104 will modify the captured data. Such data can be simply displayed on the screen without being stored in memory, or the content captured by the device sensor can be recorded and stored in memory with or without modifications (or both). In some systems, a preview feature can simultaneously display how different augmented reality content items will look in different windows in the display. For example, this may enable multiple windows with different pseudo-random animations to be viewed simultaneously on the display.

[0122] Thus, data and various systems using augmented reality content items or other such transformation systems using the data to modify content may involve: detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.) in video frames, tracking of such objects as they leave, enter, and move around the field of view, and modification or transformation of such objects as they are tracked. In various examples, different methods for implementing such transformations may be used. Some examples may involve: generating a three-dimensional mesh model of one or more objects; and using transformations of the models and animated textures within a video to implement the transformations. In some examples, tracking of points on an object may be used to place an image or texture (which may be two-dimensional or three-dimensional) at the tracked location. In yet another example, a neural network analysis of a video frame may be used to place an image, model, or texture in content (e.g., an image or video frame). Thus, augmented reality content items involve both images, models, and textures used to create transformations in content, and additional modeling and analysis information required to implement such transformations using object detection, tracking, and placement.

[0123] Real-time video processing can be performed using any kind of video data (e.g., video streams, video files, etc.) stored in the memory of any kind of computerized system. For example, a user can load video files and store them in the memory of the device, or a sensor of the device can be used to generate a video stream. In addition, any object, such as a human face and parts of a human body, an animal, or a non-living thing (e.g., a chair, a car, or other objects) can be processed using computer animation models.

[0124] In some examples, when a specific modification is selected together with the content to be transformed, the element to be transformed is identified by a computing device, and then if the element to be transformed is present in a frame of the video, it is detected and tracked. The elements of the object are modified according to the modification request, thereby transforming the frame of the video stream. For different types of transformations, the frames of the video stream can be transformed by different methods. For example, for frame transformations that mainly involve changing the form of the elements of the object, characteristic points for each element of the object are calculated (for example, using an active shape model (ASM) or other known methods). Then, a grid based on the characteristic points is generated for each element of the object. The grid is used for subsequent stages of tracking the elements of the object in the video stream. During the tracking process, the grid for each element is aligned with the position of each element. Then, additional points are generated on the grid.

[0125] In some examples, the transformation of some areas of the object using the elements of the object can be performed by calculating characteristic points for each element of the object and generating a grid based on the calculated characteristic points. Points are generated on the grid, and then various areas based on these points are generated. Then, the elements of the object are tracked by aligning the area of ​​each element with the position of each element of at least one element, and the properties of the area can be modified based on the modification request, thereby transforming the frame of the video stream. Depending on the specific requirements of the modification, the properties of the mentioned area can be transformed in different ways. Such modifications may involve: changing the color of the area; removing at least part of the area from the frame of the video stream; including new objects in the area based on the modification request; and modifying or distorting the elements of the area or object. In various examples, any combination of such modifications or other similar modifications may be used. For certain models to be animated, some characteristic points can be selected as control points of the entire state space to be used to determine the options for model animation.

[0126] In some examples of computer animation models for transforming image data using face detection, faces are detected on an image using a specific face detection algorithm (e.g., Viola-Jones). An active shape model (ASM) algorithm is then applied to the face region of the image to determine facial feature reference points.

[0127] Other methods and algorithms suitable for face detection can be used. For example, in some examples, landmarks are used to locate features, which represent distinguishable points that exist in most of the images considered. For example, for facial landmarks, the location of the left pupil can be used. If the initial landmarks are not identifiable (for example, if the person is wearing an eye patch), secondary landmarks can be used. Such a landmark identification process can be used for any such object. In some examples, a set of landmarks forms a shape. The coordinates of the points in the shape can be used to represent the shape as a vector. One shape is aligned with another shape using a similarity transformation (allowing translation, scaling, and rotation) that minimizes the average Euclidean distance between the shape points. The mean shape is the average of the aligned training shapes.

[0128] The transformation system can capture an image or video stream on a client device (e.g., user system 104) and perform complex image manipulations locally on the user system 104 while maintaining an appropriate user experience, computational time, and power consumption. Complex image manipulations can include size and shape changes, emotion transformations (e.g., changing a face from a frown to a smile), state transitions (e.g., aging a subject, reducing apparent age, changing gender), style transformations, application of graphical elements, and any other suitable image or video manipulations implemented by a convolutional neural network that has been configured to execute efficiently on the user system 104.

[0129] In some examples, a computer animation model for transforming image data may be used by a system in which a user may capture an image or video stream of the user (e.g., a selfie) using a user system 104 having a neural network operating as part of an interactive client 106 operating on the user system 104. The transformation system operating within the interactive client 106 determines the presence of a face within the image or video stream and provides a modification icon associated with the computer animation model to transform the image data, or the computer animation model may exist in association with an interface described herein. The modification icon includes changes as a basis for modifying the user's face within the image or video stream as part of the modification operation. Once the modification icon is selected, the transformation system initiates a process of transforming the user's image to reflect the selected modification icon (e.g., generating a smiley face on the user). Once the image or video stream is captured and the specified modification is selected, the modified image or video stream may be presented in a graphical user interface displayed on the user system 104. The transformation system may implement a complex convolutional neural network on a portion of the image or video stream to generate and apply the selected modification. That is, a user can capture an image or video stream and once a modification icon has been selected, can be presented with the modified result in real time or near real time. Furthermore, when the video stream is being captured, the modification can be persistent and the selected modification icon remains toggled. Machine-taught neural networks can be used to implement such modifications.

[0130] The graphical user interface presenting the modifications performed by the transformation system can provide additional interactive options to the user. Such options can be based on the interface used to initiate the selection of a particular computer animation model and content capture (e.g., initiated from a content creator user interface). In various examples, the modifications can be persistent after the initial selection of the modification icon. The user can turn the modification on or off by tapping or otherwise selecting the face modified by the transformation system, and store it for later viewing or browsing to other areas of the imaging application. In the case of multiple faces modified by the transformation system, the user can globally turn the modification on or off by tapping or selecting a single face modified and displayed within the graphical user interface. In some examples, each face in a group of multiple faces can be modified individually, or such modifications can be individually switched by tapping or selecting a single face or a series of faces displayed within the graphical user interface.

[0131] The story table 318 stores data about a collection of messages and associated images, video or audio data that are compiled into a collection (e.g., a story or gallery). The creation of a particular collection can be initiated by a particular user (e.g., each user for whom a record is maintained in the entity table 308). A user can create a "personal story" in the form of a collection of content that has been created and sent / broadcasted by that user. To this end, the user interface of the interactive client 106 can include a user-selectable icon to enable the sending user to add specific content to his or her personal story.

[0132] The collection may also constitute a "live story", which is a collection of content from multiple users created manually, automatically, or using a combination of manual and automatic techniques. For example, a "live story" may constitute a curated stream of user-submitted content from different locations and events. Users whose client devices have location services enabled and who are at a co-located event at a particular time may be presented with the option of contributing content to a particular live story, for example, via the user interface of the interactive client 106. Live stories may be identified to the user by the interactive client 106 based on his or her location. The end result is a "live story" told from a group perspective.

[0133] Another type of content collection is called a "location story" that enables users whose user systems 104 are located within a specific geographic location (e.g., on a college or university campus) to contribute to a specific collection. In some examples, contributions to location stories may require secondary authentication to verify that the end user belongs to a specific organization or other entity (e.g., is a student on a university campus).

[0134] As mentioned above, video table 314 stores video data that, in some examples, is associated with messages for which records are maintained within message table 306. Similarly, image table 316 stores image data associated with messages whose message data is stored in entity table 308. Entity table 308 may associate various enhancements from enhancement table 312 with various images and videos stored in image table 316 and video table 314.

[0135] In some examples, if a user is viewing a user input content item, the user may wish to combine the user input content item with another content item. However, this may involve several manual steps or selections, so it may be desirable to reduce and / or automate these steps or selections at least to some extent. An example technical problem that reduces and / or automates these steps or selections can be solved by providing a reply user interface that automatically provides access to a combination function (also referred to as a "remix" function). In some examples, if a user is viewing a first user input content item (e.g., a third-party image), the user is allowed to combine the first user input content item with a second user input content item (e.g., an image selected by the user), and reply to the first user input content item (e.g., "remix and reply"), the user can select a camera button to trigger the following experience, which includes the ability to access enhanced functions or tools, in the form of a remix lens. The remix lens (also referred to as a combined lens or a combined application element) allows the user to combine different user input content items to create a combined user input content item, which the user can then share and "reply" to the sharing of the first user input content item (e.g., a third-party image).

[0136] Figure 4 is a user interface diagram depicting a user interface 404 according to some examples that may be presented by the interactive client 106 in the example form of a media application executing on the mobile device 114. Although the user interface 404 is described as being presented on a screen of the mobile device 114, the user interface 404 may also be presented on an optical display of a head wearable device 116 or other wearable device presenting a user interface (e.g., a "smart contact lens" or similar technology).

[0137] The interactive system 100 allows users of the interactive system 100 to share and publish digital content, including various user-input content items. The user-input content items may include user-selected content items (which may be user-selected content data), user-generated content items (which may be user-generated content data), computer-generated content items (which may be computer-generated content data, such as data generated by an artificial intelligence-driven system), or a combination thereof.

[0138] The user interface 404 displays user input content items in the form of examples of third-party user input content items 406, which in some examples may be part of a collection of third-party user input content items generated and published by third-party users of the interactive system 100. The third-party user input content items 406 may, for example, include image data, video data, audio data, or a combination of these data types, along with enhancement or modification data (e.g., stickers, filters, or lenses).

[0139] The third party user input content item 406 may be part of a set of third party user input content items created by a third party user of the media application, and the presentation of the third party user input content item may be part of a sequential presentation of the set of third party user input content items.

[0140] The user interface 404 also includes an interactive area 402 having a functional or tool portion, exemplified in the form of a functional carousel 408. The functional carousel 408 presents various tools and functions that a user of the mobile device 114 can invoke to interact with other users of the interactive system 100 and also to interact with content accessible via the interactive system 100. The interactive area 402 includes a combining graphical element 410 (e.g., a user-selectable icon) that a user can select to invoke a combining function that enables the user to combine the third-party user input content item 406 with another content item (e.g., an image or video). The user can combine the third-party user input content item 406 with another (second) user input content item selected by the user, e.g., the second user input content item is captured using a camera of the mobile device 114; selected from a local storage of the mobile device 114, a cloud storage location, or another storage component; or generated in part or in whole by a computer system or artificial intelligence. Various examples of the user interface 404 are discussed below. Although certain examples of the present disclosure relate to combining a third-party user-input content item with another content item, it should be understood that similar techniques may be applied to combine any user-input content item (not necessarily a third-party user-input content item) with another content item.

[0141] Figure 5 1 is a flow chart illustrating a method 500 according to some examples that enables an active user to combine a third-party user-input content item with another user-input content item in a media application (eg, the interactive client 106 ). Thus, the method 500 may be performed by a component of the interactive system 100 .

[0142] exist Figure 5 In the drawings and other figures of the present disclosure, for ease of reference, the term “box” is used to refer to elements in the drawings, and it can be understood that each “box” may identify one or more operations, processes, actions or steps.

[0143] Method 500 starts at opening loop frame 502, and proceeds to frame 504, at frame 504, interactive client 106 causes third-party user input content item 406 to be presented in user interface 404 of mobile device 114. Third-party user input content item 406 can be part of a collection of third-party user input content items published by third-party users of interactive system 100. One or more third-party users can have created or generated one or more of these items, and can have selected one or more of these items from local or cloud storage. In such an example, the active user of interactive client 106 can scroll in both backward and forward directions (or upward and downward directions) in the collection of third-party user input content items within the context of user interface 404.

[0144] At block 506, the remix system 206 of the interactive system 100 detects a third-party authorization for combining the third-party user input content item 406 with another content item to create a combined user input content item. For example, an authorization setting associated with the third-party user input content item 406 may be detected at block 506 to indicate the authorization. Thus, determining the third-party authorization of the third-party user input content item to be combined may include determining a user setting associated with the third-party user input content item indicating the third-party authorization.

[0145] When a third-party content creator creates or publishes a third-party user input content item, the third-party content creator is presented with the option of allowing or not allowing the third-party user input content item 406 to be combined with other content items created by or originating from other users of the interactive system 100. The allow / disallow is then recorded and stored in the database 128 as an authorization setting associated with the relevant third-party user input content item, and is accessed at block 506 to perform a check for third-party authorization.

[0146] At block 508, in response to determining the third-party authorization, the interactive client 106 causes the combined graphical element to be automatically displayed or presented on the user's mobile device 114. Causing the combined graphical element to be automatically presented includes causing the combined graphical element to be presented without user action, such as with respect to the user interface 404, as may be presented by the interactive client 106. The combined graphical element may be selected by the user to invoke a combined function provided by the remix system 206 of the interactive system 100. In some examples, the combined graphical element is a camera indicia (e.g., a camera button) that indicates that the user may select to invoke the combined function. The combined graphical element is further caused to be combined with the third-party user input content item 406 and displayed on the user interface 404 of the interactive client 106 simultaneously with the third-party user input content item 406.

[0147] The combination function can provide several sub-functions or tools to facilitate the combination of the third-party user input content item 406 with other user input content items. These tools can include a layout function that enables the user to specify the layout relationship of the third-party user input content item 406 relative to other user input content items during the presentation of the combined user input content item, or the layout relationship within the broader combined display context. The combination function also enables several enhancement functions of the enhancement system 208 discussed herein to be applied and used when creating a combined content element.

[0148] In some examples, in addition to being able to realize the combination of multiple content items, the combination function can also trigger a reply message function (for example, provided by the message transceiving system 212) to realize the automatic message transmission and reception or sending of the combined user input content item to the third-party user of the interactive system 100. The sending of the combined user input content item can be sending a message (such as message 6000) to the third-party user, or can be selectively published in the content feed of the third-party user. Therefore, the reply message function can allow two users to collaborate on the content item together to create the combined content item. In some examples, this collaboration can occur in a private conversation. Therefore, the user's reply to the sending of the third-party user input content item can include sending the combined user input content item in a private conversation between the user and the third-party user that the interactive system 100 facilitates.

[0149] At block 510, a user selection of a combined graphical element is detected, and at block 512, in response to determining the user selection, a combining function provided by the remix system 206 is called. The combining function enables the active user to combine the third-party user input content item 406 with another user input content item to create a new combined user input content item. The method 500 then terminates at program block 514.

[0150] Although the examples herein refer to combining digital content in the form of images and videos, it should be understood that an audio item may also be combined with another content item (which may include images, videos, and / or audio content). For example, a viewing user may select a third-party user-input content item that includes an audio item (e.g., a specific song linked to an image or video clip), and use the combine function to combine the audio item with the other content item to create a combined user-input content item (e.g., combining the specific song with another image captured or selected by the user).

[0151] Figure 66 is a user interface diagram illustrating a user interface 606 according to some examples. Referring to the user interface 606, an active user is viewing a third-party user-input content item in the example form of a third-party image 604 (another example may be a third-party video clip, or an image / video with additional audio content), and in response to determining that a combining authorization is in respect of the third-party image 604, a combining graphical element in the form of a camera button 602 is displayed. The camera button 602 may be selected by the user to trigger a combining functionality (e.g., an experience) that includes the ability to access combined enhancements or shots (e.g., a "remixed" shot) to create a combined content element (e.g., a combined user-input content item) that includes the third-party image 604.

[0152] Figure 7 702 according to some examples. Referring to the user interface 702, in some examples, the active user is viewing a reference Figure 6 The third-party image 604, and in response to determining the combination authorization regarding the third-party image 604, displaying a combination graphical element in the form of a remix button 704.

[0153] The remix button 704 can be selected by the user to trigger the combined function. The remix button 704 is different from the camera button 602 in that the remix button 704 specifically includes a graphic component (e.g., Figure 7 The graphic component identifies the combined function, including its reply message function. The graphic component can provide a contextual signal to the user, indicating the option to trigger the combined function and its reply message function. Figure 7 The arrows shown in are merely examples, and it should be understood that examples of the present disclosure may include various identifiers of the combined function, such as various enhancements or supplements to the camera button to remind the active user of the option to activate the combined function.

[0154] Figure 8 802 according to some examples. When an activity viewer is viewing the recorded and accessed combined authorized third-party images 604 within the user interface 802, a user selection of the camera button 602 opens a modular reply camera interface 904 (e.g., provided by a reply system of the camera system 204), such as Fig. 9 Thus, invoking the combined function (by selecting the camera button 602 ) may include activating and opening the modular restore camera interface 904 .

[0155] Reference Fig. 9, the reply camera interface 904 presents an enhanced carousel 906. Within the enhanced carousel 906, a combined graphical element in the form of a remixed shot 908 (also referred to as a combined shot or combined application element) is located at a first position relative to the other shots within the enhanced carousel 906 and is pre-selected (e.g., indicated by visual highlighting and / or size distinction and / or its central position in the reply camera interface 904). The enhanced carousel 906 is automatically applied to the merging or combination of the third-party image 604 with the second user input content item (in this case, an image captured by the active user) to create the combined image 902 within the reply camera interface 904. Thus, in some examples, invoking the combine function causes the modular reply camera interface to be displayed and includes pre-selection of the combined shot (combined application element) by the media application, enabling the user to easily access the remix function.

[0156] Fig.10 1002 according to some examples. When an active user views a third-party image 604 that stores and accesses a combined authorization within the viewing user interface 1002, as shown in FIG. Fig.11 As shown, user selection of the camera button 602 activates and opens the modular reply camera interface 1104 (eg, provided by the camera system 204 ).

[0157] Fig.11 1 is a user interface diagram illustrating a viewing user interface 1004 according to some examples. The rewind camera interface 1104 presents a user input content image 1102, such as a "selfie" image captured by an active user (e.g., a self-portrait photo captured using a front-facing camera), which may be captured in response to a user selection of the camera button 602. The rewind camera interface 1104 automatically and by default presents an enhancement carousel 1106 within which a remix camera 1108 (or remix shot, also referred to as a combined shot) is the first shot in the enhancement carousel 1106, but is not identical to the reference shot. Figure 8 and Fig. 9 In contrast to the example discussed above, this lens is not pre-selected. The remix camera 1108 is referred to as the first lens in the enhancement carousel 1106 because it is closest to the central capture button 1112. The remix camera 1108 is not pre-selected, and the central capture button 1112, which can be used to capture a standard image (without remixing or enhancement), is pre-selected and displayed as larger than the icon in the enhancement carousel 1106.

[0158] The active user can then scroll the enhanced carousel 1106 to the left to move the remix camera 1108 to the center position (see Fig.12), or by directly pressing the remix camera 1108 to select the remix camera 1108 to activate the remix camera 1108. Using the combining function of the remix camera 1108, the active user can conveniently and easily combine the third-party image 604 of the third-party user of the interactive system 100 (such as "Sally" in this example) with the user input content image 1102 (such as a selfie captured by the active user using the remix camera 1108 or a selfie selected from the camera roll of the mobile device) to create a combined user input content item. In this example, the combined user input content item is in the form of a combined image 1202 (see Fig.12 , which will be described below).

[0159] Fig.12 1 is another user interface diagram showing the reply camera interface 1104 with the combined image 1202. Fig.12 As shown, the reply camera interface 1104 may also include a set of layout graphic elements 1204, each of which may be selected by the user to apply a different layout to the combination of the third-party image 604 and the user input content image 1102. Each of the layout options associated with the corresponding layout graphic element 1204 may assign portions of the interface canvas to each of the images, or may define the layout of the images relative to each other. The same layout options may be applied to still images and video content, or may be available with respect to still images and video content. In some examples, the combined user input content item may include a still image combined with video content, and a variety of layout options may also be used.

[0160] Fig.12 The combined image 1202 in FIG. 1 uses a layout option in which the image of the active user is superimposed on the third-party image 604 (which essentially becomes the background image). Another example may be a "split screen remix" in which the image of the active user is shown in a first portion or frame and the third-party image is shown in a second portion or frame separate from the first portion or frame within the interface.

[0161] The reply camera interface 1104 also includes a reply graphic element 1110 that provides contextual information about the composition function (e.g., in this example, indicating that the active user is creating a composite image 1202 intended for use in replying to a third-party user "Sally"). The reply graphic element 1110 can also be selected by the user to invoke the messaging system 212 of the interactive system 100 and conveniently enable the active user to send (e.g., via a message or posting to a feed) the composite image 1202 to the third-party user (e.g., "Sally") by "replying" to the third-party posted third-party image 604.

[0162] Thus, the combined functionality provided by the media application may include a reply functionality that enables a user to send a combined user input content item to a third party user of the media application by replying to the sending of the third party user input content item.

[0163] Figures 13 to 16 A user interface diagram showing a series of user interfaces 1302 in sequence according to some examples is shown. Here, when the user selects the camera button 602 ( Fig.13 ), the enhanced carousel 1502 is not automatically generated and the enhanced carousel 1502 is not presented in the user interface 1302.

[0164] Instead, the carousel icon 1402 is displayed adjacent to a central (and pre-selected) image capture graphical element 1404, which a user may select to capture an image using the camera of the mobile device 114. Fig.14 shown.

[0165] like Fig.15 As shown, the carousel icon 1402 can be selected by a user to generate a display of an enhancement carousel 1502 in which the remixed camera device 1504 or combined graphical element is the first content enhancement option (e.g., "lens") presented within a sequence of enhancement options in the enhancement carousel 1502.

[0166] like Figures 14 to 16 As shown, a reply graphical element 1406 as described elsewhere is provided in the upper area of ​​the user interface 1302.

[0167] The user is also provided with various options for obtaining layout aspects of two content items relative to each other via layout graphical element 1602 as described elsewhere, and according to Fig.16 Some examples in are shown.

[0168] Fig.17 1 is a user interface diagram illustrating a viewing user interface 1702 according to some examples that is presented when an active viewing user of an interactive client 106 views a third-party image 604. In this example, a tool tip 1704 is generated and displayed in conjunction with a remix button 1706 (in the Fig.17, the remix button 1706 is a supplemental camera button). When the user clicks on the remix button 1706 or when the user initially views the third-party image 604, the tool tip 1704 may be displayed and presented. The tool tip 1704 may also be presented for a set time (e.g., between 2 and 10 seconds and then disappear from the viewing user interface 1702) and may be presented to the user a set number of times (e.g., up to three times) within any interactive session with an interactive client.

[0169] In some examples, if a user is viewing a user input content item, such as a third-party image, to which an enhancement (e.g., a lens, filter, or other enhancement applied to the third-party image) has been applied, the user may wish to apply the same enhancement, but this may require several manual steps / selections. Methods and systems according to examples of the present disclosure may solve an example technical problem as follows: automatically provide access to camera functions together with content combination functions and / or content enhancement functions (e.g., enhancement functions that allow viewing users to access and automatically apply the same enhancements applied to third-party images), that is, provide improved reply camera functions. The user may select a camera button to trigger a function (e.g., an experience) that provides the viewing user with convenient access to the same enhancements applied to the third-party image and applies the enhancements to the user input content item. According to some examples, the viewing user interface includes an interactive area that provides a top-level camera button that is displayed simultaneously with digital content (e.g., a user input content item), which is also displayed within the context of the viewing user interface. The camera button is supplemented or modified to indicate that a user selection of the camera button triggers a specific function (e.g., an "experience", such as a remix experience or a lens experience). The camera button may be supplemented or modified in various ways, including highlighting the camera button with an appropriate thumbnail image (e.g., a lens thumbnail image or a remix thumbnail image) associated with a functionality or user experience invoked in response to selection of the modified camera button. In addition to opening a modular reply camera, selection of the modified camera button invokes an associated functionality. In other words, according to some examples, user selection of the modified camera button invokes both a camera functionality (e.g., a "reply" camera functionality, which is a camera functionality within a broader reply context and then applicable to user input content items posted by a third party) and additional functionality used with the camera functionality (e.g., a content combining functionality or a content enhancement functionality).

[0170] Fig.181 is a user interface diagram depicting a viewing user interface 1802 according to some examples that may be presented by the interactive client 106 in the example form of a media application executing on the mobile device 114. Although the viewing user interface 1802 is described as being presented on a screen of the mobile device 114, the viewing user interface 1802 may also be presented on an optical display of a head wearable device 116 or other wearable device that presents a user interface (e.g., a "smart contact lens" or similar technology).

[0171] The viewing user interface 1802 displays third-party user input content items 1804, which in some examples may be part of a collection of third-party user input content items generated and published by third-party users of the interactive system 100. The third-party user input content items 1804 may include one or more content data including image data, video data, audio data, or a combination of these data types, as well as enhancement or modification data (e.g., stickers, filters, or lenses).

[0172] The viewing user interface 1802 includes an interactive area 1806 that includes an enhance camera graphical element 1808 (e.g., a user-selectable camera button 2102), which the user can select to invoke a reply camera function that enables the user to respond to a third-party user input content item 1804 posted by a third-party user. The response may include a user input content item (e.g., a digital image captured using a camera of the mobile device 114). In addition, the invoked reply camera function may enable the user to enhance the user input content item that the user wishes to add or include with enhancements applied to the third-party user input content item 1804. To this end, user selection of the enhance camera graphical element 1808 causes the camera user interface 1902 to be displayed, as shown in FIG. Fig.19 as shown as part of the reply camera function.

[0173] Fig.19 1 is a user interface diagram depicting a camera user interface 1902 according to some examples that may be presented by the interactive client 106 in the example form of a media application executing on the mobile device 114. The camera user interface 1902 may present a live feed from a camera of the mobile device 114, which the user may then capture as a still digital image or a digital video by selecting an appropriate button within the interactive area 1806 of the camera user interface 1902.

[0174] For example, the camera user interface 1902 displays a second user input content item 1904, such as a digital image or digital video captured by a camera of the mobile device 114. The second user input content item 1904 may include image data, video data, audio data, or a combination of these data types and enhancement or modification data (e.g., stickers, filters, or lenses).

[0175] The camera user interface 1902 also includes an interactive area 1806 having a function or tool portion in the exemplary form of a function carousel 1906. The function carousel 1906 presents various tools and functions that a user of the mobile device 114 can invoke to interact with other users of the interactive system 100, to interact with content accessible via the interactive system 100, and to modify a captured second user input content item 1904 displayed within the camera user interface 1902. The interactive area 1806 includes a content enhancement selection graphical element 1908 (e.g., a user selectable icon) that a user can select to invoke an enhancement function from a set of enhancement functions. The selected enhancement function can be used to enhance or modify the second user input content item 1904.

[0176] Fig. 20 is a flow chart illustrating a method 2000 that enables an active viewing user of the interactive system 100 to apply content enhancements previously applied by a third-party user to a third-party user's input content item, according to some examples.

[0177] The method 2000 begins at an opening loop block 2002 and then proceeds to block 2004, where a media application in the example form of an interactive client 106 of the interactive system 100 causes third-party user input content items 1804 to be presented within a viewing user interface 1802 of the mobile device 114. The third-party user input content items 1804 may be part of a set or collection of third-party user input content items posted by a third-party user of the interactive system 100. In such an example, an active user of the interactive client 106 is able to scroll in both forward and backward directions through the collection of third-party user input content items within the context of the viewing user interface 1802.

[0178] At block 2006, the enhancement system 208 of the interactive system 100 detects or determines a first content enhancement of the third-party user input content item 1804, the first content enhancement being an output of a first content enhancement function of the interactive system 100. As described above, the enhancement system 208 can provide a number of enhancement or modification functions that a user can invoke to enhance a digital content item. At block 2006, a specific first content enhancement (e.g., a specific filter or lens) applied to the third-party user input content item 1804 (e.g., a digital image posted on the interactive system 100 by a third party) can be identified from metadata associated with the third-party user input content item and accessed and read by the enhancement system 208. For example, the detected content enhancement function can be an image lens function or an image filter function.

[0179] At block 2008, in response to determining or detecting the first content enhancement at block 2006, the interactive client 106 automatically (e.g., without requiring a specific user invocation, or without requiring a user operation of a user interface of a media application) causes an enhanced camera graphical element 1808 to be presented or displayed on the mobile device 114. The enhanced camera graphical element 1808 may be selected by the user to invoke a reply camera function of the interactive client 106, and further causes a content enhancement selection graphical element 1908 to be presented, which may be selected by the user to invoke the associated first content enhancement function (e.g., the particular enhancement that was applied to the third content item was detected at block 2006).

[0180] At block 2010, the interactive client 106 detects or determines a user selection of the enhanced camera graphical element 1808, and then invokes a camera function of the interactive client 106 at block 2012. At block 2014, the camera function enables the user to select or capture a user input content item (which may be referred to as a "second user input content item" and the third user input content item may be referred to as a "first user input content item"), and at block 2016, causes the content enhancement selection graphical element 1908 to be presented. Fig.19As noted, the camera function presents a camera user interface 1902 that allows the active user to capture and select the second user input content item 1904 by selecting a capture icon within the interactive zone 1806. Additionally, a content enhancement selection graphical element 1908 is presented within the function carousel 1906 of the interactive zone 1806 of the camera user interface 1902. The content enhancement selection graphical element 1908, in the example form of a lens icon, corresponds to the first content enhancement of the third-party user input content item detected at block 2006. User selection of the content enhancement selection graphical element 1908, which is associated with a particular enhancement function (e.g., lens) of the interactive client 106 within the camera user interface 1902, invokes the associated enhancement function and conveniently allows the active user to apply the associated enhancement to the second user input content item 1904 within the context of the camera user interface 1902.

[0181] In the case where multiple enhancements are applied to the third-party user input content item 1804, the enhancement system 208 of the interactive system 100 can detect each of these content enhancements, in which case multiple content enhancement selection graphical elements 1908 corresponding to the detected multiple enhancements can be presented in the function carousel 1906 for user selection to conveniently and selectively apply each of these multiple enhancements to the second user input content item 1904. The method 2000 then ends at the closed-loop block 2018.

[0182] Fig.21 2 is a user interface diagram illustrating a viewing user interface 2104 according to some examples. The viewing user interface 2104 shows a photo of a cat with enhancements in the form of a lens applied to the photo. The lens superimposes a first enhancement 2110 in the form of sunglasses on the cat's eyes and a second enhancement 2112 in the form of the words "Deal with it" on the image.

[0183] View user interface 2104 has an interactive area 2106 that includes a camera graphical element in the form of an example camera button 2102. Camera button 2102 invokes lens access experience 2108 (e.g., Fig.21 ), where an active user of an interactive client 106 can apply a lens (e.g., as shown in FIG. 1 ) to a third-party user input content item (e.g., a cat photo) Fig.21 The sunglass lenses with the words “Deal with it” shown in ) are applied to another second user input content item, such as a digital image captured by the active user using the camera function of the interactive client 106.

[0184] Accordingly, user selection of the camera button 2102 opens a modular reply camera of the interactive client 106 within which the sunglasses and associated text lenses are pre-selected or presented for easy user selection to apply to another content item.

[0185] exist Fig.21 , two enhancements 2110, 2112 are applied as part of the same lens. However, it will be appreciated that, for example, the sunglasses enhancement and the "deal with it" text enhancement can be two separate lenses, in which case they can be presented separately to facilitate user selection (e.g., the user may wish to select only one of the lenses).

[0186] Fig. 22 is a user interface diagram illustrating a viewing user interface 2202 according to some examples. Fig. 22 An example is shown in which enhancement 2204 in the form of an example of a lens is detected and applied to a third-party user input content item within viewing user interface 2202 (in Fig. 22 Based on the detection, a modified camera button in the form of a lens button 2206 is generated that identifies the detected lens and causes it to be displayed within the interactive area of ​​the viewing user interface 2202.

[0187] In this example, the lens button 2206 is a lens thumbnail that reminds the user that selecting the lens button 2206 will activate the functions described below. Thus, the lens thumbnail indicates the content enhancement functions that have been used by the third party.

[0188] Now go to Fig.23 , Fig.23 206 invokes the lens access functionality, in which regard the modular back camera functionality is activated, causing the back camera user interface 2302 to be displayed. The back camera user interface 2302 includes a lens carousel 2304 (enhanced carousel) within the interactive area that is populated with a scrolling set of lensable icons or buttons, including a lens button 2306 that is associated with a detected third-party user input content item ( Fig. 22 In this example, the lens button 2306 is pre-selected, and the associated enhancement function (e.g., enhancement 2308) is automatically applied to the user input content item displayed in the camera user interface 2302. Specifically, as shown in FIG. Fig. 22 As shown, enhancement 2204 is a dog filter applied to the human in the image and Fig.232302, the enhancement 2308 is also a dog filter, in this case applied to a single human being shown in the camera user interface 2302. Thus, in some examples, the content enhancement function used by a third party can be pre-selected in the reply camera function, allowing an enhanced version of the second user input content item to be automatically created and the content enhancement to be detected as having been applied by the third party. This allows the user to conveniently and automatically access the same enhancements used by third-party users of the interactive system 100.

[0189] A remix button 2310 is shown in a second adjacent position within shot carousel 2304, and a user may select remix button 2310 to invoke a remix function, as described elsewhere.

[0190] Fig.24 is a user interface diagram illustrating a viewing user interface 2402 according to some examples. Fig.24 An example is shown in which an enhancement 2404 in the form of an example of a lens is detected as having been applied to a third-party user input content item within the viewing user interface 2402. Fig.24 Based on the detection, a modified camera button in the form of an example of a lens button 2206 corresponding to the detected lens is generated and displayed in the interactive area of ​​the viewing user interface 2402.

[0191] The lens button 2206 may be a generic lens thumbnail, or may be a thumbnail of a specific lens or enhancement that is detected as having been applied by a third party. For example, if the third-party user input content item is detected as including a dog filter, the lens button 2206 may be generated or adapted to graphically illustrate the dog filter, e.g., through a suitable thumbnail or dog image. In this way, the user may be more easily reminded of the option to apply the same enhancement as that used by the third party.

[0192] User selection of the lens button 2206 invokes the lens access function, wherein the modular replay camera is activated and causes the replay camera user interface 2502 to be displayed.

[0193] Fig.25 A rewind camera user interface 2502 is shown according to some examples. The rewind camera user interface 2502 includes a shot carousel 2504 within an interactive area that is automatically presented and populated with a scrollable set of shot icons or buttons, including a shot button 2506 and a remix button 2508. A standard camera capture button 2510 is pre-selected. In this example, no lens or enhancement is pre-selected, and the shot button 2506 is displayed in a first position within the shot carousel 2504, while the remix button 2508 is displayed in a second position within the shot carousel 2504.

[0194] As described above, the remix button 2508 is displayed only if the third-party user input content item viewing the user interface 2402 is properly permitted for use in creating a combined user input content item (eg, authorized by the third-party content creator for use in the remix functionality).

[0195] Thus, in addition to allowing a user to conveniently use lenses or enhancements used by a third party, the modular back-to-camera interface may also include (e.g., in its enhancement or lens carousel) a composition graphical element. The composition graphical element (e.g., remix button 2508) may be selected by the user to invoke a composition function or "remix" function, as described elsewhere.

[0196] Fig.26 is a user interface diagram illustrating a viewing user interface 2602 according to some examples. Fig.26 An example is shown in which enhancement 2604 in the example form of a lens is detected as having been applied to a third-party user input content item within viewing user interface 2402. Based on the detection, a modified camera button in the example form of a lens button 2206 is generated that corresponds to and identifies the detected lens and is displayed within the interactive area of ​​viewing user interface 2602.

[0197] While the examples herein refer to enhancements such as those applied to a lens of a digital image or video, it should be understood that in some examples, an "enhancement" may also refer to an audio item added to the content. For example, a viewing user may select a third-party user-input content item that includes an audio item (e.g., a particular song linked to an image or video clip) and use a modified camera button to combine the audio item with, or add the audio item to, a user-input content item selected or created by the user.

[0198] Go to Fig. 27 The user interface diagram shown is Fig.262206 invokes a lens access function, wherein the modular reply camera is activated and causes the reply camera user interface 2704 to be displayed. The reply camera user interface 2704 includes a lens carousel 2706 within the interactive area, which is automatically presented and populated with a scrollable set of lens icons or buttons, including a lens button 2708 and a remix button 2710. In this example, in addition to determining the enhancements 2604 to be applied to the third-party user input content item, a third-party authorization for using the third-party user input content item in generating the combined user input content item is also detected. Here, the remix button 2710 is automatically pre-selected, and a corresponding remix function (also referred to as a combination function) is applied to the third-party user input content item and the user-selected content item to display the combined user input content item within the reply camera user interface 2704. The user can then scroll the lens carousel 2706 and select the lens button 2708 to apply the corresponding lens to the combined user input content item displayed within the reply camera user interface 2704.

[0199] It is noteworthy that reply camera user interface 2704 also includes reply graphical element 2712 (indicated by the words "Reply SALLY"), which can be selected by the user to invoke a reply message function, whereby the combined user input content item displayed in camera user interface 2704 can be communicated to the third party user who generated and published the third party user input content item displayed in viewing user interface 2602. Example messaging functionality is described with reference to messaging system 212. Fig. 27 Also shown is a set of layout graphic elements 2702 as described above.

[0200] A user of a media application may wish to create a combined user input content item and / or an enhanced user input content item in a preview state, and then share these items with other users. However, a media application may only allow the user to combine and / or enhance certain items after sharing or publishing occurs. Alternatively, such a combination and / or enhancement may involve several manual steps or selections, if any. According to an example of the present disclosure, an example technical problem of providing a switchable or iterative preview and combination / enhancement state on a computer device can be solved by providing a user interface that allows the user to combine, contrast or enhance multiple user input content items. According to some examples, a method and system for implementing a combination (e.g., remix) of user input content items in a series of interfaces are provided, which allow the user to combine different user input content items generated by a single user or by multiple users of an interactive system into a combined user input content item. For example, a user can enhance, contrast or contextualize a first user input content item with a second user input content item or even multiple user input content items by switching between a preview interface and a combination interface any number of times before publishing the combined user input content item.

[0201] Fig.28 28 is an interface sequence diagram illustrating interface sequences 2802 according to some examples, which may be presented by an interactive client 106 hosted and executed by a user system 104 .

[0202] The sequence 2802 includes a camera interface 2804 that enables a user of the user system 104 to create a user input content item in the exemplary form of a digital image or digital video. Specifically, the camera interface 2804 is presented by the camera system 204 of the interactive system 100 and presents a viewing and capturing mechanism. The viewing and capturing mechanism includes a viewing window 2806 that presents a real-time image feed from any one of a plurality of cameras of the user system 104 to the user and a capture button 2808 that can be selected by the user to allow the user to capture one or more images from the image feed as a digital image or digital video. The interactive area 2810 presents a number of enhancements or modifications supported by the enhancement system 208 that can be applied in real time to the digital image presented within the viewing window 2806. The camera interface 2804 also presents a number of camera controls 2812 related to the capture of the user input content item, such as a camera selection mechanism, a flash activation mechanism, and the like.

[0203] The preview interface 2814 is also displayed by the camera system 204 of the interactive system 100 after capturing the user input content item. In some examples, the user input content item can be selected from a media library or file storage and can be presented via the preview interface 2814 without the user having captured the user input content using the camera (or the user has captured or created the user input content item at an earlier point in time and stored the item for later use). The preview interface 2814 presents a preview of the captured or selected user input content item in a preview window 2816. The preview interface can present the content item to the user for review, modification, and / or enhancement before finalizing and sharing the content item, for example, sharing the content item to a user's content collection accessible to a third party, sharing to a third-party user via a direct message, sharing to a shared content collection, or adding to a public content collection.

[0204] The interactive area 2810 presents a plurality of tools (e.g., tools of the enhancement system 208) to enable the user to enhance or otherwise modify the user input content item to generate an enhanced user input content item. The preview interface 2814 also presents a send button 2818 to invoke the communication system 210 of the interactive system 100 and send the user input content item presented in the preview window 2816 using the communication system 210 of the interactive system 100. The preview interface 2814 also includes a remix button 2820 that the user can select to invoke a content combining function supported by the remix system 206, which is a subsystem of the camera system 204.

[0205] A remix interface 2822 (also referred to as a combination interface) is also presented by the camera system 204, and in particular the remix system 206, to enable a user to capture supplemental user input content items and combine the supplemental user input content items with previously captured user input content items to generate a combined user input content item within a combination window 2824. The remix interface 2822 presents various layout tools 2826 for laying out multiple user input content items relative to each other within a particular layout, as well as a number of camera controls 2812. The interactive area 2810 of the combination window 2824 presents various enhancement and modification tools that can be applied to the combined content within the context of the remix interface 2822.

[0206] In some examples, once the combined user input content item is created, the user may be presented with a preview interface 2814 again that shows the combined user input content item and enables the user to combine the combined user input content item with another user input content item to create another combined user input content item. For example, instead of selecting the send button 2818 in the preview state, the user may select the remix button 2820 to switch back to the remix interface 2822 and further "remix" the content.

[0207] about Fig.28 Additional details of some examples of the interface will be combined Fig.30 Have a discussion.

[0208] Fig.29 2900 is a flowchart illustrating a method 2900 according to some examples for facilitating the generation of a combined user input content item. Although some examples are described with reference to user input content items, the present disclosure is broadly applicable to other types of digital content. In some examples, the method 2900 is performed by referring to Figure 1 The various components of the interactive system 100 are executed.

[0209] The method 2900 begins at an open loop block 2902 and proceeds to block 2904, where the user system 104, and in particular the interactive client 106 (as an example of a media application), causes a preview interface 2814 to be presented, the preview interface 2814 including a first user input content item (e.g., a first digital image 3004). Fig.28 As described, the first digital image 3004 may have been captured using the camera interface 2804 of the interactive client 106 and then presented within the preview interface 2814 after an image or video capture operation is performed by the user via the camera interface 2804 .

[0210] The interactive client 106 provides camera functionality supported by the camera system 204, which causes a camera interface 2804 to be presented. Within the camera interface 2804, digital image feeds from one or more cameras of the user system 104 (e.g., the mobile device 114, the head wearable device 116, and / or the computer device 118) are displayed within a viewing window 2806. In response to a user selection of a capture button 2808 within the camera interface 2804, a captured input is received, in response to which a first digital image 3004 may be captured as an example of a first user input content item. The first digital image 3004 is then stored locally or remotely so that the interactive client 106 can access it.

[0211] A preview interface is a user interface presented to a user of a media application before a content item (shown in the preview interface) is sent to a third-party user of the media application and before the content item is shared with a third-party user. In some examples, when a user is viewing related content in a preview mode or preview state, the content has not yet been finalized or submitted / approved for publication and can only be viewed by the user.

[0212] From block 2904, method 2900 proceeds to block 2906, where the interactive client 106 causes a number of modification indicators (e.g., Fig.30The authoring tool 3008 shown in FIG. 28 is presented in the preview interface 2814. Fig.30 As shown, each authoring tool 3008 is associated with a corresponding modification function of the interactive client 106. In addition, the authoring tool 3008 includes a user input content combination indicator (also called a combination graphical element) in the form of a remix button 2820, which is associated with a user input content combination function (e.g., a "remix" function) of the interactive system 100.

[0213] At block 2908, the interactive system 100 detects a user selection of the remix button 2820, and at block 2910, in response to detecting the user selection of the remix button 2820, activates a combined function (e.g., a remix function). As described herein, the combined function enables a user of the interactive client 106 to access a second user input content item (e.g., newly created digital content data, such as Fig.30 , or a stored digital content item selected from a local or remote storage device accessible to the user system 104 and then displayed in a preview state). The combine functionality further causes the first user input content item (e.g., first content data in the form of the first digital image 3004) to be combined with the second user input content item (e.g., second content data in the form of the second digital image 3010) to create a combined user input content item (combined content data).

[0214] To this end, the user input content combination function may invoke a camera function provided by the camera system 204 of the interactive system 100 and enable the user to capture a second digital image 3010 from a digital image feed presented within the context of the remix interface 2822 (the combination interface). The first digital image 3004 and the complementary digital image feed may be presented simultaneously or synchronously within the remix interface 2822. Then, a user selection of a capture button within the remix interface 2822 causes one or more digital images 3010 from the complementary digital image feed to be captured as the second digital image 3010. At block 2912, the second digital image 3010 is displayed simultaneously or synchronously with the first digital image 3004 within the preview interface after capture as a combined user input content item.

[0215] As mentioned above about Fig.28As described, the combination function can also call a layout function to enable the user to layout the first digital image relative to the second digital image within the preview interface 2814. The layout function can enable the user to manually define or specify the relative position and size of the first digital image and the second digital image and other features (such as enhancements). The layout function can also present several predefined layouts that specify the relative position and size of the first digital image and the second digital image within the combined user input content item. In addition to specifying the relative position and size, a particular layout can also automatically apply several other visual or audio features to the combined user input content item. For example, in the case where a particular layout has a theme, visual features (such as enhanced and supplementary images) and audio features (such as sounds or songs) can be automatically applied to the combined user input content item.

[0216] After the combined user input content item is presented within the remix interface 2822 at block 2912, the user may then select Fig.30 2914. In some examples, the preview interface 2814 can be automatically presented to the user again without selecting a back element or performing a back user action. At box 2912, the user can decide within the preview interface 2814 to add additional user input content items to the combined user input content item (e.g., create and add additional video clips). Accordingly, the user can select the remix button 2820, and cause the user selection of the remix button 2820 to be detected at box 2914.

[0217] At box 2916, in response to detecting the user selection of remix button 2820, the combination function can be activated again to present remix interface 2822. In other words, the combination function can be reactivated to create additional combination items. Within the context of remix interface 2822, the user can access both the combination user input content item and the third user input content item (e.g., a newly captured digital image or a digital video content item captured using a camera function within the context of remix interface 2822), and the previous combination user input content item is combined with the third user input content item to create additional combination user input content items (additional combination content data). Therefore, the user can continuously cycle between the preview interface and the remix interface to create several digital user input content items, which can be combined into a single project. According to some examples, the problem that the user cannot create and modify a user content combination containing multiple content items (e.g., before publishing or sharing these content items) can be solved. Method 2900 then terminates at closed loop box 2918.

[0218] Fig.30is an interface sequence diagram illustrating an interface sequence 3002 that may be presented by an interactive client 106 hosted and executed on a user system 104 according to some examples. The interface sequence 3002 is a reference to Fig.28 Some examples of more general interfaces described. For simplicity, the functionality of the preview interface 2814 and the remix interface 2822 will be described with respect to digital images, which may include a single or multiple still digital images or a sequence of digital images constituting a digital video clip.

[0219] Sequence 3002 presents two views of preview interface 2814 and a single view of remix interface 2822. The first view of preview interface 2814 ( Fig.30 ) shows a first digital content item in the form of a digital image 3004, which is displayed in the preview window 2816. The digital image 3004 can be a user input content item captured and created using the interactive client 106. Fig.30 , the digital image includes an original digital image of fruit captured using camera interface 2804, to which a "fruit" enhancement is applied within the context of camera interface 2804 or preview interface 2814 to create an enhanced digital image in the form of digital image 3004. Fruit enhancement 3022 has the effect of superimposing an avocado-like character on the original image (photograph) of fruit.

[0220] The preview interface 2814 has an interactive area 2810 that presents a collection of enhancements that can be applied to the digital image 3004. The interactive area 2810 also allows the user to select additional digital content to be included in the transmission to the receiving user of the interactive system 100 (e.g., by selecting a "plus" button in the media list 3020 before selecting a transmission indicator in the form of a send button 3006).

[0221] The right side of the preview interface 2814 displays a menu of indicators of user-selectable authoring tools 3008. These authoring tools 3008 enable the user to perform various modifications to the digital image 3004 (e.g., add text, draw on it, crop, add music, and flip it). The menu also includes a user-selectable indicator in the form of a remix button 2820 to invoke the remix system 206 and the remix interface 2822 (e.g., a modular remix camera) to provide the user with a combination function that enables the user to combine the digital image 3004 with a second content item (in the form of a second digital image 3010). Fig.30As shown in the second preview interface 2814 in FIG, a remix tool tip 3012 is displayed in conjunction with the remix button 2820 to inform the user about the function invoked by the remix button 2820 and to invite the user to “remix” the digital content items (e.g., the user input content items) to create a combined digital content item.

[0222] The remix interface 2822 (also referred to as a combined interface) enables a user to capture a second digital image 3010, and presents both the first digital image 3004 and the second digital image 3010 in a layout configuration relative to each other. The second digital image 3010 can be captured within the context of the remix interface 2822 by user selection of a capture button 2808 presented within the interactive area 2810. The interactive area 2810 also presents a number of enhancements that a user can selectively apply to the second digital image 3010. The remix interface 2822 also includes a number of camera controls 3014 (e.g., flip camera, flash, etc.) that can be invoked within the context of the remix interface 2822 to capture the second digital image 3010.

[0223] The layout of the first digital image 3004 and the second digital image 3010 may be manually determined by the user (e.g., by dragging the digital images to relative positions within the remix interface 2822), or may be selected by the user from a number of preconfigured layouts 3016 presented by the layout tool 2826. The preconfigured layouts 3016 may include different split-screen layouts, picture-in-picture layouts, foreground / background layouts, etc., as described elsewhere.

[0224] The remix interface 2822 also includes a return element in the example form of a user-selectable return arrow 3018, which the user can select to return the user to the preview interface 2814 and create a combined content within the remix interface 2822. For example, after combining the first digital image 3004 with the second digital image 3010 within the remix interface 2822, the user can select the return arrow 3018. In response to this selection, the interactive client 106 returns to the preview interface 2814, which then presents the combined digital image. The user can then select the remix button again and return to the remix interface 2822, where the user can generate or select a third digital content to combine with the previous combined content. In this way, the user can reconstruct a combination of multiple instances of user-input content into a single instance or item of user-input content. Providing various enhancements that can be applied to discrete digital content components of a combined digital content item and providing various layout options and functionality enables users to find a wide range of creative expressions using both self-generated (or self-selected) user input content and user input content obtained from other users of the interactive system 100 (or even from other sources).

[0225] Fig.31 3100 is a flow chart showing a method 3100 according to some examples, which facilitates generating additional combined user input content items. The method 3100 begins at an open loop block 3102 and proceeds to block 3104, where the user system 104, particularly the interactive client 106 (as an example of a media application), causes a camera interface to be presented.

[0226] At block 3106 , a digital image feed is presented within the camera interface. The user then selects the camera button and the media application receives a corresponding user capture input at block 3108 .

[0227] According to some examples, in response to the received user capture input, the digital image content is captured as a third user input content item at block 3110. The method 3100 then proceeds to block 3112 where the third user input content item is stored by the user system 104 to be accessible by the combined function of the media application.

[0228] Then, the user creates a combined user input content item using the combination function of the media application. This can be done according to the examples described herein. The combined user input content item is a combination of a first user input content item and a second user input content item. For example, the combined user input content item may include a first user input content item as a background image and a second user input content item superimposed on a portion of the first user input content item. At box 3114, the combined user input content item is presented to the user via a preview interface. The preview interface includes a user input content combination indicator in the example form of a remix button, which enables the user to create another combined user input content item.

[0229] At block 3116, the media application detects a user selection of a user input content combining indicator within the preview interface. From block 3116, method 3100 proceeds to block 3118, where a user input content combining function is activated. This may allow a user to add (e.g., overlay) a third user input content item to the combined user input content item to create an additional combined user input content item.

[0230] According to some examples, at block 3120, the user then selects a third user input content item (e.g., from a file storage or media library), and the media application receives the selection. At block 3122, the additional combined user input content item is presented within the remix interface. As described herein, according to some examples, the user can select a layout for the additional combined user input content item, or manually adjust the layout of the third user input content item relative to the combined user input content item.

[0231] The user may share the additional combined user input content item with other users, or as described herein and according to some examples, the user may combine the additional combined user input content item with more content items, for example, to create an additional combined user input content item consisting of, for example, 4, 5, 6, or 7 different user input content items. The method ends at closed loop block 3124.

[0232] A user may wish to create a combined video item in a media application to share with other users. However, the media application system may be limited in the ability to combine content items and / or the way to combine items. For example, a media application may not be able to provide users with flexible time relationship selections and cannot generate a combined video item from a component. The method and system according to the example of the present disclosure solves an exemplary technical problem as follows: a system is provided, which allows a user to enhance, contrast or contextualize a first user input content item with a second user input content item or even multiple user input content items in a time-based manner, a spatially relative manner or a combination thereof. According to some examples, a method and system for implementing a combination (e.g., remixing) of user input content items in a series of interfaces is provided, which allows a user to combine different user input content items generated by a single user or multiple users of an interactive system into a combined user input content item in a time-based manner, a spatially relative manner or a combination thereof. In some examples, a user can generate a combined user input content video based on one or more combination options (e.g., a specified time relationship between content items forming a combined user input content video).

[0233] Fig.32 is an interface sequence diagram illustrating an interface sequence 3202 that may be presented by an interactive client 106 (eg, a media application) hosted and executed on a user system 104 according to some examples.

[0234] Sequence 3202 shows several interfaces that allow a user to, for example, capture or access user input content video 3234 and combine the user input content video 3234 or a portion thereof with another content item. These interfaces include a camera interface 3212, a viewing interface 3214, and a remix interface 3216. Each of these interfaces provides specific functionality within the interactive client 106 and enables a user to capture, view, or modify a content item (e.g., user input content video 3234). Each of these multiple interfaces also includes a user input content combination indicator 3218 that can be selected by the user to invoke a combination function that presents a trim interface 3206 and a remix camera interface 3222 (also as shown in FIG. 1 ). Fig.32 ). The user input content combination indicator 3218 may be in the form of a “remix” button, an “add to” button, a “combine” button, etc.

[0235] Camera interface 3212 enables a user of user system 104 to create a user input content item in the example form of a digital image or digital video. The user can also add audio to the project using a media application, for example, selecting a soundtrack to play simultaneously with a digital video.

[0236] The camera interface 3212 is presented by the camera system 204 of the interactive system 100 and provides a viewing and capturing mechanism. Specifically, the viewing and capturing mechanism includes a viewing window that presents a real-time image feed from any one of a plurality of cameras of the user system 104 to the user and a capture button 3240 that can be selected by the user to allow the user to capture one or more images from the image feed as digital images or digital videos. The timeline within the camera interface 3212 presents a series of enhancements or modifications supported by the enhancement system 208 that can be applied in real time to the digital image presented within the viewing window of the camera interface 3212.

[0237] The viewing interface 3214 enables a user to view content items (e.g., user input content video 3234) retrieved from the memory of the user system 104, where the storage of the content items is the result of being received from another user of the interactive system 100 in the form of a message or posting stream, or is the result of the user system 104 previously capturing and storing it in memory.

[0238] The remix interface 3216 enables a user to combine multiple content items (e.g., user input content images, user input content videos, audio, augmented reality enhancements, etc.), some of which may have been received from different users of the interactive system 100, to create a combined user input content item (e.g., a combined user input content video clip).

[0239] A remix interface 3216 is also presented by the camera system 204, and in particular the remix system 206, to enable a user to capture or select supplemental user input content items, and to combine the supplemental user input content items with previously captured or selected user input content items to generate a combined user input content item within a combination window. The remix interface 3216 presents various layout tools for spatially arranging (e.g., laying out or overlaying) multiple user input content items relative to each other within some predefined or free-form layout, as well as several camera controls. The remix interface 3216 also presents various enhancement and modification tools that can be applied to the combined content within the context of the remix interface 3216. About Fig.32 Additional details of some examples of interfaces are discussed in subsequent figures.

[0240] The trimming interface 3206 is also presented by the camera system 204 of the interactive system 100 after receiving, selecting, or capturing the user input content item. The trimming interface 3206 presents a "preview" of the user input content item in a preview window 3208. The timeline 3204 presents a plurality of tools (e.g., tools of the enhancement system 208) to enable the user to enhance or otherwise modify the user input content item to generate an enhanced user input content item. The trimming interface 3206 also presents a check indicator 3210 to invoke the communication system 210 of the interactive system 100 and cause the user input content item to be presented within the remix camera interface 3222. The trimming interface 3206 also includes a remix button that the user can select to invoke content combining functionality supported by the remix system 206, which is a subsystem of the camera system 204.

[0241] The remix camera interface 3222 enables the user to capture additional digital content (e.g., content item 3236) and add, append, or otherwise "mix" the content item 3236 with the user input content video 3234 to create a combined user input content item 3238. The remix camera interface 3222 also includes a progress bar 3220 that indicates the relative duration of each content item within the combined user input content item 3238 relative to each other and relative to the overall duration of the combined user input content item 3238. To enable the capture of more digital content, the remix camera interface 3222 includes a capture button 3226 that is presented in the camera window 3224. The media library selector 3230 enables the user to select and retrieve stored digital content, while the enhancement selector 3232 enables the user to select any of a number of enhancements and apply those enhancements to the content portions of the combined user input content item 3238, either individually or as a whole. The remixed camera interface 3222 also presents a number of camera controls 3228 related to the capture of the user input content item, such as a camera selection mechanism, a flash activation mechanism, and the like.

[0242] In some examples, as described below, sequence 3202 may also include a reordering interface 3802 that enables a user to adjust the order in which two or more content items appear in the combined user input content item.

[0243] Refer again Fig.32 , the combined user input content item 3238 is created within the remix camera interface 3222 and is available for sharing (e.g., with other users of the interactive system 100).

[0244] Fig.33is a flow chart describing a method 3300 according to some examples. The method 3300 starts at block 3302 and proceeds to block 3304 where, using at least one processor of the user system 104, the media application causes the user input content video 3234 to be presented.

[0245] At box 3306, the media application presents a user-selectable user input content combination indicator 3218, and at box 3308, the media application detects a user selection of the user input content combination indicator.

[0246] Selection of the user input content combination indicator 3218 activates the user input content combination function (block 3310). The user input content combination function enables the user input content video 3234 timeline (e.g., see Fig.35 The timeline 3204 shown in FIG. 3204 is presented in the editing interface 3206. The timeline 3204 allows the user to select a portion of the user input content video 3234.

[0247] At block 3312, the method 3300 includes detecting a selection of a portion of the user input content video 3234. The detection may include detecting a user action with respect to the timeline 3204 of the user input content video 3234. Causing the timeline 3204 to be presented may include causing a trim function with respect to the user input content video 3234 to be presented to enable the user to trim the user input content video to generate the portion of the user input content video 3234.

[0248] In some examples, the trim function defaults to selecting a predetermined portion (e.g., the first five seconds) of the user input content video 3234. It will be appreciated that in examples where the user input content video 3234 has a certain duration (e.g., shorter than 5 seconds), the trim function may not be presented, and the user may not be enabled to select only a portion of the user input content video 3234, but rather the entire user input content video 3234 is selected for combination as described below.

[0249] In some examples, the user input content combining functionality causes a progress indicator in the example form of a progress bar 3220 to be presented to indicate the duration of the portion of the user input content video 3234 relative to the combined user input content item 3238 .

[0250] At block 3314, the user inputs the content combination function to access the second content item 3236. The second content item 3236 may be a second video clip.

[0251] The method 3300 may include detecting a user selection of one or more combination options at block 3316. The combination options may include a sequential selection made by the user, such as whether the user input content video 3234 appears at the beginning or end of the combined user input content item 3238, or whether a particular item appears before or after another item. Thus, the sequential selection may indicate a selected temporal relationship between the user input content items that make up the combined user input content item.

[0252] The combination options may include a split selection made by the user indicating, for example, whether to split the user input content video 3234 into two or more independent parts (e.g., the user input content video 3234 may be split into two parts, one of which appears at the beginning of the combined user input content item 3238 and the other appears at the end of the combined user input content item 3238, with the second user input content item appearing between the two parts). Similarly, the split selection may indicate that the second item 3236 is to be split into two or more independent parts. These independent parts may be arranged in any order in the combined user input content video.

[0253] At block 3318, the user input content combining function combines a portion of the user input content video 3234 with the second content item 3236 to create a combined user input content item 3238 (e.g., a combined user input content video). The combining may be performed based on one or more combining options selected by the user, for example, to ensure that the video portions or clips are arranged in a particular order, and the method ends at closed-loop block 3320.

[0254] It will be appreciated that while examples have been described that focus on creating composite video items, similar techniques may be used to create other composites, such as composite slides where each slide has a defined timing.

[0255] Now refer to Figure 34 to Figure 38 The interface diagrams shown are used to describe additional details about some examples.

[0256] Figure 34 to Figure 36 is an interface diagram illustrating a series of user interfaces that may be generated and displayed by the interactive client 106 according to some examples.

[0257] like Fig.34 As shown, viewing interface 3214 presents the user input content item in the form of a user input content video 3234 depicting a dancer. Viewing interface 3214 may be presented as a result of the user selecting a user input content video within a feed (e.g., a feed of content items posted by another user of interactive system 100) or having received a user input content video within a message from another user of interactive system 100.

[0258] Within the viewing interface 3214, the user may select a context menu item or perform some other action (e.g., press and hold) relative to the user input content video 3234, in response to which the interactive client 106 causes a context menu 3402 to be presented, the context menu 3402 including a plurality of options with associated text and user-selectable tags. The user-selectable tags shown include two examples of user input content combination tags, namely a remix button 3408 and an add button 3404. As described elsewhere, the remix button 3408 may enable a user to combine multiple content items into a collage or otherwise spatially arranged collection of content items that are displayed simultaneously (e.g., in a split-screen configuration or / and an overlay configuration). According to some examples, the add button 3404 may enable a user to combine multiple content items for sequential display (e.g., as a continuous video including multiple content items or portions thereof), wherein the user is enabled to specify or adjust the temporal relationship between the items.

[0259] User selection of the add button 3404 from the context menu 3402 invokes the trim interface 3206, which is displayed in Fig.35 35 and 36. A full screen preview of the user input content video 3234 and trimming controls are shown and displayed. In some examples, the trimming controls include a timeline 3204 with a pair of handles 3502 and a playhead marker 3504. User manipulation of the handles 3502 allows the user to trim the original user input content video 3234 to a desired length, and in this manner select a portion of the original user input content video 3234 to be included in the combined user input content item 3238. The trimming controls may select a predetermined portion of the original user input content video 3234 (e.g., the first five seconds) by default or select a full clip, where the original user input content video 3234 is less than a predetermined length (e.g., five seconds).

[0260] A check indicium 3506 in the example form of a checkmark is displayed adjacent to the timeline 3204 and is user-selectable to indicate to the user that the trimming operation is complete within the trimming interface 3206, in which case the interactive client 106 advances to the remix cam interface 3222. If the user drags the handle 3502 to select the entire duration of the original user input content video 3234, the check indicium 3506 is disabled because the user does not have any time to access any additional content items (e.g., additional video clips recorded by the user in the context of the remix cam interface 3222) within the fixed duration of the combined user input content video.

[0261] After completing the selection of a length (e.g., portion) of the user input content video 3234 to include in the combined user input content video, or where the length is selected by default or automatically by the messaging application, the user may then selectively activate the check mark 3506 to advance the display to the remix camera interface 3222, as described in accordance with the embodiment of the present invention. Fig.36 3238 . The remix camera interface 3222 presents a modular camera screen to the user. The remix camera interface 3222 presents a continuous image feed from one or more cameras of the user system 104. The remix camera interface 3222 presents a capture button 3606 that the user can select to record digital content (e.g., a still image or a video clip) as a second content item 3236 from the continuous image feed. The captured content item 3236 is then appended or added to the selected portion of the user input content video 3234, or otherwise "mixed" with it to form a combined user input content item 3238. The capture button 3606 can be used to capture any number of additional digital content items, which can then be combined with the selected portion of the user input content video 3234 within the combined user input content item 3238.

[0262] In addition to enabling "live" capture using the modular camera of the remix camera interface 3222, the remix camera interface 3222 also includes a media library selector 3610 that enables a user to select additional digital content from a media library (e.g., a local library of the user system 104 or a remote server-based library of the interactive system 100) to be included in the combined user input content item 3238.

[0263] The trimming interface 3206 and the remix camera interface 3222 also include a progress bar 3508 that initially presents a visual display of the duration of the selected portion of the user input content video 3234 relative to the total duration of the possible combined user input content items. Specifically, the highlighted portion of the progress bar visually depicts the duration of the selected portion of the user input content video 3234. When additional content items (e.g., content item 3236) are added to the combined user input content item 3238 within the context of the remix camera interface 3222, the progress bar 3508 updates to indicate the total occupied duration of the potential maximum duration of the combined user input content item 3238. The progress bar 3508 can display the duration of each of the content items that make up the combined user input content item 3238 in a different color or using a visual distinction for the duration of each content item.

[0264] The remix camera interface 3222 also includes an enhancement selector 3612 that the user can select to present a scrolling list of enhancements (e.g., filters or stickers) that the user can optionally apply to the captured content item 3236. A check mark 3602 is displayed within the remix camera interface 3222 and remains disabled until the user has identified at least one content item 3236 through capture or retrieval to include within the combined user input content item 3238. Once activated, the user can select the check mark 3602 to indicate completion of creation of the combined user input content item 3238, thereafter making the combined user input content item 3238 available for preview within the viewing interface prior to communication with other users of the interactive system 100 through direct messaging or sharing to a user's publication feed supported by the interactive system 100.

[0265] The user can select the undo button 360 within the remix camera interface 3222 to return the user from the remix camera interface 3222 to the trim interface 3206. Within the remix camera interface 3222, camera controls 3608 enable the user to apply various controls when capturing the content item 3236.

[0266] It will also be noted that each of the viewing interface 3214 and the remix camera interface 3222 includes attribute data 3406 that indicates and identifies the source (e.g., user of the interactive system 100) and context of the displayed user input content item (e.g., user input content video 3234). The attribute data 3406 may be displayed upon playback of the combined user input content item 3238 for each of the constituent digital content items to provide source and context information for a particular portion or portions of the entire combined user input content item 3238. Within any interface, the attribute data 3406 is user-selectable to present a profile page for a particular source (e.g., a user profile page), if such a profile page exists and is published pursuant to appropriate permissions received from the relevant source.

[0267] In some examples, when the combining function is activated or invoked, it may cause, for example, the simultaneous presentation of a first user input content video and a digital image feed from a user's camera within a combining (remixing) interface. The camera function, as invoked by the combining function, may receive a user capture input and, in response to receiving the user capture input, capture digital content as a second user input content item. The combining function may cause, for example, the simultaneous presentation of a first user input content video (or a portion thereof) and a second user input content item in a combining interface.

[0268] Fig.37 and Fig.38 is an interface diagram illustrating a user interface that may be generated and displayed by the interactive client 106 according to some examples.

[0269] As described above, one of the combination options associated with the creation of a combined user input content item (e.g., a combined video) may be an order selection. In some examples, the order selection is an instruction related to the temporal arrangement of a portion or a part of the combined user input content video. According to some examples, a user may be allowed to reorder the user input content items in the example form of a video clip.

[0270] Fig.37 A camera interface 3702 is shown according to some examples. The camera interface 3702 can be presented to allow a user to capture digital content using a camera button 3708. The camera interface 3702 also includes an undo button 3706 and a preview button 3710.

[0271] Reference Fig.37 , the camera interface 3702 displays thumbnails 3704 to identify and indicate a sequence of video clips that have been captured or selected by the user. These clips may be captured and / or selected, for example, in the process of creating a composite user input content video item. Fig.37 , thumbnails 3704 indicate that the user has captured or selected a sequence of four clips: a first clip of 2.1 seconds, a second clip of 1.2 seconds, a third clip of 4.3 seconds, and a fourth clip of 2.2 seconds. If the user chooses to publish the clips in their current form, the clips are shown in a particular order as they would appear in the final combined user input content video.

[0272] Fig.37 Also shown is a progress bar 3712 according to some examples, as shown in camera interface 3702. The user is able to play the clips in their current sequence in camera interface 3702, and progress bar 3712 provides a visual indication of the playback position and duration of each clip's portion relative to the combined video.

[0273] In response to detecting a reordering selection made by the user, a reordering interface 3802 may be generated and presented. The reordering interface presents the content items (e.g., constituent clips) that make up the composite user input content video in their current order, and enables the user to modify the temporal relationship between the items in the sequence. For example, a reordering selection may be made by pressing and holding one of the clips that form part of the thumbnails 3704, performing a drag gesture with respect to one of the clips, or selecting a dedicated clip order button or option (not shown). In response to making a reordering selection, the media application may thus activate a reordering mode or reordering state, thereby presenting the reordering interface 3802.

[0274] Fig.38A reordering interface 3802 is shown according to some examples. In the reordering state, a user is allowed to reorder video clips, delete video clips, watch a replay of clips, and edit or enhance one or more video clips.

[0275] In some examples, the media application can initiate tactile feedback to notify the user that a reordering mode or state has been activated. The media application can move clip 3804 relative to the remaining elements in reordering interface 3802 (e.g., through a "wiggling" motion) to notify the user that a reordering mode or state has been activated.

[0276] The adjustable sequence of clips 3804 can be reordered by dragging the clip to the desired position in the sequence of clips 3804. The user can delete a clip by tapping the "(-)" icon displayed in the upper right corner of each thumbnail. When the user is finished, select the check mark button or check button 3806.

[0277] It will be appreciated that in some examples such reordering techniques may be employed as part of a combining functionality of a media application to allow a user to reorder content items that form or are intended to form part of a combined user input content item.

[0278] The interactive client 106 may provide various entry points to invoke the combined functionality. For example, Fig.34 The viewing interface 3214 shown provides two entry points: a remix button 3408 and an add button 3404 . Fig.39 and Fig.40 Other examples of entry points are shown. Fig.39 The viewing interface 3904 shown in shows an example in which the remix tray 3902 is presented in response to a user selecting an appropriate menu icon or performing a user activity (eg, a press and hold operation) relative to the displayed user input content video.

[0279] The remix tray 3902 includes two user-selectable combination tags: an add before or after button 3906 and an add together button 3908. For example, the add before or after button 3906 can initiate a reference to the Fig.33 For example, the add together button 3908 may initiate the combination process described in Fig.29 These buttons provide the user with the option of combining the relative time or visual placement of multiple content items within a user-entered content item.

[0280] Fig.40The camera interface 4004 shown in FIG. 4 (of which the remix camera interface 3222 is an example) includes a menu presenting several layout tools 4002, the menu including a user input content combination mark 4006, which is user-selectable to invoke a user input content combination function to allow the user input content items to be arranged or placed in time (e.g., time-based ordering) in a video or slide show, as described herein. The layout tools 4002 also include multiple spatial layout options that enable a side-by-side or overlay layout of multiple digital media content items. The user can select the user input content combination mark 4006 to invoke the combination function and display the trim interface 3206 and the remix camera interface 3222.

[0281] The media application may provide access to a media platform (e.g., the interactive system 100 or a portion thereof). Thus, a user may be a media platform user, wherein the media platform facilitates sharing of a combined user input content video with another user of the media platform. The media platform may store sharing permissions for the user and other users related to content sharing via the media platform, and may perform sharing of the user input content video by the user with other users according to the sharing permissions.

[0282] Media applications (e.g., messaging applications) can allow users to view content items and share the content items with other user devices. Content items can be shared by multiple content sharing destinations. However, in some examples, media applications can be designed to limit or prevent users from sharing third-party content items to one or more content sharing destinations, for example, when a user is viewing a third-party image, the user can be prevented from directly publishing the third-party image to the user's own content collection by disabling or deactivating the relevant content sharing destination. Although restrictions of this nature may encourage users to create and share original content, these restrictions may cause user dissatisfaction and even cause the use of media applications to decrease. An example technical problem of unlocking a previously locked or deactivated content sharing destination based on the interaction of a user with a media application can be solved by providing a method and system with improved functionality, which automatically links the creation of a combined user input content item to the presentation of a content sharing destination and the interaction with the content sharing destination.

[0283] Fig.41 is an interface sequence diagram illustrating an interface sequence 4102 according to some examples, which interface sequence 4102 may be presented by an interactive client 106 hosted on and executed by a user system 104 .

[0284] Sequence 4102 includes a viewing interface 4104 that enables a user of user system 104 to view user input content items that are published and / or shared by third parties. The third-party user input content items may, for example, be content that forms part of a collection of content, for example, the third-party user input content items may form part of a third-party user's "personal story" that has been created and sent / broadcasted by the user, or may form part of a public feed of curated third-party items presented to the user by a media application.

[0285] Viewing window 4106 presents the user with a view of these one or more user input content items, e.g., as a sequential collection of content items presented by a media application on user system 104. Interaction area 4110 in viewing window 4106 presents a number of options available to the user. For example, interaction area 4110 may include a user-selectable button that allows the user to react to the content (e.g., "like" the content) and another user-selectable button that allows the user to comment or initiate a message to a third-party user associated with the content. Fig.41 As shown, according to some examples, interactive area 4110 includes a user-selectable content sharing graphical element 4108. For example, content sharing graphical element 4108 may be implemented as a "share" button or a "send to" button.

[0286] Selection of content sharing graphical element 4108 results in presentation of sharing interface 4112. For example, such an interface may be implemented as a "send to" interface or "send to" tray that presents the user with selectable options related to sharing of a content item viewed by the user.

[0287] The sharing interface 4112 may be presented by the communication system 210 to enable a user to share content items presented in the viewing interface 4104 with other users. To this end, the sharing interface 4112 presents various sharing options identified by respective destination graphic elements 4114, 4116, 4118a in the sharing interface 4112, referred to herein as "destinations". The destination graphic elements 4114 and 4116 are user selectable to enable the user to share the relevant content items to the sharing destinations associated with those destination graphic elements 4114 and 4116. For example, the destination graphic element 4114 may identify a first sharing destination, such as "share this image with a friend in a private message", while the destination graphic element 4116 may identify a second sharing destination, such as "share this image via email". If the user selects the relevant destination graphic element 4114, 4116, then the sharing of the content item to the destination associated with the particular destination graphic element 4114, 4116 is initiated.

[0288] In some examples, the content item being viewed by the user may not be shared to one or more content sharing destinations. For example, the interactive system 100 may be configured so that the user is not allowed to directly share third-party user input content items to the user's "story" or directly to the public feed / collection. For example, such restrictions can be implemented to prevent users from simply re-sharing or re-publishing content that has been shared or published by other users of the interactive system (or making it more difficult for them to do so). For example, the interactive system can be designed to facilitate sharing or publishing "original" or "first time" content items to stories and / or public collections / feeds, while preventing or reducing sharing or publishing for presentation to other users of the interactive system. "Non-original" or "existing" content items. For this reason, the sharing interface 4112 can present a destination graphic element 4118a that identifies a locked destination. For example, the destination graphic element 4118a can remind the user that a certain content sharing destination, such as "share to your story" is not available (locked or inactive) for the specific content item being viewed by the user.

[0289] In examples of the present disclosure, various content sharing destinations may be available or presented to the user. For example, where the user is using an interactive client 106, these may include sharing to another user of the interactive client 106 via direct message, sharing to a user's story or other content collection, sharing to a public content collection available to the user of the interactive client 106, sharing to a third party application such as a third party messaging application and / or a third party social networking platform, sharing via email, sharing via text or multimedia message, etc.

[0290] like Fig.41 As shown, destination graphical element 4118a is displayed in conjunction with remix button 4120. In some examples, remix button 4120 allows a user to unlock or activate a locked destination by remixing related content items, thereby creating a new combined content item that can be shared to a previously locked or unavailable destination.

[0291] In response to the user's selection of the remix button 4120, a remix interface 4122 (also referred to as a combination interface) is presented by the camera system 204, and specifically by the remix system 206, to enable the user to capture another content item and combine the other content item with the third-party content item being viewed in the viewing interface 4104 to generate a combined user input content item within a combination window 4126. The remix interface 4122 presents various layout tools 4128 for laying out the multiple content items relative to each other and within certain layouts, as well as a plurality of camera controls 4124. The interactive area 4110 of the combination window 4126 presents various enhancement and modification tools that can be applied to the combined content within the context of the remix interface 4122.

[0292] In some examples, once the combined content item has been created, the user may again be presented with a sharing interface 4112 presenting various sharing destinations, but with the previously locked sharing destination unlocked, as represented by an updated destination graphical element 4118b. As with the other destination graphical elements 4114, 4116, the updated destination graphical element 4118b is user-selectable to enable the user to share the combined content item created using the remix interface 4122 to a content sharing destination that was previously unavailable.

[0293] To enable the user to share the combined content item to an unlocked content sharing destination, the media application may therefore cause presentation of an updated sharing interface 4112 having an updated destination graphical element 4118b, and determine that the user has selected the updated destination graphical element 4118b. In response to determining the user selection of the updated destination graphical element, the media application causes the combined content item to be published to the unlocked content sharing destination.

[0294] about Fig.41 Additional details of some examples of interfaces will be discussed with reference to the following examples.

[0295] Fig.42 4200 is a flowchart illustrating a method 4200 for facilitating generating a combined user input content item and unlocking a locked content sharing destination according to some examples. Although some examples are described with reference to user input content items, the present disclosure is broadly applicable to other types of digital content. In some examples, the method 4200 is performed by reference to Figure 1 The various components of the interactive system 100 described above are executed.

[0296] The method 4200 begins at open-loop block 4202 and proceeds to block 4204, in which the user system 104, and specifically the interactive client 106 (as an example of a media application), causes presentation of a third-party user input content item. The third-party user input content item may be, for example, an image or video clip viewed by an active user on the user system 104. The third-party user input content item may include multiple types of content (e.g., audio content combined with image content), and may include one or more enhancements or modifications, as described elsewhere herein.

[0297] The third-party user input content item may be presented within the viewing interface 4104 in conjunction with a content sharing graphical element 4108. The content sharing graphical element 4108 may be user selectable to invoke a content sharing function of the interactive client 106.

[0298] According to some examples, method 4200 includes determining at box 4206 a third-party authorization for combining a third-party user input content item with other content items to create a combined user input content item. For example, an authorization setting related to a third-party user input content item indicating the authorization can be detected at box 4206. Therefore, determining the third-party authorization to the third-party user input content item to be combined can include determining the user setting related to the third-party user input content item indicating the third-party authorization. When creating or publishing a third-party user input content item, an option of allowing or not allowing the third-party user input content item to be used in combination with other content items created or initiated by other users of the interactive system 100 is presented to the third-party content creator. This permission / not allowing is then recorded and stored in the database 128 as the authorization setting related to the relevant third-party user input content item, and is accessed at box 4206 to perform detection of the third-party authorization.

[0299] The method 4200 proceeds to block 4208 where a user selection of the content sharing graphical element 4108 is determined. At block 4210, in response to determining the user selection of the content sharing graphical element 4108, a content sharing function of the interactive client 106 is invoked. This causes the presentation of destination graphical elements 4114, 4116, 4118a on the user system 104, e.g., to display to the user various destinations to which the content the user is viewing may be sent.

[0300] As reference Fig.41 As described, each destination graphical element identifies a content sharing destination (or at least one content sharing destination). Destination graphical element 4118a may identify a first locked content sharing destination, while two additional content sharing destinations (identified by 4114 and 4116) may be unlocked (available) content sharing destinations.

[0301] Method 4200 may include determining or checking the content type associated with the third-party user input content item being viewed by the user at box 4208. Based on the content type, the interactive client 106 can determine the content sharing destination to be presented to the user and / or the availability of each content sharing destination (e.g., whether it is locked or unlocked). For example, the interactive client 106 can determine that the user has selected the content sharing graphical element 4108 about the third-party user input content item, and check the sharing settings associated with the third-party user input content item in the database 128. The sharing settings may indicate that the third-party user input content item may not be directly shared to the first content sharing destination. In some examples, the sharing settings may be user-specific. In response to this, the first content sharing destination can be presented as locked or unavailable in the sharing interface 4112.

[0302] According to some examples, a first content sharing destination is locked with respect to a third-party user input content item or even with respect to all third-party user input content items, while an additional content sharing destination is unlocked for the third-party user input content item or all third-party user input content items. To this end, at box 4210, a destination graphic element 4114, 4116 identifying each respective available (unlocked) content sharing destination is presented, and the destination graphic elements 4114, 4116 are user-selectable to enable the user to share the third-party user input content to the content sharing destination associated with each respective element 4114, 4116.

[0303] According to some examples, the method includes causing presentation of a combined graphical element in the form of a remix button 4120 by the interactive client 106 at block 4212. The remix button 4120 is user-selectable to invoke a combined function (or "remix" function) of the interactive client 106, and a user selection thereof is determined at block 4214. Fig.42 In block 4206 , it will be appreciated that in the event that it is determined that the third party has not authorized the combination of the third party item with the other items, according to some examples, the remix button 4120 is not displayed.

[0304] In response to determining the user selection of the remix button 4120, the interactive client 106 activates the combine function at box 4216. The combine function allows the user to access (e.g., select, capture, or create) a second user input content item, and combine the second user input content item with a third-party user input content item to create a combined user input content item (box 4218). The combine function may provide a plurality of sub-functions or tools to facilitate combining the third-party user input content item with other user input content items. These tools may include a layout function to enable the user to specify the layout relationship of the third-party user input content item relative to other user input content items or within the context of a wider combined display during the presentation of the combined user input content item. The combine function also enables the plurality of enhancements of the enhancement system 208 discussed herein to be applied and used in the creation of a combined content element.

[0305] In some examples, at block 4220, in addition to enabling the combination of multiple content items, the combination functionality also enables unlocking of a previously locked content sharing destination. In some examples, once the user has created the combined user input content item (or once the user has chosen to create it), the interactive client 106 causes the locked content sharing destination to become unlocked. In other words, while the user cannot share the original third-party user input content item to the destination, the interactive system 100 allows the user to share the combined user input content item that includes the original third-party user input content item as a component to the destination.

[0306] According to some examples, at block 4222, method 4200 includes enabling the user to share the combined user input content item to the unlocked first content sharing destination using interactive client 106 by presenting updated destination graphical element 4118b in sharing interface 4112, and making updated destination graphical element 4118b user-selectable to initiate sharing to the (unlocked) content sharing destination associated with element 4118b. Alternatively, in some examples, once the combined user input content item has been created / selected, it may be automatically shared to the newly unlocked content sharing destination based on the user's initial selection of remix button 4120. Remix button 4120 may be located within, adjacent to, or otherwise associated with the destination graphical element of the content sharing destination that the user desires to share.

[0307] In other words, the remix button 4120 can be positioned closer to the destination graphical element of the locked destination than the destination graphical elements of other destinations, thereby alerting the user to the fact that the locked destination can be unlocked by the remix operation and / or that the remix will automatically cause a transmission to the destination. In some examples, the sharing interface can include a tooltip or other message indicating to the user that a particular content sharing destination is not available for the content item being viewed, but can be made available by remixing the content item. The method then ends at closed-loop block 4224.

[0308] Fig.43 is a flow chart illustrating a method 4300 for facilitating the generation of a combined user input content item and sharing the combined user input content item to a content sharing destination according to some examples. In some examples, the method 4300 is performed by referring to Figure 1 The described interactive system 100 is executed by various components, and in particular by the interactive client 106 (as an example of a media application).

[0309] Method 4300 begins at open loop block 4302 and proceeds to block 4304 where, in response to a user selecting remix button 4120 (as described with reference to FIG. Fig.41 and Fig.42 ), presenting a remix interface 4122, and the interactive client 106 causing a digital image feed from at least one camera of the user system 104 to be presented.

[0310] The third-party user input content item and the second (supplemental) digital image feed may be presented simultaneously or concurrently within the remix interface 4122. The interactive client 106 receives the user capture input at block 4306 and, in response thereto, causes one or more digital images from the digital image feed to be captured as the second user input content item at block 4308.

[0311] According to some examples, method 4300 includes storing, at block 4310 , a second user input content item to be accessed by the interactive client 106 .

[0312] Method 4300 proceeds to block 4312, where the third-party user input content item and the second user input content item are combined to generate a combined user input content item. According to some examples, at block 4314, the interactive client 106 causes the updated destination graphical element to be presented within the sharing interface, as shown in FIG. Fig.41 and Fig.42 as described.

[0313] The method 4300 may include determining or checking the content type associated with the combined user input content item at box 4314. Based on the content type, the interactive client 106 may determine the content sharing destinations to be presented to the user and / or the availability of each content sharing destination (e.g., whether it is locked or unlocked). For example, the interactive client 106 may determine that the user has created a combined user input content item that can be classified as an "original" content item, and may check the database 128 for sharing settings related to items of this type. The sharing settings related to items of this type may include, for example: the "original" content item can be shared with another user via a direct message, but not shared in its original form or republished to a public feed. Therefore, the sharing settings may provide information about sharing permissions and / or sharing restrictions.

[0314] At block 4316, a user selection of the updated destination graphical element (e.g., element 4118b) is detected. In response to detecting the user selection of the updated destination graphical element, at block 4318, the interactive system 100 causes the combined user input content item to be published to the unlocked first content sharing destination. The method ends at closed-loop block 4320.

[0315] Figures 44 to 47 is an interface diagram illustrating a series of user interfaces that may be generated and displayed by the interactive client 106 according to some examples.

[0316] Fig.44 4402 is a user interface diagram illustrating a viewing interface 4402 according to some examples. Fig.44 , the active user is viewing third-party user input content item 4404. Fig.44, third-party user input content item 4404 is an augmented image. Specifically, third-party user input content item 4404 includes a photo of a person's hand holding an ice cream cone, with a person also holding an ice cream cone superimposed on the lower right area of ​​the photo. Third-party user input content item 4404 forms part of a collection of content of a third party referred to as "stories." For example, stories can be presented to active users as part of a personal feed or a public feed, or in response to the user accessing the third party's profile and selecting a story. The name of the third party is included in viewing interface 4402, as indicated by reference numeral 4406.

[0317] As indicated by progress bar 4416, presentation of third-party user input content item 4404 is part of a sequential presentation of a group of third-party user input content items.

[0318] The viewing interface 4402 includes an interaction area 4410 that presents the active user with options for interacting with the third-party user input content item 4404 or with the third party 4406. The user may select a camera button 4408 to activate a camera of the user system 104 of the user, e.g., to capture an image in response to the third-party user input content item 4404. The user may also choose to send a message to the third party by selecting a reply graphical element 4412.

[0319] Interaction area 4410 also includes a content sharing graphical element in the example form of a send button 4414. In some examples, if the user selects send button 4414, a content sharing function is invoked in response to determining combined authorization regarding third-party user input content item 4404. The content sharing function causes presentation of a sharing interface 4502 on user system 104.

[0320] Fig.45 45 is a user interface diagram illustrating a sharing interface 4502 according to some examples. The sharing interface 4502 includes a plurality of destination graphic elements. Each destination graphic element identifies a content sharing destination available to an active user. Examples of these elements are shown in FIG. Fig.45 Identified by reference numerals 4504, 4506, 4508, 4510 and 4512.

[0321] The destination graphic element 4504 identifies the first content sharing destination. Fig.45 , the first content sharing destination is "Stories", which means that selection of this option will allow the user to share third-party user input content item 4404 to the user's story (as an example of a content collection). However, in some examples, and as Fig.45As shown, the "Stories" destination is locked. In other words, the user is restricted from (directly) selecting the first content sharing destination. Within the destination graphical element 4504, a remix button 4514 is provided as a selectable unit. A tooltip 4516 below the remix button 4514 informs the user that the destination is unavailable / locked and how to make the destination available: "Add your shots to share to stories".

[0322] In other words, the user is advised via the sharing interface 4502 that before the content can be shared or published to the first content sharing destination (e.g., the user's story, which is generally a collection of content created and published by the user via the interactive client 106 to be accessible by multiple other users of the interactive client 106), a remix operation must be performed to create a combined user input content item (including the third-party user input content item 4404 as a component). According to examples of the present invention, various different tooltips and / or messages indicating that the content sharing destination can be unlocked via the combining function or similar indications may be adopted.

[0323] A destination graphical element such as destination graphical element 4504 may include multiple related destinations, for example, a "stories" destination may be a category containing multiple stories, such as personal stories, shared stories, and / or public stories of an active user. Fig.45 In Stories, when the entire Stories content sharing destination is locked, these sub-destinations are unavailable.

[0324] Destination graphical elements 4506, 4508, 4510, and 4512 identify additional content sharing destinations, each of which is unlocked / available. Destination graphical element 4506 is user selectable to enable the user to share content via MESSAGES TM The application shares the third party user input content item 4404, and the destination graphical element 4508 is user selectable to enable the user to share the content via WHATSAPP TM The third-party user input content item 4404 is shared with the application (both of which are external or third-party applications). The destination graphical element 4510 and the destination graphical element 4512 are user-selectable to enable the user to share the third-party user input content item 4404 to another user of the interactive system 100 by way of a direct message via the interactive client 106. When the destination graphical elements 4506, 4508, 4510 and 4512 are selected, the third-party user input content item 4404 is shareable in its original form to the corresponding content sharing destinations because these content sharing destinations are unlocked / available, in contrast to the destination graphical element 4504, which identifies a locked / unavailable destination (or multiple locked destinations) for the third-party user input content item 4404 in its original form.

[0325] The user may also use the search bar 4518 located at the top area of ​​the sharing interface 4502 to search for other content sharing destinations.

[0326] As described above, the interactive system 100 may detect a sharing setting associated with the third-party user input content item 4404, for example, indicating that the third-party user input content item 4404 is not directly shareable to the first content sharing destination ( Fig.45 The sharing setting may indicate that the third-party user input content item 4404 is shareable or may become shareable to the first content sharing destination as part of the combined user input content item. In response to detecting the setting, the interactive client 106 may generate a sharing interface 4502 with respect to the destination graphical element 4504 as described above.

[0327] Reference Fig.46 , presents a remix interface 4602, also referred to as a combination interface. Remix interface 4602 is automatically activated in response to a user selecting remix button 4514.

[0328] Automatically apply a remix shot or remix enhancement to combine the third-party user input content item 4404 with the second user input content item 4604 (in Fig.46 The first and second user input content items 4612 and 4613 are merged or combined with one another (e.g., an image captured as a “selfie” by an active user) to create a combined user input content item 4612 within remix interface 4602. Thus, in some examples, invoking the combine functionality allows the user to conveniently access the remix feature to create the combined user input content item 4612, and may automatically cause presentation of a digital image feed from a camera device (e.g., a “selfie” camera device) of user system 104 to facilitate capturing a second user input content item.

[0329] The remix interface 4602 includes a set of layout graphical elements 4608, each of which is user-selectable to apply a different layout to the combination of the third-party user input content item 4404 and the second user input content item 4604. Each of the layout options associated with the layout graphical elements 4608 can assign certain portions of the interface canvas to each of the images, or can define the layout of the images relative to each other. The same layout options can be applied to the video content. Once the user is satisfied with the second user input content item 4604 and the layout, the remix capture button 4606 can be selected to finalize the combined user input content item 4612. Fig.46 , a user takes a "selfie" that is superimposed onto a portion of the original third-party user input content item 4404. However, this is merely an example of a layout for positioning two content items relative to each other.

[0330] It should be noted that the remix interface 4602 also includes a context message 4610 that provides contextual information about the combining function (e.g., in this example, indicating that the active user is creating a combined user input content item 4612 based on the original content item of user "John"). The context message 4610 can also be user-selectable to invoke the messaging system 212 of the interactive system and conveniently enable the active user to send the combined user input content item 4612 to the third-party user (e.g., "John") by way of a "reply" to the third party's posting of the third-party user input content item 4404 (e.g., via a message or posting to a feed).

[0331] Fig.47 4502 according to some examples. Once the combined user input content item 4612 is captured by selecting the remix capture button 4606, the user returns to the sharing interface 4502.

[0332] Destination graphic elements 4506, 4508, 4510, and 4512 maintain the same Fig.45 As shown and referenced in Fig.45 described above (in other words, they remain unlocked and available for sharing), but Fig.45 The destination graphical element 4504 of FIG. 4 is replaced by an updated destination graphical element 4702. The updated destination graphical element 4702 is generated and presented by the interactive client 106, and identifies the previously locked content sharing destination (“Story”) that was unlocked by using the remix functionality. The updated destination graphical element 4702 carries the label “Shared to Your Story”. This alerts the user to the fact that the combined user input content item 4612 can be shared to the “Stories” destination, which is not available for sharing of the original third-party user input content item 4404. User selection of the updated destination graphical element 4702 then results in the publication of the combined user input content item 4612, for example, to the user's Story accessible to a limited set of other users of the interactive system, or in some examples to a public feed.

[0333] As described above, a media application (e.g., a messaging application) may allow a user to view a content item and share the content item to other user devices. According to some examples, a content item may be shareable to different content collections (e.g., "stories" or "public feeds") managed by a collection management system 226 of the interactive system 100. A user may wish to interact with a content item that forms part of a collection (e.g., create a "remix" thereof) and / or may wish to view other users' interactions with the content item. By providing methods and systems with improved functionality (e.g., according to reference Figures 48 to 59An example technical problem of configuring a media application to enable a user to create a composite content item from basic content items and to enable such a composite content item to be accessed automatically and in a consolidated manner is addressed.

[0334] Fig.48 is an interaction diagram illustrating a process 4800 involving devices and components of the interactive system 100 according to some examples. Although some examples are described with reference to user input content items, the present disclosure is broadly applicable to other types of digital content. Fig.48 Four user systems 104 are shown in the example form of a first user device 4802, a second user device 4804, a third user device 4806, and a fourth user device 4808. Each device 4802, 4804, 4806, 4808 has installed thereon an instance of a media application in the example form of an interactive client 106. The user devices 4802, 4804, 4806, 4808 communicate with each other and with the interactive server system 102 of the interactive system 100 (and other components of the interactive system 100 as required) using the media application.

[0335] Although in Fig.48 Four user devices are shown, but it should be understood that the example may involve other numbers of user devices (e.g., a large number of user devices) using various components of the interactive system 100 (e.g., the collection management system 226) to share data and content with each other and to share data and content to content collections via the components of the interactive system 100.

[0336] Turning now in more detail to process 4800, at block 4810, a first user of a media application creates and shares a first user input content item using a first user device 4802. Figures 48 to 59 In the described examples, the first user input content item is also referred to as a "base item". The term "base item" refers to the use of content items to create a combined content item. For example, a base item can be a digital image or video clip, or content data including a combination thereof, and can include any number of modifications or enhancements, as described elsewhere. A base item can also include audio content, for example, a base item can be a relatively short video clip (e.g., 15 seconds or 20 seconds) with an audio track selected by the first user to be played simultaneously with the video clip.

[0337] In an example according to the present disclosure, when sharing the base item, the first user specifically chooses to share the base item to the first content collection in the example form of a public or semi-public feed provided by the interactive system 100. Therefore, the first content collection can be a public content collection. However, it should be understood that the techniques described herein can be applied to processes related to various content collections.

[0338] The media application may have a dedicated tab or category that is accessible by all users to view and navigate the public feed (however, it should be appreciated that this may also apply to private or semi-public feeds). The public feed may include a large number of content items, all of which are designated as "public", e.g., accessible by all users of the interactive system 100 (a feed may also be "semi-public" in that it is accessible by a subset of all users). In some examples, the public feed may primarily or exclusively include content in a video format having a limited duration (e.g., only clips shorter than 60 seconds or 30 seconds per clip may be posted to the public feed). The interactive system 100 (e.g., using the collection management system 226) may dynamically update the public feed, and each user of the interactive system 100 may be provided with a personalized or curated public feed, e.g., based on their activity on the media application, items they have liked or interacted with, search history, relationships with other users, or a combination thereof.

[0339] For example, the first user may select a "share to public feed" button after creating the base item. The first user device 4802 causes the base item to be sent to the interactive server system 102 (see Fig.48 Then, at box 4814, the interactive server system 102 makes the base item available to users of the interactive system 100 via a public feed.

[0340] The base item then becomes available for other users of the interactive system 100 to view and interact with (e.g., "like", share, or react to). If the base item is designated as "public", then in some examples, users of the interactive system 100 can access it in various ways. For example, the user can find the base item in the user's curated public feed. Alternatively, the user can view a "live story" that includes the base item, or directly access the first user's profile to view the base item.

[0341] At box 4816, the second user of the interactive system 100 uses the second user device 4804 to access the basic items and uses the basic items to create a combined user input content item ("remixed item"). In other words, the second user creates a combined user input content item including the basic items and one or more other content items. The second user chooses to share the combined user input content item back to the public feed. The second user device 4804 sends the combined user input content item to the interactive server system 102 together with the sharing instruction (arrow 4818). Similarly, at box 4820, the fourth user uses the fourth user device 4808 to access the same basic items and creates another combined user input content item that is shared back to the public feed. The fourth user device 4808 sends the combined user input content item to the interactive server system 102 together with the sharing instruction (arrow 4822).

[0342] The interactive system 100 (eg, using the collection management system 226) then makes the remix items created by both the second user and the fourth user available in a public feed. Fig.48 As shown, at box 4824, the interactive system 100 makes the two remix items available in a second content collection. The second content collection is a collection of all remix items created from the basic items. In other words, the second content collection is a group of content items, and all content items are combined items generated by combining the basic items with another content item. The collection management system 226 can dynamically update the second content collection in response to users of the interactive system 100 creating a public "remix" of the basic items. Similar to the first content collection, the second content collection can be a public content collection that can be accessed by the entire user base of the interactive system 100. However, it should be understood that the technology described herein can be applied to processes related to various content collections (e.g., semi-public content collections that have certain restrictions on users who are allowed to access them).

[0343] A third user of the interactive system 100 then requests to view all "remixes" made from the base item across the user community of the interactive system 100 using the third user device 4806. For example, the third user may be viewing the base item in a public feed and is interested in seeing how other users interact with, react to, or build upon the content in the base item. The interactive system 100 may provide the third user with a list of remixes that the base item has been made by other users (in the form of a video or video) on the viewing interface of the third user device 4806. Fig.48 In the case of the second content collection, the second user and the fourth user) are provided with an indication of the remix with the user-selectable element. In response to the third user requesting to view all "remixes", for example by selecting the user-selectable element at arrow 4826, the collection interface can be presented on the third user device 4806. At block 4828, the collection interface allows the third user to navigate the second content collection.

[0344] The second public content set, for example, can consist only of a combined user input content item, which is created using the first user input content item as a basic item and is shared back to the first public content set (e.g., a public feed). In the case where the first user input content item and the supplementary content added to create the combined user input content item both include visual content (e.g., still images and / or videos), the second public set therefore provides the user with an automatic and convenient view of all visual responses to the basic (original) items of the first user. Each combined user input content item in the second public content set may include an attribute element identifying the first user. In some examples, the collection interface may include an attribute visual element that identifies the first user or basic item separately from a list or display of a "remix" item.

[0345] Fig.49 4900 is a flowchart illustrating a method 4900 for enabling an active user to access a collection interface in a media application (e.g., an interactive client 106) according to some examples. Thus, the method 4900 may be performed by components of the interactive system 100. For ease of reference and understanding, the method 4900 is described in detail in conjunction with FIG. Fig.49 and Fig.50 Reference again in the description provided Fig.48 Four users and their corresponding user devices 4802, 4804, 4806 and 4808.

[0346] Method 4900 starts at open loop frame 4902 and proceeds to frame 4904, in which the media application causes a first public content set in the example form of a public feed to be presented on a second user device 4804 of a second user. The public feed includes a number of user input content items that the second user can scroll, react to, share, etc. The second user wishes to react to one of these user input content items as a basic item by creating a visual reaction to it, and at frame 4906, the media application receives an indication of a combined user input related to the basic item from the second user device 4804 of the second user. The indication of the combined user input related to the basic item can be, for example, a selection of a combined graphical element by the second user. The media application can present a combined graphical element in conjunction with the basic item on a user interface, wherein the combined graphical element is user selectable to invoke a combination function ("remix" function) of the media application.

[0347] The method 4900 then proceeds to block 4908, where, in response to receiving an indication to combine user input from the second user device 4804, the media application invokes a combine function of the media application on the second user device 4804. The combine function allows the second user to access the second user input content item and combine the first user input content item (the base item) with the second user input content item to create a first combined user input content item.

[0348] The remix system 206 of the interactive system 100 can detect third-party authorization of the base item to be combined with other content items to create a combined user input content item. For example, an authorization setting associated with the base item indicating the authorization can be detected. When creating or publishing an item, the content creator (e.g., the creator of the base item) can be presented with an option to allow or not allow the item to be used in combination with other content items created or initiated by other users of the interactive system 100. The allow / not allow is then recorded and stored in the database 128 as an authorization setting associated with the item and is accessed to perform detection of third-party authorization.

[0349] A remix interface may be presented by the camera system 204 and specifically by the remix system 206 to enable a user to capture another content item and combine the other content item with a base item viewed in a common feed to generate a combined user input content item. The remix interface presents various layout tools for laying out multiple content items relative to each other and within certain layouts. Various enhancement and modification tools may be applied to the combined content within the context of the remix interface.

[0350] Once the combined user input content item has been created, the media application can present various sharing destinations to the second user. The second user selects to share the first combined user input content item to the public feed, and the interactive system 100 makes the first combined user input content item available in the public feed. In addition, at block 4910, in response to sharing the first combined user input content item to the public feed, the collection management system 226 causes the first combined user input content item to be stored in association with the base item in the second public content collection, as described above with reference to Fig.48 This allows users accessing the second public content collection to view the newly created "remix" of the base item.

[0351] At block 4912, method 4900 includes receiving an indication of a combined collection presentation user input related to the base item from a third user device 4806 of a third user of the media application. This may include causing presentation of a combined collection presentation graphical element in conjunction with the base item by the media application on the third user device 4806. The combined collection presentation graphical element is then user-selectable to invoke a combined collection presentation function.

[0352] In response to receiving an indication of user input for presentation of a combined set (e.g., a user selection of a graphical element for presentation of a combined set), a combined set presentation function of the media application is invoked on the third user device 4806 (block 4914). The combined set presentation function causes presentation as described in reference to FIG. Fig.48 The described collection interface, examples of which are further described below. At block 4916, the collection interface enables a third user to navigate the second public content collection, and may also automatically present combined elements to allow the third user to create their own visual response to the base item. Method 4900 ends at closed-loop block 4918.

[0353] Fig.50 5000 is a flow chart illustrating a method 5000 for enabling an active user to access a collection interface in a media application (eg, an interactive client 106 ), according to some examples. The method 5000 may be performed by components of the interactive system 100 .

[0354] The method 5000 starts at open-loop block 5002 and proceeds to block 5004, where the media application causes the common feed to be presented on a fourth user device 4808 of a fourth user. At block 5006, the media application receives an indication of a combined user input related to an elementary item from the fourth user device 4808 of the fourth user. Similar to the second user, at block 5008 the fourth user is then enabled to create a combined user input content item by activating a combining function.

[0355] Method 5000 proceeds to block 5010 where a third user input content item is accessed and combined with the base item to create a second combined user input content item. It should be appreciated that this process may include laying out the third user input content item relative to the first user input content item, as described elsewhere.

[0356] At box 5012, in response to determining that the second combined user input content item is shared to the public feed by the fourth user, the interactive system 100 causes the second combined user input content item to be stored in the second public content set in association with the basic item. The interactive system 100 can access the metadata of the second combined user input content item, identifying it as a visual combination response to the first user input content item. Based on this identification, the interactive system 100 automatically adds the second combined user input content item to the second public content set. In some examples, the combined user input content item is added to the second public content set only when it is shared back to the public feed or designated as "public" (or similar designation), to ensure that the combined user input content item intended for private use or intended only for connection with the user or "friend" sharing is not added to the second public content set for public navigation.

[0357] A user of the interactive system 100 (eg, a third user) can then access the second public content collection to view the first combined user input content item, the second combined user input content item, and any other visual combined responses to the base item (block 5014). The method 5000 then ends at closed-loop block 5016.

[0358] Figure 51 to Figure 54 is an interface diagram illustrating a series of user interfaces that may be generated and displayed by the interactive client 106 according to some examples.

[0359] Fig.51 5102. When scrolling through the public feed 5104, the active (viewing) user can view the basic items 5106. The public feed 5104 is accessible via the dedicated public feed tab 5116 of the interactive client 106. The basic item 5106 is a video clip of a person dancing, with an audio clip (of the same duration) playing at the same time. Fig.51 , base item 5106 is created and posted to public feed 5104 by a first user "John" using a user device running an instance of interactive client 106. For example, the first user "John" may have captured a person dancing to the tune of a third-party song, and the third-party song may be selected to be added to the video clip as a feature of the interactive system 100.

[0360] Viewing interface 5102 includes a profile data element 5108 identifying the first user, and the active user can select profile data element 5108 to view the profile of the first user. Viewing interface 5102 also includes an audio attribute data element 5110 (as an example of an audio identifier) ​​identifying the audio clip and an enhancement identifier 5112 identifying the enhancement applied to the base item by the first user. In the case where the audio clip is a third-party song, the audio identifier can therefore give appropriate identification to the artist associated with the song.

[0361] In response to user selection of action menu button 5114, action menu 5202 is presented.

[0362] Fig.52 5202.

[0363] In some examples, the active user can perform a different action, such as a press-and-hold gesture on the base item 5106, to invoke a combination function (wherein the action menu is not presented).

[0364] According to some examples, Fig.53 5302 (also referred to as a combination interface). In response to a user selection of the remix button 5204, the remix interface 5302 is presented by the camera system 204, and specifically by the remix system 206, to provide a modular camera that enables the user to capture other content items and combine the other content items with the third-party content items (primary items) viewed in the viewing interface 5102 to generate a combined user input content item. The remix interface 5302 presents various enhancement and modification tools that can be applied to the combined content within the context of the remix interface 5302 in an enhancement carousel 5304, and also presents various camera controls 5306.

[0365] Although in Fig.53 5304 with selected shots 5312 is shown, but in other examples, the remix interface 5302 may only include similar Fig.46 The remix capture button of the remix interface 4602 is shown (without any footage or enhancements).

[0366] from Fig.53 Obviously, in some examples, the user can capture the second user input content item so that it is superimposed on the primary item. Fig.53 , the active user captures a second user input content item 5308 in the form of an example of a video reaction to a video clip of the base item 5106. However, as in Fig.46 As shown in remix interface 4602, various other layout options may be applied.

[0367] The media application on the active user's device includes camera functionality such that a digital image feed from at least one camera of the user's device is presented in the remix interface 5302. The media application receives user capture input from the active user via the remix interface 5302 and, in response thereto, captures digital image content (in Fig.53 In the example above, this is in the form of a video clip) as a second user input content item 5308 and is stored as accessible by a media application on the user device. Once captured, this will create a combined user input content item 5310.

[0368] exist Fig.53, combined user input content item 5310 is a combined video clip showing both the video of base item 5106 and the video reaction / response of the active user. In addition, the audio content associated with base item 5106 is carried into combined user input content item 5310 so that when combined user input content item 5310 is played, the "original" audio clip from base item 5106 is also played. Thus, the audio identifier of base item 5106 can be carried into combined user input content item 5310 so that when combined user input content item 5310 is presented to another user, it is presented in conjunction with the audio identifier.

[0369] After creating the combined user input content item 5310 , the active user may be enabled to save the combined user input content item 5310 , for example to a “memories” tab, and may also share the combined user input content item 5310 , for example to a public feed of the interactive system 100 .

[0370] Fig.54 5402 is a user interface diagram showing a bottom area of ​​a finalized and published combined user-entered content item 5310 viewed by a third-party user in viewing interface 5402. The third-party user views the combined user-entered content item 5310 via a public feed tab 5116 that provides access to a public feed.

[0371] Viewing interface 5402 includes profile data element 5404 identifying the user (in this case, "Sally") responsible for creating the "remix", and base item attribute element 5408. Attribute element 5408 identifies the creator ("first user", in this case "John") of the base item of the original content that is "remixed". In some examples, base item attribute element 5408 can identify the base item itself. Element 5404 and element 5408 are user-selectable so that a third-party user can view the profile of the user associated with element 5404 and element 5408. In some examples, base item attribute element 5408 is user-selectable so that the (original) base item is presented. In other words, a third-party user can select base item attribute element 5408 to view the base item without any combined content.

[0372] View interface 5402 also includes an audio attribute data element 5110 (as an example of an audio identifier) ​​that identifies the audio clip. Fig.51 As described above, in the example using a third party song, the audio identifier gives appropriate credit to the artist. Enhancement identifier 5406 identifies the enhancement that is applied to the combined user input content item.

[0373] The interactive system 100 may be configured to handle attribute data differently for users with different profile types. For example, if the user has a public profile, an attribute element similar to the basic item attribute element 5408 may be shown, which includes the identity of the creator. If the user has a private profile, the attribute element may simply indicate, for example, that the content item has been "remixed" from a public feed, without revealing the identity of the creator.

[0374] As described above, an active user may wish to view a combined user input content item (a "remix") made from certain base items in a merged location. Fig.55 and Fig.56 is a user interface diagram illustrating different entry points to a collection interface providing such views, as well as other functionality, according to some examples.

[0375] exist Fig.55 , viewing interface 5502 displays elementary items 5504 being viewed on public feed 5104. Viewing interface 5502 presents combined set presentation graphical element 5506 to the active user. Combined set presentation graphical element 5506 is shown in conjunction with elementary items 5504 and is user selectable to invoke combined set presentation functionality. In other words, in response to receiving combined set presentation user input in the form of an example user selection of combined set presentation graphical element 5506, combined set presentation functionality is invoked.

[0376] In some examples, the interactive system 100 may therefore determine that the base item has been used to create one or more combined user input content items that are publicly shared in the interactive system 100, and in response, cause the combined collection presentation graphical element 5506 to be automatically presented in the viewing interface 5502. If the base item has not been shared publicly, e.g., only shared to a particular user as a direct message or to a private story of the sharing user, the combined collection presentation graphical element is not presented.

[0377] exist Fig.56 , viewing interface 5602 does not automatically display a combined collection presentation graphical element. Instead, in response to a user selection of an action menu button (similar to action menu button 5114) while viewing base item 5604, an action menu 5608 is presented. Action menu 5608 includes a combined collection presentation graphical element 5606 ("View All Remixes").

[0378] Fig.57 5702 is a user interface diagram illustrating a collection interface 5702 according to some examples. Collection interface 5702 is presented in response to user selection of combination collection presentation graphical element 5606 and allows viewing of all combined user input content items created from base items 5604 by user navigation.

[0379] The collection interface 5702 includes a basic item thumbnail 5704 identifying the basic item 5604 and a written basic item identifier 5706 at the top portion. Fig.57 In this case, combined data 5710 indicating that the basic item 5604 has been "remixed" a total of 106 times is also displayed in the top portion.

[0380] Below the top portion, the collection interface 5702 presents a scrollable (or otherwise navigable) grid 5708 of all the "remixes" created from the base item 5604. Fig.57 In the case of a "remix", a viewing user can browse all "remixes", select them, react to them, etc. This collection of "remixes" that all link to the same basic item is referred to in some examples as the "second public content collection" (in some examples, the general public feed is the "first public content collection").

[0381] The collection interface 5702 presents a remix button 5712 to allow viewing of other “remixes” of the user-created base item 5604, and if the user chooses to do so, to be added to the second public content collection. Thus, in some examples, activating a combined collection presentation function of a media application on a user device (e.g., user system 104) may include causing a combined graphic element (e.g., a remix button) to be presented within the collection interface, wherein the combined graphic element is user-selectable to invoke a combination function, thereby allowing the user to access (e.g., capture) another user input content item, and combine the original (base) user input content item with the other user input content item to create another combined user input content item. This provides a convenient way for viewing users to not only view all “remixes” linked to certain original content items, but also to automatically choose to create their own “remixes.” “Remixes” can be viewed as a way to respond, react, or reply to another user’s posting, and examples of the present invention therefore provide technical tools that promote community creativity, thereby allowing users to express themselves and interact on such communities. In some examples, such as in Fig.57 In the case of , a "remix" is a visually combined response to the creator's original / base item, and each user who creates a combined user input content item can add new visual content in response to the original / base item.

[0382] Fig.58 5 is a user interface diagram showing a viewing interface 5802 presenting a view of a base item from which a combined user input content item has been made according to some examples. Viewing interface 5802 is presented to the creator of the base item. In other words, when a user is viewing their own user input content item that has been posted to a public feed, an interface such as viewing interface 5802 may be presented.

[0383] Viewing interface 5802 includes combined information unit 5804. In some examples, combined information unit 5804 displays a count of all "remixes" made by users from the base item. The count can include all "remixes" regardless of whether they are publicly shared. In other words, private content such as "remixes" shared only via direct messages can also be added to the count. This provides the creator with an indication of the degree of user interaction with the base item, as well as other counts such as the number of likes (see element 5806) and the number of shares (see element 5808).

[0384] In some examples, the combined information unit 5804 is user-selectable to invoke a combined set presentation function. In other words, in response to receiving a combined set presentation user input in the form of an example user selection of the combined information unit 5804, the combined set presentation function is invoked to present a collection interface to the creator of the base item. However, the collection interface only includes "remixes" that have been publicly shared or published, so the number of combined user input content items in the second public content collection may be less than Fig.58 The count displayed to the creator in .

[0385] To inform users and facilitate use of the aforementioned features, a tooltip may be displayed to users to remind them to choose to create a composite user input content item from base items viewed in a public feed. Fig.59 5902 presents a view of the basic items in the common feed tab 5116 according to some examples. The action menu button 5114 is presented with a tool tip 5904 pointing to the action menu button 5114. This reminds the user of the options available in the action menu, such as reference Fig.52 described.

[0386] Data communication architecture

[0387] Fig.60 6 is a schematic diagram illustrating the structure of a message 6000 according to some examples, the message 6000 being generated by an interactive client 106 for transmission to another interactive client 106 via an interactive server 124. The content of a particular message 6000 is used to populate a message table 306 stored in a database 304 accessible by the interactive server 124. Similarly, the content of the message 6000 is stored in memory as "in-transit" or "in-flight" data of the user system 104 or the interactive server 124. The message 6000 is shown as including the following example components:

[0388] Message identifier 6002: A unique identifier that identifies the message 6000.

[0389] Message text payload 6034 : Text to be generated by the user via the user interface of the user system 104 and included in the message 6000 .

[0390] Message image payload 6004: Image data captured by the camera component of the user system 104 or retrieved from the memory component of the user system 104 and included in the message 6000. The image data for the message 6000 sent or received may be stored in the image table 6006.

[0391] Message Video Payload 6008: Video data captured by the camera component or retrieved from the memory component of the user system 104 and included in the message 6000. The video data for the message 6000 sent or received may be stored in the video table 6010.

[0392] Message audio payload 6012 : Audio data captured by a microphone or retrieved from a memory component of the user system 104 and included in the message 6000 .

[0393] Message enhancement data 6014: Enhancement data (e.g., filters, stickers, or other annotations or enhancements) representing enhancements to be applied to the message image payload 6004, message video payload 6008, or message audio payload 6012 of the message 6000. The enhancement data for a sent or received message 6000 may be stored in an enhancement table 6016.

[0394] · Message duration parameter 6018: A parameter value indicating the amount of time in seconds that the content of the message (e.g., message image payload 6004, message video payload 6008, message audio payload 6012) is to be presented to the user via the interactive client 106 or made accessible to the user.

[0395] Message geolocation parameters 6020: Geolocation data (e.g., latitude and longitude coordinates) associated with the content payload of the message. Multiple message geolocation parameter 6020 values ​​may be included in the payload, each of these parameter values ​​being associated with a content item included in the content (e.g., a specific image within a message image payload 6004 or a specific video in a message video payload 6008).

[0396] Message story identifier 6022: An identifier value that identifies one or more content collections (e.g., a "story" identified in stories table 6024) associated with a particular content item in message image payload 6004 of message 6000. For example, an identifier value may be used to associate multiple images within message image payload 6004 with each of multiple content collections.

[0397] Message tags 6026: Each message 6000 may be tagged with a plurality of tags, each of which indicates the subject of the content included in the message payload. For example, where a particular image included in the message image payload 6004 depicts an animal (e.g., a lion), a tag value may be included within the message tags 6026 indicating the relevant animal. The tag value may be manually generated based on user input, or may be automatically generated using, for example, image recognition.

[0398] • Message sender identifier 6028: An identifier (eg, a messaging system identifier, an email address, or a device identifier) ​​that indicates the user of the user system 104 on which the message 6000 was generated and from which the message 6000 was sent.

[0399] • Message recipient identifier 6030: Indicates an identifier of the user of the user system 104 to which the message 6000 is addressed (eg, a messaging system identifier, an email address, or a device identifier).

[0400] The content (e.g., value) of each component of message 6000 may be a pointer to a location in a table in which the content data value is stored. For example, the image value in message image payload 6004 may be a pointer to a location in image table 6006 (or the address of a location in image table 6006). Similarly, the value in message video payload 6008 may point to data stored in video table 6010, the value stored in message enhancement data 6014 may point to data stored in enhancement table 6016, the value stored in message story identifier 6022 may point to data stored in story table 6024, and the values ​​stored in message sender identifier 6028 and message recipient identifier 6030 may point to user records stored in entity table 6032.

[0401] Machine Architecture

[0402] Fig.616100 within which instructions 6102 (e.g., software, programs, applications, applet, app, or other executable code) may be executed for causing the machine 6100 to perform any one or more of the methods discussed herein. For example, the instructions 6102 may cause the machine 6100 to perform any one or more of the methods described herein. The instructions 6102 convert a general-purpose, unprogrammed machine 6100 into a specific machine 6100 that is programmed to perform the functions described and illustrated in the manner described. The machine 6100 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 6100 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 6100 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web device, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing instructions 6102 specifying actions to be taken by the machine 6100. In addition, although a single machine 6100 is shown, the term "machine" should also be taken to include a collection of machines that individually or jointly execute instructions 6102 to perform any one or more of the methods discussed herein. For example, the machine 6100 may include any of the user system 104 or a plurality of server devices that form part of the interactive server system 102. In some examples, the machine 6100 may also include both a client system and a server system, wherein some operations of a particular method or algorithm are performed on the server side, and wherein some operations of a particular method or algorithm are performed on the client side.

[0403] The machine 6100 may include a processor 6104, a memory 6106, and an input / output (I / O) component 6108 that may be configured to communicate with each other via a bus 6110. In an example, the processor 6104 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 6112 that executes instructions 6102 and a processor 6114. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. Although Fig.61 Multiple processors 6104 are shown, but the machine 6100 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0404] The memory 6106 includes a main memory 6116, a static memory 6118, and a storage unit 6120, all of which are accessible by the processor 6104 via the bus 6110. The main memory 6106, the static memory 6118, and the storage unit 6120 store instructions 6102 that embody any one or more of the methodologies or functions described herein. The instructions 6102 may also reside, completely or partially, within the main memory 6116, within the static memory 6118, within the machine-readable medium 6122 within the storage unit 6120, within at least one of the processors 6104 (e.g., within a cache memory of a processor), or within any suitable combination thereof during execution thereof by the machine 6100.

[0405] The I / O components 6108 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O components 6108 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It will be appreciated that the I / O components 6108 may include Fig.616108. In various examples, the I / O components 6108 may include a user output component 6124 and a user input component 6126. The user output component 6124 may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The user input component 6126 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a point-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or other pointing instrument), a tactile input component (e.g., a physical button, a touch screen that provides the location and force of a touch or touch gesture, or other tactile input component), an audio input component (e.g., a microphone), etc.

[0406] In another example, the I / O component 6108 may include a biometric component 6128, a motion component 6130, an environment component 6132, or a position component 6134, as well as various other components. For example, the biometric component 6128 includes components for determining expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 6130 includes: an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope).

[0407] The environmental components 6132 include, for example, one or more cameras (with still image / photo and video capabilities), an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers to determine ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones to determine background noise), a proximity sensor component (e.g., an infrared sensor to determine nearby objects), a gas sensor (e.g., a gas detection sensor for detecting concentrations of hazardous gases for safety purposes or for measuring pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.

[0408] With respect to cameras, user system 104 may have a camera system including, for example, a front-facing camera on a front surface of user system 104 and a rear-facing camera on a rear surface of user system 104. The front-facing camera may, for example, be used to capture still images and videos (e.g., “selfies”) of a user of user system 104, which may then be enhanced with the enhancement data (e.g., filters) described above. For example, the rear-facing camera may be used to capture still images and videos in a more traditional camera mode, where the images are similarly enhanced with the enhancement data. In addition to the front-facing camera and the rear-facing camera, user system 104 may also include a 360° camera for capturing 360° photos and videos.

[0409] Additionally, the camera system of the user system 104 may include dual rear cameras (e.g., a main camera and a depth sensing camera), or even triple, quad, or quintuple rear camera configurations on the front and back sides of the user system 104. For example, these multiple camera systems may include a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, a macro camera, and a depth sensor.

[0410] The location component 6134 includes a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or a barometer that determines the air pressure that can obtain the altitude), an orientation sensor component (e.g., a magnetometer), etc.

[0411] Various technologies may be used to achieve communication. The I / O components 6108 also include a communication component 6136 that is operable to couple the machine 6100 to a network 6138 or device 6140 via a corresponding coupling or connection. For example, the communication component 6136 may include a network interface component or another suitable device that interfaces with the network 6138. In other examples, the communication component 6136 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, Components (e.g. Low power consumption), Components and other communication components that provide communication via other modalities. Device 6140 can be another machine or any peripheral device among various peripheral devices (for example, a peripheral device coupled via a serial bus (USB)).

[0412] In addition, the communication component 6136 can determine the identifier or include components that are operable to determine the identifier. For example, the communication component 6136 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for determining one-dimensional bar codes such as universal product code (UPC) bar codes, multi-dimensional bar codes such as quick response (QR) codes, Aztec codes, data matrix, data symbol (Dataglyph), maximum code (MaxiCode), PDF417, UltraCode, UCC RSS-2D bar codes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be obtained via the communication component 6136, such as a location obtained via Internet Protocol (IP) geolocation, ... Positioning derived from signal triangulation, positioning derived via determining NFC beacon signals that can indicate a specific position, etc.

[0413] Various memories (e.g., main memory 6116, static memory 6118, and memory of processor 6104) and storage unit 6120 may store one or more sets of instructions and data structures (e.g., software) embodied or used by any one or more of the methods or functions described herein. These instructions (e.g., instructions 6102) when executed by processor 6104 cause various operations to implement the disclosed examples.

[0414] The instructions 6102 may be sent or received via a network interface device (e.g., a network interface component included in the communications component 6136) using a transmission medium and using any of several well-known transmission protocols (e.g., HTTP) over the network 6138. Similarly, the instructions 6102 may be sent or received via a coupling (e.g., a peer-to-peer coupling) with the device 6140 using a transmission medium.

[0415] System with head wearable device

[0416] Fig.62 A user system 104 is shown including a head wearable device 116 with a selector input device according to some examples. Fig.62 is a high-level functional block diagram of an example head wearable device 116 communicatively coupled to a mobile device 114 and a server system 6204 via various networks 6206.

[0417] The head wearable device 116 includes one or more cameras, each of which may be, for example, a visible light camera 6208 , an infrared transmitter 6210 , and an infrared camera 6212 .

[0418] The mobile device 114 is connected to the head wearable device 116 using both a low power wireless connection 6214 and a high speed wireless connection 6216. The mobile device 114 is also connected to the server system 6204 and the network 6206.

[0419] The head wearable device 116 also includes two image displays of the optical assembly's image display 6218. The two image displays 6218 of the optical assembly include one image display associated with the left lateral side of the head wearable device 116 and one image display associated with the right lateral side of the head wearable device 116. The head wearable device 116 also includes an image display driver 6220, an image processor 6222, low power circuits 6224, and high speed circuits 6226. The image displays 6218 of the optical assembly are used to present images and videos to a user of the head wearable device 116, including images that may include a graphical user interface.

[0420] The image display driver 6220 commands and controls the image display 6218 of the optical assembly. The image display driver 6220 may deliver the image data directly to the image display 6218 of the optical assembly for presentation, or may convert the image data into a signal or data format suitable for delivery to an image display device. For example, the image data may be video data formatted according to a compression format such as H.264 (MPEG-4 Part 10), HEVC, Theora, Dirac, RealVideo RV40, VP8, VP9, ​​etc., and the static image data may be formatted according to a compression format such as Portable Network Graphics (PNG), Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), or Exchangeable Image File Format (Exif), etc.

[0421] The head wearable device 116 includes a frame and rods (or temples) extending from lateral sides of the frame. The head wearable device 116 also includes a user input device 6228 (e.g., a touch sensor or a push button) including an input surface on the head wearable device 116. The user input device 6228 (e.g., a touch sensor or a push button) is used to receive input selections from a user to manipulate a graphical user interface of the presented image.

[0422] For head wearable device 116 Fig.62The components shown in are located on one or more circuit boards (e.g., PCBs or flexible PCBs) in the frame or temples. Alternatively or additionally, the depicted components may be located in a block, frame, hinge, or nose bridge of the head wearable device 116. The left visible light camera and the right visible light camera 6208 may include digital camera elements, such as complementary metal oxide semiconductor (CMOS) image sensors, charge coupled devices, camera lenses, or any other corresponding visible light or light capturing elements that can be used to capture data (including images of scenes with unknown objects).

[0423] The head wearable device 116 includes a memory 6202 that stores instructions for performing a subset or all of the functions described herein. The memory 6202 may also include a storage device.

[0424] like Fig.62 As shown, the high-speed circuit 6226 includes a high-speed processor 6230, a memory 6202, and a high-speed wireless circuit 6232. In some examples, the image display driver 6220 is coupled to the high-speed circuit 6226 and is operated by the high-speed processor 6230 to drive the left image display and the right image display of the image display 6218 of the optical assembly. The high-speed processor 6230 can be any processor capable of managing high-speed communications and operations of any general computing system required by the head wearable device 116. The high-speed processor 6230 includes the processing resources required to manage high-speed data transmission on the high-speed wireless connection 6216 to the wireless local area network (WLAN) using the high-speed wireless circuit 6232. In some examples, the high-speed processor 6230 executes an operating system (e.g., a LINUX operating system) or other such operating system of the head wearable device 116, and the operating system is stored in the memory 6202 for execution. In addition to any other duties, the high-speed processor 6230 that executes the software architecture for the head wearable device 116 is used to manage data transmission with the high-speed wireless circuit 6232. In some examples, high-speed wireless circuit 6232 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, also referred to herein as WiFi. In some examples, high-speed wireless circuit 6232 can implement other high-speed communication standards.

[0425] The low power wireless circuit 6234 and the high speed wireless circuit 6232 of the head wearable device 116 may include a short range transceiver (Bluetooth TM) and a wireless wide area network transceiver, a wireless local area network transceiver, or a wide area network transceiver (e.g., cellular or WiFi). Mobile device 114 - including a transceiver that communicates via low power wireless connection 6214 and high speed wireless connection 6216 - can be implemented using details of the architecture of head wearable device 116, and can also be implemented using other elements of network 6206.

[0426] The memory 6202 includes any storage device capable of storing various data and applications, including camera data generated by the left and right visible light cameras 6208, the infrared camera 6212, and the image processor 6222, and the generated images for display on the image display 6218 of the optical assembly through the image display driver 6220. Although the memory 6202 is shown as integrated with the high-speed circuit 6226, in some examples, the memory 6202 can be a separate independent element of the head wearable device 116. In some such examples, electrical wiring can provide a connection from the image processor 6222 or the low-power processor 6236 to the memory 6202 through a chip including the high-speed processor 6230. In some examples, the high-speed processor 6230 can manage the addressing of the memory 6202 so that the low-power processor 6236 will start the high-speed processor 6230 at any time when a read or write operation involving the memory 6202 is required.

[0427] like Fig.62 As shown, the low-power processor 6236 or the high-speed processor 6230 of the head wearable device 116 can be coupled to a camera device (a visible light camera device 6208, an infrared emitter 6210 or an infrared camera device 6212), an image display driver 6220, a user input device 6228 (for example, a touch sensor or a push button), and a memory 6202.

[0428] The head wearable device 116 is connected to a host computer. For example, the head wearable device 116 is paired with the mobile device 114 via a high-speed wireless connection 6216 or connected to a server system 6204 via a network 6206. The server system 6204 can be one or more computing devices as part of a service or network computing system, for example, including a processor, a memory, and a network communication interface to communicate with the mobile device 114 and the head wearable device 116 through the network 6206.

[0429] The mobile device 114 includes a processor and a network communication interface coupled to the processor. The network communication interface allows communication via a network 6206, a low-power wireless connection 6214, or a high-speed wireless connection 6216. The mobile device 114 may further store at least a portion of the instructions for generating two-channel audio content in a memory of the mobile device 114 to implement the functions described herein.

[0430] The output components of the head wearable device 116 include visual components, such as a display such as a liquid crystal display (LCD), a plasma display panel (PDP), a light emitting diode (LED) display, a projector, or a waveguide. The image display of the optical component is driven by the image display driver 6220. The output components of the head wearable device 116 also include acoustic components (e.g., speakers), tactile components (e.g., vibration motors), other signal generators, etc. The input components of the head wearable device 116, the mobile device 114, and the server system 6204 (e.g., user input devices 6228) may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric input, optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing instruments), tactile input components (e.g., physical buttons, touch screens or other tactile input components that provide the location and force of touch or touch gestures), audio input components (e.g., microphones), etc.

[0431] The head wearable device 116 may also include additional peripheral elements. Such peripheral elements may include biometric sensors, additional sensors, or display elements integrated with the head wearable device 116. For example, the peripheral elements may include any I / O components including output components, motion components, position components, or any other such components described herein.

[0432] For example, the biometric component includes a component for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biological signals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion component includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The position component includes a positioning sensor component for generating position coordinates (e.g., a global positioning system (GPS) receiver component), a Wi-Fi or Bluetooth transceiver for generating positioning system coordinates, an altitude sensor component (e.g., an altimeter or barometer that detects air pressure, and the altitude can be obtained according to the air pressure), an orientation sensor component (e.g., a magnetometer), etc. Such positioning system coordinates can also be received from the mobile device 114 via the low power wireless connection 6214 and the high speed wireless connection 6216 via the low power wireless circuit 6234 or the high speed wireless circuit 6232.

[0433] Software Architecture

[0434] Fig.63 6300 is a block diagram illustrating a software architecture 6302 that can be installed on any one or more of the devices described herein. The software architecture 6302 is supported by hardware, such as a machine 6304 including a processor 6306, a memory 6308, and an I / O component 6310. In this example, the software architecture 6302 can be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 6302 includes layers such as an operating system 6312, a library 6314, a framework 6316, and an application 6318. In operation, the application 6318 activates an API call 6320 through the software stack and receives a message 6322 in response to the API call 6320.

[0435] The operating system 6312 manages hardware resources and provides public services. The operating system 6312 includes, for example, a kernel 6324, a service 6326, and a driver 6328. The kernel 6324 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 6324 provides memory management, processor management (e.g., scheduling), component management, network and security settings, and other functions. The service 6326 can provide other public services for other software layers. The driver 6328 is responsible for controlling or interfacing with the underlying hardware. For example, the driver 6328 may include a display driver, a camera driver, or Low-power drivers, Flash drivers, Serial communication drivers (e.g., USB drivers), Drivers, audio drivers, power management drivers, etc.

[0436] The library 6314 provides a common low-level infrastructure used by the application 6318. The library 6314 may include a system library 6330 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 6314 may include an API library 6332, such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for presenting in two dimensions (2D) and three dimensions (3D) in graphics content on a display), a database library (e.g., SQLite providing various relational database functions), a web library (e.g., WebKit providing web browsing functions), etc. Library 6314 may also include various other libraries 6334 to provide many other APIs to application 6318.

[0437] The framework 6316 provides a common high-level infrastructure used by the applications 6318. For example, the framework 6316 provides various graphical user interface functions, advanced resource management, and advanced positioning services. The framework 6316 can provide a wide range of other APIs that can be used by the applications 6318, some of which can be specific to a particular operating system or platform.

[0438] In an example, applications 6318 may include a home application 6336, a contacts application 6338, a browser application 6340, a book reader application 6342, a positioning application 6344, a media application 6346, a messaging application 6348, a game application 6350, and a variety of other applications such as third-party applications 6352. Applications 6318 are programs that execute functions defined in the program. Various programming languages ​​may be used to create one or more of the applications 6318 constructed in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C language or assembly language). In a specific example, third-party applications 6352 (e.g., those created by an entity other than the vendor of a particular platform using ANDROID TM or IOS TM SDK-developed applications) can be used on platforms such as IOS TM ANDROID TM , Mobile software running on the mobile operating system of the phone or other mobile operating system. In this example, third-party application 6352 can activate API call 6320 provided by operating system 6312 to facilitate the functions described in this article.

[0439] Example

[0440] In view of the implementation of the above-mentioned subject matter, the present application discloses the following list of examples, wherein a single feature of the example or more than one feature of the example combined, and optionally, combined with one or more features of one or more other examples, are additional examples that also fall within the scope of the disclosure of the present application.

[0441] Example 1 is a method for facilitating content sharing among a first user, a second user, and a third user, the method comprising: causing a first public content set to be presented on a second user system of the second user, the first public content set comprising a plurality of user-input content items; receiving an indication of a combined user input related to a first user-input content item from the second user system of the second user, the first user-input content item forming a part of the first public content set and comprising at least one of image content or video content, the first user-input content item having been shared to the first public content set by the first user using the first user system; in response to receiving the indication of the combined user input from the second user system, causing a combined function to be called on the second user system, the combined function accessing the second user-input content item and combining the first user-input content item with the second user system. The second user input content items are combined to create a first combined user input content item, the second user input content item including at least one of image content or video content; in response to determining that the first combined user input content item is shared to the first public content collection by the second user, the first combined user input content item is stored in a second public content collection in association with the first user input content item; an indication of a combined collection presentation user input related to the first user input content item is received from a third user system of the third user; and in response to receiving the indication of the combined collection presentation user input, a combined collection presentation function is called on the third user system, the combined collection presentation function being used to: cause a collection interface to be presented on the third user system; and enable the third user to navigate the second public content collection within the collection interface.

[0442] In Example 2, the subject matter of Example 1 includes: causing the first public content set to be presented on a fourth user system of a fourth user; receiving an indication of a combined user input related to the first user input content item from the fourth user system of the fourth user; in response to receiving the indication of the combined user input from the fourth user system, causing the combination function to be called on the fourth user system, the combination function accessing a third user input content item, and combining the first user input content item with the third user input content item to create a second combined user input content item, the third user input content item including at least one of image content or video content; and in response to determining that the second combined user input content item is shared to the first public content set by the fourth user, causing the second combined user input content item to be stored in the second public content set in association with the first user input content item, wherein the second public content set is navigable to view the first combined user input content item and the second combined user input content item.

[0443] In Example 3, the subject matter of Examples 1 to 2 includes: wherein, calling the combined collection presentation function on the third user system also includes causing a combined graphic element to be presented within the collection interface, the combined graphic element being selectable by a user to call the combined function on the third user system, the combined function accessing a fourth user input content item, and combining the first user input content item with the fourth user input content item to create a third combined user input content item.

[0444] In Example 4, the subject matter of Examples 1 to 3 includes wherein the second common content set includes only combined user input content items created using the first user input content items as base items.

[0445] In Example 5, the subject matter of Example 4 includes: wherein each combined user input content item in the second set of common content is shared to the first set of common content.

[0446] In Example 6, the subject matter of Examples 1 to 5 includes causing the first combined user input content item to be shared to the first public content set.

[0447] In Example 7, the subject matter of Examples 1 to 6 includes: causing a combined graphic element to be presented on the second user system in conjunction with the first user input content item, the combined graphic element being selectable by a user to invoke the combined function, wherein the combined user input associated with the first user input content item includes a user selection of the combined graphic element presented in conjunction with the first user input content item.

[0448] In Example 8, the subject matter of Examples 1 to 7 includes: causing a combination set presentation graphic element to be presented on the third user system in combination with the first user input content item, the combination set presentation graphic element being selectable by a user to invoke the combination set presentation function, wherein the combination set presentation user input related to the first user input content item is a user selection of the combination set presentation graphic element presented in combination with the first user input content item.

[0449] In Example 9, the subject matter of Examples 1 to 8 includes: wherein the first combined user input content item includes a property element that identifies at least one of the first user or the first user input content item.

[0450] In Example 10, the subject matter of Examples 1 to 9 includes allowing the third user to access the first combined user input content item on the third user system via the second public content set.

[0451] In Example 11, the subject matter of Example 10 includes: wherein, in response to the third user accessing the first combined user input content item on the third user system, the first combined user input content item is presented on the third user system in conjunction with an enhancement identifier, and the enhancement identifier identifies an enhancement applied to the image content or video content in the first combined user input content item.

[0452] In Example 12, the subject matter of Example 11 includes: wherein the first user input content item includes audio content in addition to at least one of the image content or video content.

[0453] In Example 13, the subject matter of Example 12 includes: wherein the first combined user input content item includes the audio content of the first user input content item, and the first combined user input content item is presented on the third user system in conjunction with an audio identifier identifying the audio content.

[0454] In Example 14, the subject matter of Examples 1 to 13 includes: wherein the first user input content item is presented on the second user system in conjunction with an enhancement identifier, the enhancement identifier identifying an enhancement applied to the image content or video content in the first user input content item.

[0455] In Example 15, the subject matter of Examples 1 to 14 includes: wherein, causing the first combined user input content item to be stored in association with the first user input content item in the second common content set includes accessing metadata of the first combined user input content item to identify it as a visual combined response to the first user input content item, and the second common content set includes multiple visual combined responses to the first user input content item.

[0456] In Example 16, the subject matter of Examples 1 to 15 includes: wherein the combined functionality includes a camera functionality that enables the second user to capture the second user input content item.

[0457] In Example 17, the subject matter of Examples 1 to 16 includes: wherein the method further includes: causing a digital image feed from at least one camera of the second user system to be presented in a camera interface; receiving user capture input from the second user via the camera interface; in response to receiving the user capture input, capturing digital image content as the second user input content item; and storing the second user input content item on the second user system.

[0458] In Example 18, the subject matter of Examples 1 to 17 includes: wherein the combining functionality includes a layout functionality that enables the second user to layout the first user input content item relative to the second user input content item within the first combined user input content item.

[0459] Example 19 is a computing device comprising: at least one processor; and at least one memory component storing instructions, wherein the instructions, when executed by the at least one processor, configure the system to perform operations that facilitate sharing of content between a first user, a second user, and a third user, the operations comprising: causing a first public content set to be presented on a second user system of the second user, the first public content set comprising a plurality of user-input content items; receiving an indication of a combined user input related to a first user-input content item from the second user system of the second user, the first user-input content item forming part of the first public content set and comprising at least one of image content or video content, the first user-input content item having been shared to the first public content set by the first user using the first user system; in response to receiving the indication of the combined user input from the second user system, causing a combining function to be called on the second user system, the The combination function accesses a second user input content item and combines the first user input content item with the second user input content item to create a first combined user input content item, wherein the second user input content item includes at least one of image content or video content; in response to determining that the first combined user input content item is shared to the first public content collection by the second user, the first combined user input content item is stored in the second public content collection in association with the first user input content item; an indication of a combined collection presentation user input related to the first user input content item is received from a third user system of the third user; and in response to receiving the indication of the combined collection presentation user input, a combined collection presentation function is called on the third user system, the combined collection presentation function being used to: cause a collection interface to be presented on the third user system; and enable the third user to navigate the second public content collection within the collection interface.

[0460] Example 20 is a non-transitory computer-readable storage medium, the computer-readable storage medium comprising instructions, which when executed by a computer system causes the computer system to perform operations, the operations comprising: at least one processor; and at least one memory component storing instructions, the instructions, when executed by the at least one processor, configure the system to perform operations that facilitate sharing of content between a first user, a second user, and a third user, the operations comprising: causing a first public content set to be presented on a second user system of the second user, the first public content set comprising a plurality of user-input content items; receiving from the second user system of the second user an indication of a combined user input related to a first user-input content item, the first user-input content item forming part of the first public content set and comprising at least one of image content or video content, the first user-input content item having been shared to the first public content set by the first user using the first user system; in response to receiving from the second user system an indication of a combined user input related to a first user-input content item In response to determining that the first combined user input content item is shared to the first public content collection by the second user, the first combined user input content item is stored in the second public content collection in association with the first user input content item; an indication of a combined collection presentation user input related to the first user input content item is received from the third user system of the third user; and in response to receiving the indication of the combined collection presentation user input, a combined collection presentation function is called on the third user system, the combined collection presentation function is used to: cause a collection interface to be presented on the third user system; and enable the third user to navigate the second public content collection within the collection interface.

[0461] Example 21 is at least one machine-readable medium comprising instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations for implementing any of Examples 1-20.

[0462] Example 22 is a device comprising a device for implementing any one of Examples 1 to 20.

[0463] Example 23 is a system implementing any one of Examples 1 to 20.

[0464] Example 24 is a method of implementing any one of Examples 1 to 20.

[0465] in conclusion

[0466] Although various aspects have been described with reference to specific examples, it will be apparent that various modifications and changes may be made to these examples without departing from the broader scope of the present disclosure. Therefore, the description and the accompanying drawings should be considered in an illustrative sense, rather than a restrictive sense. The accompanying drawings forming a part of the present invention show specific examples in which the subject matter can be practiced by way of illustration and not limitation. The examples shown are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other examples may be utilized and derived therefrom so that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, this specific embodiment should not be understood in a restrictive sense, and the scope of the various examples is limited only by the appended claims and the full scope of the equivalents given by these claims.

[0467] Although the examples, including the exemplary flow charts, may depict a particular order of operations, the order may be changed without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different order without materially affecting the functionality of the depicted components, systems, or processes. In other examples, different components of an example device or system implementing a method or process may perform functions substantially simultaneously or in a particular order.

[0468] The various features, steps, operations and processes described herein can be used independently of each other, or can be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. In addition, certain methods or processing blocks or operations can be omitted in some implementations.

[0469] As used in this disclosure, phrases of the form "at least one of one A, one B, or one C", "at least one of A, B, or C", "at least one of A, B, and C", etc. should be interpreted as selecting at least one from the group including "A, B, and C". In this disclosure, unless otherwise explicitly stated in conjunction with a specific example, this wording does not mean "at least one of A, at least one of B, and at least one of C". As used in this disclosure, the example "at least one of one A, one B, or one C" will cover any of the following selections: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, and {A, B, C}.

[0470] Unless the context clearly requires otherwise, in this disclosure, the words "include", "comprises", etc. should be understood to be inclusive, not exclusive or exhaustive, i.e., "including but not limited to". As used herein, the terms "connect", "couple" or any variant thereof refer to any direct or indirect connection or coupling between two or more elements; the coupling or connection between elements can be physical, logical, or a combination thereof. In addition, the words "herein", "above", "below", and words of similar meaning used in this application refer to the application as a whole, not to any particular part of this application. Where the context permits, words used in the singular or plural may also include the plural or singular, respectively. With respect to a list of two or more items, the word "or" covers all the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Similarly, the term "and / or" also covers all the following interpretations of the word when referring to a list of two or more items: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Similarly, the term "and / or" also covers all the following interpretations of the word when referring to a list of two or more items: any one of the items in the list, all of the items in the list, and any combination of the items in the list.

[0471] The terms "combined" and "combination", for example when used to refer to combined content items and / or combined functionality, refer to content items or content data that have been or can be combined in any suitable manner. Examples may include a combination of two or more images, a combination of two or more video clips, etc. Examples may also include combinations of different content types, for example, a combination of a video clip with a still image, or a combination of a video clip with an audio item, or a combination of the above. Such combinations may be arranged in different ways, for example, the items may be displayed or played simultaneously, sequentially, or in an overlay, and the items may be arranged using different layout options (e.g., overlay, split screen, etc.), examples of which are described herein.

[0472] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media to facilitate communication of such instructions. Instructions may be sent or received over a network via a network interface device using a transmission medium.

[0473] "Client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a PDA, a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communication device that a user may use to access a network. In the case where a user uses such a device, a "client device" may be referred to as a "user device."

[0474] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a WLAN, a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a Plain Old Telephone Service (POTS) network, a cellular telephone network, a wireless network, The coupling may be a network, another type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate for GSM evolution (EDGE) technology, including the third generation partnership project (3GPP) of 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.

[0475] "Component" refers to a device, physical entity, or logic with the following boundaries: the boundary is defined by a function or subroutine call, a branch point, an API, or other technology provided for dividing or modularizing a specific processing or control function. A component can be combined with other components via its interface to perform machine processing. A component can be a packaged functional hardware unit designed for use with other components, and can be part of a program that generally performs a specific function in a related function. A component can constitute a software component (e.g., a code embodied on a machine-readable medium) or a hardware component. A "hardware component" is a tangible unit that can perform certain operations and can be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., an independent computer system, a client computer system, or a server computer system) or one or more hardware components (e.g., a processor or a processor group) of a computer system can be configured by software (e.g., an application or an application part) to be a hardware component that operates to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic that is permanently configured to perform certain operations. The hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an ASIC. The hardware component may also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine), which is uniquely customized to perform the function of the configuration and is no longer a general-purpose processor. It will be recognized that it can be determined for cost and time considerations whether to mechanically implement the hardware component in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., configured by software) circuit system to implement the hardware component. Therefore, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an example in which a hardware component is temporarily configured (e.g., programmed), it is not necessary to configure or instantiate each hardware component in the hardware component at any one time. For example, where a hardware component includes a general purpose processor that is configured by software to be a special purpose processor, the general purpose processor may be configured as different special purpose processors (e.g., including different hardware components) at different times. The software configures one or more specific processors accordingly, such as to constitute a specific hardware component at one time and to constitute different hardware components at different times. Hardware components may provide information to other hardware components and receive information from other hardware components.Therefore, the hardware components described can be considered to be coupled in communication. In the case of multiple hardware components simultaneously, communication can be achieved by signal transmission between or among two or more hardware components in the hardware components (for example, by appropriate circuits and buses). In the example where multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessible to multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device coupled to it in communication. Then, other hardware components can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (for example, a collection of information). The various operations of the example methods described herein can be performed at least in part by a temporary configuration (for example, by software) or one or more processors that are permanently configured to perform related operations. Whether it is a temporary configuration or a permanent configuration, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operation of the method can be performed by one or more processors or the components implemented by the processor. In addition, one or more processors can also operate to support the execution of related operations in a "cloud computing" environment or operate as "software as a service" (SaaS). For example, at least some of the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (for example, the Internet) and via one or more appropriate interfaces (for example, API). The execution of certain operations in the operation can be distributed between processors, not only resident in a single machine, but also deployed across multiple machines. In some examples, a processor or a component implemented by a processor can be located in a single geographic location (for example, in a home environment, an office environment, or a server cluster). In other examples, a processor or a component implemented by a processor can be distributed across multiple geographic locations.

[0476] "Computer-readable storage media" refers to both machine storage media and transmission media. Therefore, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium" and "device-readable medium" mean the same thing and can be used interchangeably in this disclosure.

[0477] “Machine storage media” refers to a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage media,” “device storage media,” and “computer storage media” mean the same thing and may be used interchangeably in this disclosure. The terms “machine storage media,” “computer storage media,” and “device storage media” explicitly exclude carrier waves, modulated data signals, and other such media, at least some of which are encompassed by the term “signal media.”

[0478] “Non-transitory computer-readable storage medium” refers to tangible media capable of storing, encoding, or carrying instructions to be executed by a machine.

[0479] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be deemed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0480] "User input content" refers to any form of content that a user can post, publish, share, send, receive or interact with, for example, using an online platform. For example, the user input content may include digital content in the form of one or more images, photos, video clips, audio clips, or a combination thereof. In some examples, the user input content item may be a UGC (user generated content) item. For example, enhancements (e.g., lenses, filters, media overlays, text overlays, or other enhancements) may be applied to the user input content. For example, in the case where the content item is captured by the user using a digital camera device of a user device, or in the case where the content item is created by the user by applying enhancements to another content item, the user input content item may be generated by the user. The user input content item may be selected by the user, for example, the content item is selected from a media library or file storage, or, for example, originates from a third party or a third party device. The user input content may also include content generated in part or in whole by a computer, for example, content created by generative artificial intelligence. For example, a content item may be content data, such as a picture, video, or audio data (or a combination thereof).

Claims

1. A method for facilitating sharing of content among a first user, a second user, and a third user, the method comprising: causing a first public content set to be presented on a second user system of the second user, the first public content set comprising a plurality of user-input content items; receiving, from the second user system of the second user, an indication of a combined user input relating to a first user input content item, the first user input content item forming part of the first public content collection and comprising at least one of image content or video content, the first user input content item having been shared to the first public content collection by the first user using the first user system; in response to receiving an indication of the combined user input from the second user system, causing a combining function to be invoked on the second user system, the combining function to access a second user input content item and combine the first user input content item with the second user input content item to create a first combined user input content item, the second user input content item comprising at least one of image content or video content; In response to determining that the first combined user input content item is shared by the second user to the first public content set, causing the first combined user input content item to be stored in a second public content set in association with the first user input content item; receiving, from a third user system of the third user, an indication that the combined set presents user input related to the first user input content item; as well as In response to receiving the indication of the combined set presentation user input, a combined set presentation function is invoked on the third user system, the combined set presentation function being used to: causing the collective interface to be presented on the third user system; and The third user is enabled to navigate the second collection of public content within the collection interface.

2. The method according to claim 1, further comprising: causing the first set of public content to be presented on a fourth user system of a fourth user; receiving, from said fourth user system of said fourth user, an indication of a combined user input related to said first user input content item; in response to receiving an indication of the combined user input from the fourth user system, causing the combining function to be invoked on the fourth user system, the combining function being operable to access a third user input content item and combine the first user input content item with the third user input content item to create a second combined user input content item, the third user input content item comprising at least one of image content or video content; as well as In response to determining that the second combined user input content item is shared by the fourth user to the first public content set, such that the second combined user input content item is stored in the second public content set in association with the first user input content item, Wherein, the second common content set is navigable to view the first combined user-input content item and the second combined user-input content item.

3. The method according to claim 1, wherein: Calling the combined collection presentation function on the third user system also includes causing a combined graphic element to be presented within the collection interface, the combined graphic element being selectable by a user to call the combined function on the third user system, the combined function being used to access a fourth user input content item, and combining the first user input content item with the fourth user input content item to create a third combined user input content item.

4. The method according to claim 1, wherein: The second common content set includes only combined user input content items created using the first user input content items as base items.

5. The method according to claim 4, wherein: Each combined user input content item in the second common content set is shared to the first common content set. 6 . The method of claim 1 , further comprising causing the first combined user input content item to be shared to the first public content collection.

7. The method of claim 1 , further comprising causing a combined graphical element to be presented on the second user system in conjunction with the first user input content item, the combined graphical element being selectable by a user to invoke the combined functionality, wherein The combined user input associated with the first user input content item comprises a user selection of the combined graphical element presented in conjunction with the first user input content item.

8. The method according to claim 1 further comprises causing a combined set presentation graphic element to be presented on the third user system in conjunction with the first user input content item, the combined set presentation graphic element being selectable by a user to invoke the combined set presentation function, wherein The combined set presentation user input associated with the first user input content item is a user selection of the combined set presentation graphical element presented in conjunction with the first user input content item.

9. The method according to claim 1, wherein: The first combined user input content item includes a property element that identifies at least one of the first user or the first user input content item.

10. The method of claim 1, further comprising allowing the third user to access the first combined user input content item on the third user system via the second public content set.

11. The method according to claim 10, wherein: In response to the third user accessing the first combined user input content item on the third user system, the first combined user input content item is presented on the third user system in conjunction with an enhancement identifier, wherein the enhancement identifier identifies an enhancement applied to the image content or video content in the first combined user input content item.

12. The method according to claim 11, wherein: The first user input content item includes audio content in addition to at least one of the image content or the video content.

13. The method according to claim 12, wherein: The first combined user input content item comprises the audio content of the first user input content item, the first combined user input content item being presented on the third user system in conjunction with an audio identifier identifying the audio content.

14. The method according to claim 1, wherein: The first user input content item is presented on the second user system in conjunction with an enhancement identifier identifying an enhancement applied to the image content or video content in the first user input content item.

15. The method according to claim 1, wherein: Causing the first combined user input content item to be stored in association with the first user input content item in the second common content set includes accessing metadata of the first combined user input content item to identify it as a visual combined response to the first user input content item, and the second common content set includes multiple visual combined responses to the first user input content item.

16. The method according to claim 1, wherein: The combined functionality includes a camera functionality enabling the second user to capture the second user input content item.

17. The method according to claim 1, wherein: The method further comprises: causing a digital image feed from at least one camera of the second user system to be presented in a camera interface; receiving user capture input from the second user via the camera interface; In response to receiving the user capture input, capturing digital image content as the second user input content item; and The second user input content item is stored on the second user system.

18. The method according to claim 1, wherein: The combining functionality comprises a layout functionality enabling the second user to layout the first user input content item relative to the second user input content item within the first combined user input content item.

19. A system comprising: at least one processor; as well as at least one memory component storing instructions that, when executed by the at least one processor, configure the system to perform operations that facilitate sharing of content among a first user, a second user, and a third user, the operations comprising: causing a first public content set to be presented on a second user system of the second user, the first public content set comprising a plurality of user-input content items; receiving, from the second user system of the second user, an indication of a combined user input relating to a first user input content item, the first user input content item forming part of the first public content collection and comprising at least one of image content or video content, the first user input content item having been shared to the first public content collection by the first user using the first user system; in response to receiving an indication of the combined user input from the second user system, causing a combining function to be invoked on the second user system, the combining function to access a second user input content item and combine the first user input content item with the second user input content item to create a first combined user input content item, the second user input content item comprising at least one of image content or video content; In response to determining that the first combined user input content item is shared by the second user to the first public content set, causing the first combined user input content item to be stored in a second public content set in association with the first user input content item; receiving, from a third user system of the third user, an indication that the combined set presents user input related to the first user input content item; and In response to receiving the indication of the combined set presentation user input, a combined set presentation function is invoked on the third user system, the combined set presentation function being used to: causing the collective interface to be presented on the third user system; and The third user is enabled to navigate the second collection of public content within the collection interface.

20. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform operations that facilitate sharing of content between a first user, a second user, and a third user, the operations comprising: causing a first public content set to be presented on a second user system of the second user, the first public content set comprising a plurality of user-input content items; receiving, from the second user system of the second user, an indication of a combined user input relating to a first user input content item, the first user input content item forming part of the first public content collection and comprising at least one of image content or video content, the first user input content item having been shared to the first public content collection by the first user using the first user system; in response to receiving an indication of the combined user input from the second user system, causing a combining function to be invoked on the second user system, the combining function to access a second user input content item and combine the first user input content item with the second user input content item to create a first combined user input content item, the second user input content item comprising at least one of image content or video content; In response to determining that the first combined user input content item is shared by the second user to the first public content set, causing the first combined user input content item to be stored in a second public content set in association with the first user input content item; receiving, from a third user system of the third user, an indication that the combined set presents user input related to the first user input content item; as well as In response to receiving the indication of the combined set presentation user input, a combined set presentation function is invoked on the third user system, the combined set presentation function being used to: causing the collective interface to be presented on the third user system; and The third user is enabled to navigate the second collection of public content within the collection interface.

Images