Network-based real-time special effects

Abstract

A system and method for enabling augmented reality effects in a web browser without requiring additional software installation are disclosed. A web server provides a range of selectable effects to a website. When an effect is selected, the website loads a page specific to that effect, including a live preview of the effect applied to a video feed from the user's webcam. This allows the user to see themselves with the effect applied in real time. The website requests access to the webcam and microphone via the browser's built-in permissions system. Captured photos and videos with the effect applied can be saved locally or shared via native operating system tools. This system provides an engaging augmented reality experience that is directly accessible via a standard web browser without the need for a dedicated app.

Description

[0001] Priority requirements

[0002] This application claims the benefit of priority to U.S. Application No. 18 / 631,354, filed April 10, 2024, which claims the benefit and priority to U.S. Provisional Patent Application No. 63 / 584,284, filed September 21, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Embodiments of the present invention relate to techniques for providing special effects—specifically, augmented reality effects—in a web browser environment, wherein the augmented reality effects include real-time manipulation of images captured by an image sensor of a user's computing device according to special effects selected by the end user, presented within a webpage, without the need to install software applications. Background Technology

[0004] Client software applications (including mobile applications designed for smartphones and tablets, and desktop applications designed for personal computers) enable users to access online services and generate engaging content through the various input and output capabilities of their devices. For example, these client applications are typically specifically programmed for the operating system of the device on which they are running and utilize application programming interfaces (APIs) and other technologies to gain access to hardware resources such as built-in cameras, microphones, touchscreens, etc. This allows client applications to create images, videos, and other media enhanced with special effects and augmented reality experiences.

[0005] However, the web and desktop versions of these same online services accessed via web browsers rather than dedicated applications typically have limited functionality compared to their corresponding client application versions. When people first discover a service through a web browser, they may not experience all the features available in the client application. This can make it difficult for online services to attract and retain new users through web browsers.

[0006] Brief description of several views in the attached figures

[0007] In accompanying drawings that are not necessarily drawn to scale, the same numbers may describe similar parts in different views. For ease of identification of any particular element or behavior being discussed, one or more highest-digit numbers in the drawing reference numerals indicate the drawing number in which that element was first introduced. Some non-limiting examples are shown in the accompanying drawings:

[0008] Figure 1 It is a graphical representation of a networked environment in which innovative themes can be deployed, based on some examples.

[0009] Figure 2 It is a graphical representation of a messaging system with both client-side and server-side functionality, based on some examples.

[0010] Figure 3 It is a graphical representation of the data structures maintained in the database based on some examples.

[0011] Figure 4 It is a graphical representation based on some example messages.

[0012] Figure 5 This is a flowchart illustrating how to operate a method for rendering a live view of special effects via a webpage, based on some examples.

[0013] Figure 6 These are user interface diagrams based on some examples.

[0014] Figure 7 These are user interface diagrams based on some examples.

[0015] Figure 8 These are user interface diagrams based on some examples.

[0016] Figure 9 These are user interface diagrams based on some examples.

[0017] Figure 10 These are user interface diagrams based on some examples.

[0018] Figure 11 These are user interface diagrams based on some examples.

[0019] Figure 12 These are user interface diagrams based on some examples.

[0020] Figure 13 It is a graphical representation of a machine in the form of a computer system based on some examples, within which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed herein.

[0021] Figure 14 It is a block diagram illustrating an example of a software architecture that can be implemented therein. Detailed Implementation

[0022] This document describes methods, systems, and computer program products for enabling users to experience special effects (including augmented reality effects) directly within a web browser without installing any additional software. Web pages provide a selection of effects that can be applied in real-time to live video captured by a device's camera and displayed back to the user via the browser. Users do not need to download and install separate applications or register as members of the service to access and try the available effects. Captured photos and videos containing the effects can then be saved and shared using the capabilities of the local operating system, providing an attractive augmented reality experience through standard web technologies. In the following description, numerous specific details are set forth for illustrative purposes to provide a thorough understanding of various aspects of different embodiments of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without all of these specific details.

[0023] Many popular online services and social media platforms offer special effects (such as augmented reality experiences) through dedicated mobile applications (apps). These apps are installed on the end user's smartphone or tablet and utilize the client device's camera, microphone, and other hardware capabilities. By accessing native application programming interfaces (APIs) and frameworks, apps can apply graphics, animations, and other enhancements as filters and overlays to photos and videos captured by the end user. For example, mobile apps can offer various selfie effects that use the front-facing camera to change or animate parts of the user's face in real time.

[0024] While these mobile apps offer attractive experiences, a major drawback is that users must first find, download, and install the app before gaining access to effects. This operational barrier means some users will never discover the app or will abandon the download process before installation. Even for installed apps, users may not spend time exploring features like effects if they are not easily discovered. Furthermore, mobile apps typically require creating an account and completing a registration process before unlocking all features. In short, these barriers limit an app's ability to attract and delight new users. Web experiences, accessible directly from a browser, offer a much lower operational barrier for users exploring new apps and features.

[0025] Furthermore, the requirement to install the app and register as an end-user or member presents a challenge when existing users want to share specific effects with friends or contacts who are not registered end-users of the online service. For example, if a registered end-user or member discovers a fun or interesting effect in the mobile app and wants to share it with friends who are not members of the online service (e.g., registered end-users), there is no easy way for non-members to try the effect without first downloading the app and creating an account. This severely limits members' ability to virally share their knowledge of effects.

[0026] The embodiments of the present invention solve the aforementioned technical problems by allowing non-members of online services (e.g., unregistered end users) to experience special effects (including augmented reality effects and experiences) directly in a web browser without needing to install software, create an account, or register. A web server hosts a website that provides a catalog or library of special effects that any visitor can preview. When a user selects an effect, the website loads a page specific to that effect. This page includes a live preview window that displays a video feed from the user's webcam and the selected effect applied in real time. Therefore, users can view and test various special effects.

[0027] One advantage over traditional online services is that users can immediately try and experiment with effects without any operational barriers. There's no need to search for, download, or install a mobile or desktop app, nor is it necessary to become a registered end-user. Each effect works directly within a web browser, utilizing standardized web technologies such as HTML5, JavaScript, and WebRTC. The web server provides all the necessary code to access the webcam stream and render graphics / animations as overlays on live video.

[0028] Consistent with some implementations, in order to apply special effects, the website uses the browser's licensing system to request access to the user's camera and microphone. Once authorized, all video and audio processing is performed locally within the browser. At least in some implementations, the media stream is not sent to the server. This maintains user privacy while providing attractive special effects. After previewing a specific effect, the user can capture still images (e.g., photos) and record video clips containing the effects via the browser API.

[0029] The captured photos and videos can then be saved locally or shared via the native operating system (OS) sharing tools. For example, on a smartphone, a user can save them to the camera's album or share them via a messaging app. Alternatively, if the user has a registered account, photos / videos captured via a web browser can be saved and shared within the web-based experience itself. Furthermore, in some cases, if the user is a member or registered end-user, after capturing media content items (e.g., images or video clips), the user can authenticate via an online service to share the captured content items via the app experience.

[0030] By removing barriers associated with traditional in-app experiences, such as complex installations and mandatory end-user registration, the implementation makes special effects more easily discovered and shared by everyday users. Web technologies allow users to enjoy special effects directly in their browsers without being locked into a specific app ecosystem. Further aspects, features, and advantages arising from the disclosed methods and systems will become more apparent after reading the description of the following figures.

[0031] Networked computing environment

[0032] Figure 1 This is a block diagram illustrating an example interactive system 100 for facilitating interactions on a network, such as exchanging text messages, making text-to-audio and video calls, or playing games. Interactive system 100 includes multiple user systems 102, each hosting multiple applications, including interactive client 104 and other applications 106. Each interactive client 104 is communicatively coupled to (e.g., hosted on corresponding other user systems 102) other instances of interactive client 104, interactive server system 110, and third-party server 112 via one or more communication networks, including network 108 (e.g., the Internet). Interactive client 104 can also communicate with locally hosted applications 106 using application programming interfaces (APIs).

[0033] Each user system 102 may include a native application (e.g., an interactive client) tailored to the device's operating system (e.g., a mobile app on a smartphone or tablet). This mobile app can provide the user system with a mechanism to communicate with the interactive server hosted on the interactive server system 110. For example, the interactive client 104 and associated application 106 can process captured images and stream them via API server 122 to the interactive server 124 for processing augmented reality effects. However, consistent with some implementations, each user system 104 can use a regular web browser application to access the various features provided by the interactive server 124 without needing to install any native app on the user system 102. Therefore, the functionality of the interactive server (e.g., augmented reality effects and services) can be directly accessed via standard web browser application 106 using public network protocols such as HTTP, web RTC, and JavaScript. This allows temporary users to quickly try out interesting effects without first locating, downloading, installing, creating an account, and learning how to use a new native app. The web browser application 104 can access device components of the user system (e.g., camera devices) via APIs and process effects directly within the browser application, allowing users to fully enjoy participating in augmented reality experiences through open web standards.

[0034] Interactive client 104 interacts with other interactive clients 104 and with interactive server system 110 via network 108. The data exchanged between interactive clients 104 (e.g., interactive 120) and between interactive client 104 and interactive server system 110 includes functions (e.g., commands to invoke functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0035] Interactive server system 110 provides server-side functionality to interactive client 104 via network 108. While some functions of interactive system 100 are described herein as being performed by interactive client 104 or interactive server system 110, whether a function resides within interactive client 104 or interactive server system 110 can be a design choice. For example, it is technically preferred that specific technologies and functions are initially deployed within interactive server system 110, but that technology and functions are later migrated to interactive client 104 of user system 102, which has sufficient processing power.

[0036] The interactive server system 110 supports various services and operations provided to the interactive client 104. Such operations include sending data to and receiving data from the interactive client 104, and processing data generated by the interactive client 104. This data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, entity relationship information, and on-site event information. Data exchange within the interactive system 100 is initiated and controlled via functions available through the user interface (UI) of the interactive client 104.

[0037] Now, specifically, the interactive server system 110 has an application programming interface (API) server 122 coupled to the interactive server 124 and providing it with a programming interface, making the functionality of the interactive server 124 accessible to the interactive client 104, other applications 106, and third-party servers 112. The interactive server 124 is communicatively coupled to a database server 126, facilitating access to a database 128 that stores data associated with the interactions processed by the interactive server 124. Similarly, a web server 130 is coupled to the interactive server 124 and provides a web-based interface to it. For this purpose, the web server 130 handles incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0038] API server 122 receives and sends interactive data (e.g., command and message payloads) between interactive server 124 and user system 102 (as well as, for example, interactive client 104 and other applications 106) and third-party server 112. Specifically, application programming interface (API) server 122 provides a set of interfaces (e.g., routines and protocols) that interactive client 104 and other applications 106 can invoke or query to activate the functionality of interactive server 124. API server 122 discloses various functions supported by interaction server 124, including: account registration; login function; sending interactive data from one interactive client 104 to another interactive client 104 via interaction server 124; transmission of media files (e.g., images or videos) from interactive client 104 to interaction server 124; setting up collections of media data (e.g., stories); retrieval of the user's friend list in user system 102; retrieval of messages and content; adding and deleting entities (e.g., friends) in entity relationship graphs (e.g., entity graph 310); locating friends within entity relationship graphs; and opening application events (e.g., related to interactive client 104).

[0039] Interactive server 124 hosts multiple systems and subsystems, as detailed below. Figure 2 Describe it.

[0040] System Architecture

[0041] Figure 2 This is a block diagram illustrating further details of an interactive system 100 according to some examples. Specifically, the interactive system 100 is shown as including an interactive client 104 and an interactive server 124. The interactive system 100 includes multiple subsystems supported on the client side by the interactive client 104 and on the server side by the interactive server 124. In some examples, these subsystems are implemented as microservices. A microservice subsystem (e.g., a microservice application) may have components that enable it to operate independently and communicate with other services. Example components of a microservice subsystem may include:

[0042] Functional logic: Functional logic implements the functions of the microservice subsystem and represents the specific capabilities or functions provided by the microservice.

[0043] API Interface: Microservices can communicate with each other using well-defined APIs or interfaces, employing lightweight protocols such as REST or messaging. The API interface defines the inputs and outputs of a microservice subsystem and how it interacts with other microservice subsystems within the interactive system 100.

[0044] Data storage: The microservice subsystem can be responsible for its own data storage, which can be in the form of a database, cache, or other storage mechanisms (e.g., using database server 126 and database 128). This allows the microservice subsystem to operate independently of other microservices in the interactive system 100.

[0045] Service discovery: Microservice subsystems can find and communicate with other microservice subsystems in the interacting system 100. The service discovery mechanism enables microservice subsystems to locate and communicate with other microservice subsystems in a scalable and efficient manner.

[0046] Monitoring and logging: Microservice subsystems may need to be monitored and logged to ensure availability and performance. Monitoring and logging mechanisms enable the tracking of the health and performance of microservice subsystems.

[0047] In some examples, the interactive system 100 may employ a monolithic architecture, a service-oriented architecture (SOA), a function-as-a-service (FaaS) architecture, or a modular architecture. Example subsystems are discussed below.

[0048] The image processing system 202 provides various functions that enable users to capture and enhance (e.g., annotate or otherwise modify or edit) media content associated with a message.

[0049] The camera device system 204 includes (e.g., in a camera device application) control software that interacts with and controls the hardware camera device hardware of the user system 102 (e.g., directly or via operating system controls) to modify and enhance real-time images captured and displayed via the interactive client 104.

[0050] Enhancement system 206 provides functionality related to the generation and distribution of special effects, including enhancements (e.g., media overlays) for images captured in real-time by the camera device of user system 102 or retrieved from the memory of user system 102. For example, enhancement system 206 can operatively select, present, and display media overlays (e.g., image filters or effects) for interactive client 104 to enhance real-time images received via camera device system 204 or stored images retrieved from memory 502 of user system 102. These effects or enhancements are selected and presented to the user of interactive client 104 by enhancement system 206 based on multiple inputs and data, such as:

[0051] The geographical location of user system 102; and

[0052] User entity relationship information of users in user system 102.

[0053] Enhancements may include audio and visual content and visual effects. Examples of audio and video content include images, text, logos, animations, and sound effects. Examples of visual effects include color overlays. Audio and video content or visual effects may be applied to media content items (e.g., photos or videos) at user system 102 to be transmitted in messages or applied to video content, such as a video content stream or feed transmitted from interactive client 104. Therefore, image processing system 202 can interact with and support various subsystems of communication system 208, such as messaging system 210 and video communication system 212.

[0054] Media overlays may include text or image data that can be superimposed on photographs taken by user system 102 or video streams generated by user system 102. In some examples, media overlays may be location overlays (e.g., Venice Beach), names of on-site events, or names of businesses (e.g., beach cafes). In other examples, image processing system 202 uses the geographic location of user system 102 to identify the media overlay, which includes the name of a business at the geographic location of user system 102. Media overlays may include other tags associated with businesses. Media overlays may be stored in database 128 and accessed through database server 126.

[0055] Image processing system 202 provides a user-based publishing platform that allows users to select geographic locations on a map and upload content associated with those locations. Users can also specify which media overlays should be provided to other users. Image processing system 202 generates a media overlay that includes the uploaded content and associates it with the selected geographic location.

[0056] The augmented creation system 214 supports augmented reality developer platforms and includes applications that enable content creators (such as artists and developers) to create and publish interactive clients 104, such as augmented reality experiences. The augmented creation system 214 provides content creators with a library of built-in features and tools, including, for example, custom shaders, tracking technologies, and templates.

[0057] In some examples, enhancement creation system 214 provides a merchant-based publishing platform that allows merchants to select specific enhancements associated with a geographic location through a bidding process. For example, enhancement creation system 214 associates the media overlay of the highest bidder with the corresponding geographic location for a predefined amount of time.

[0058] Communication system 208 is responsible for enabling and processing various forms of communication and interaction within interactive system 100, and includes messaging system 210, audio communication system 216, and video communication system 212. Messaging system 210 is responsible for enabling temporary or time-limited access to content by interactive client 104. Messaging system 210 incorporates multiple timers (e.g., within a short-lived timer system) that selectively enable access (e.g., for presentation and display) of messages and associated content via interactive client 104 based on duration and display parameters associated with a message or set of messages (e.g., a story). Audio communication system 216 enables and supports audio communication (e.g., real-time audio chat) between multiple interactive clients 104. Similarly, video communication system 212 enables and supports video communication (e.g., real-time video chat) between multiple interactive clients 104.

[0059] User management system 218 is operationally responsible for managing user data and profiles, and maintaining entity information about users and relationships between users in interactive system 100 (e.g., stored in entity table 308, entity diagram 310, and profile data 302).

[0060] The collection management system 220 is operationally responsible for managing collections or sets of media (e.g., collections of text, image, video, and audio data). Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into “event libraries” or “event stories.” Such collections can be made available for a specified time period (e.g., the duration of the event to which the content relates). For example, content related to a concert can be made available as a “story” for the duration of the concert. The collection management system 220 can also be responsible for publishing icons to the user interface of the interactive client 104, which provide notifications for specific collections. The collection management system 220 includes curation functionality that allows collection managers to manage and curate specific content collections. For example, the curation interface enables event organizers to curate collections of content related to a specific event (e.g., removing inappropriate content or redundant messages). Furthermore, the collection management system 220 employs machine vision (or image recognition technology) and content rules to automatically curate content collections. In some examples, users may be compensated for including user-generated content in the collection. In such a situation, the collection management system 220 operates to automatically pay such users for access to its content.

[0061] External resource system 226 provides interactive client 104 with an interface to communicate with remote servers (e.g., third-party server 112) to launch or access external resources (i.e., applications or applets). Each third-party server 112 hosts applications or smaller versions of applications (e.g., games, utilities, payment, or ride-sharing apps) based on markup languages ​​(e.g., HTML5). Interactive client 104 can launch web-based resources (e.g., applications) by accessing HTML5 files from the third-party server 112 associated with the web-based resource. The application hosted by third-party server 112 is programmed in JavaScript using a software development kit (SDK) provided by interactive server 124. The SDK includes application programming interfaces (APIs) with functions that can be invoked or called by the web-based application. Interactive server 124 hosts a JavaScript library that provides access to a given external resource for specific user data of interactive client 104. HTML5 is an example of a technology used for programming games, but applications and resources programmed using other technologies can be used.

[0062] To integrate the functionality of the SDK into the web-based resource, the SDK is downloaded by third-party server 112 from interaction server 124, or otherwise received by third-party server 112. Once downloaded or received, the SDK is included as part of the application code of the web-based external resource. The code of the web-based resource can then call or invoke certain functions of the SDK to integrate the features of interaction client 104 into the web-based resource.

[0063] The SDK stored on the interactive server system 110 effectively provides a bridge between external resources (e.g., application 106 or applet) and the interactive client 104. This gives users a seamless experience of communicating with other users on the interactive client 104 while retaining the look and feel of the interactive client 104. To bridge communication between external resources and the interactive client 104, the SDK facilitates communication between a third-party server 112 and the interactive client 104. A bridge script running on the user system 102 establishes two unidirectional communication channels between the external resources and the interactive client 104. Messages are sent asynchronously between the external resources and the interactive client 104 via these communication channels. Each SDK function is invoked as a message and a callback. Each SDK function is implemented by constructing a unique callback identifier and sending a message with that callback identifier.

[0064] By using the SDK, not all information from the interactive client 104 is shared with the third-party server 112. The SDK limits what information is shared based on the needs of the external resource. Each third-party server 112 provides the interactive server 124 with an HTML5 file corresponding to the web-based external resource. The interactive server 124 can add a visual representation (such as box art or other graphics) of the web-based external resource to the interactive client 104. Once the user selects the visual representation or instructs the interactive client 104 to access the features of the web-based external resource through the interactive client 104's GUI, the interactive client 104 obtains the HTML5 file and instantiates the resource to access the features of the web-based external resource.

[0065] Interactive client 104 presents a graphical user interface (GUI) for an external resource (e.g., a login page or title screen). During, before, or after presenting the login page or title screen, interactive client 104 determines whether the initiated external resource has previously been authorized to access the user data of interactive client 104. In response to determining that the initiated external resource has previously been authorized to access the user data of interactive client 104, interactive client 104 presents another GUI for the external resource, including its functionality and characteristics. In response to determining that the initiated external resource has not previously been authorized to access the user data of interactive client 104, after displaying the login page or title screen of the external resource for a threshold time period (e.g., 3 seconds), interactive client 104 slides up a menu for authorizing the external resource to access user data (e.g., animating the menu to appear from the bottom of the screen to the middle or other part of the screen). The menu identifies the type of user data that the external resource will be authorized to use. In response to receiving a user's selection of the accept option, interactive client 104 adds the external resource to the list of authorized external resources and allows the external resource to access user data from interactive client 104. The interactive client 104 authorizes external resources to access user data under the OAuth2 framework.

[0066] Interactive client 104 controls the type of user data shared with external resources based on the type of authorized external resource. For example, access to a first type of user data (e.g., two-dimensional avatars of users with or without different avatar characteristics) is provided to external resources including full-scale applications (e.g., application 106). As another example, access to a second type of user data (e.g., payment information, two-dimensional avatars of users, three-dimensional avatars of users, and avatars with various avatar characteristics) is provided to external resources including smaller versions of applications (e.g., web-based versions of applications). Avatar characteristics include different ways of customizing the appearance and feel of an avatar, such as different poses, facial features, clothing, etc.

[0067] The advertising system 228 is operationally designed to enable third parties to purchase advertisements for presentation to end users via interactive client 104, and also handles the delivery and presentation of these advertisements.

[0068] Artificial intelligence and machine learning system 230 provides various services to different subsystems within interaction system 100. For example, AI and machine learning system 230 operates in conjunction with image processing system 202 and camera device system 204 to analyze images and extract information such as objects, text, or faces. This information can then be used by image processing system 202 to enhance, filter, or manipulate the image. Enhancement system 206 can use AI and machine learning system 230 to generate enhanced content and augmented reality experiences, such as adding virtual objects or animations to real-world images. Communication system 208 and messaging system 210 can use AI and machine learning system 230 to analyze communication patterns and provide insights into how users interact with each other, and provide intelligent message classification and tagging, such as classifying messages based on sentiment or topic. AI and machine learning system 230 can also provide chatbot functionality for messaging interactions 120 between user systems 102 and between user systems 102 and interaction server system 110. The artificial intelligence and machine learning system 230 can also work with the audio communication system 216 to provide speech recognition and natural language processing capabilities, allowing users to interact with the interactive system 100 using voice commands.

[0069] Data Architecture

[0070] Figure 3 This is a schematic diagram illustrating a data structure 300 that can be stored in a database 304 of an interactive server system 110, according to certain examples. Although the contents of the database 304 are shown to include multiple tables, it should be understood that the data can be stored in other types of data structures (e.g., as an object-oriented database).

[0071] Database 304 includes message data stored in message table 306. For any given message, this message data includes at least message sender data, message receiver (or recipient) data, and a payload. See below for reference. Figure 3 Additional details describe information that can be included in a message and is contained within message data stored in message table 306.

[0072] Entity table 308 stores entity data and (e.g., by reference) links to entity diagram 310 and profile data 302. Entities for which records are maintained within entity table 308 can include individuals, company entities, organizations, objects, locations, events, etc. Regardless of entity type, any entity whose data is stored by the interactive server system 110 can be an identifiable entity. Each entity is provided with a unique identifier and an entity type identifier (not shown).

[0073] Entity graph 310 stores information about relationships and associations between entities. As an example only, such relationships can be social, professional (e.g., working in a common company or organization), interest-based, or activity-based. Some relationships between entities can be one-way, such as an individual user subscribing to digital content from a business or publishing user (e.g., a newspaper or other digital media channel or brand). Other relationships can be two-way, such as the "friendship" relationship between individual users of interactive system 100.

[0074] Certain licenses and relationships can be attached to each relationship, and also to each direction of the relationship. For example, a two-way relationship (e.g., a friendship between individual users) can include authorization for the publication of digital content items between individual users, but certain restrictions or filters can be imposed on the publication of such digital content items (e.g., based on content characteristics, location data, or time of day data). Similarly, a subscription relationship between an individual user and a business user can impose varying degrees of restrictions on the publication of digital content from the business user to the individual user, and can significantly restrict or prevent the publication of digital content from the individual user to the business user. As an example of an entity, a specific user can record certain restrictions in a record for that entity within entity table 308 (e.g., via privacy settings). Such privacy settings can be applied to all types of relationships in the context of interaction system 100, or selectively applied to certain types of relationships.

[0075] Profile data 302 stores various types of profile data about a specific entity. Profile data 302 can be selectively used and presented to other users of the interactive system 100 based on privacy settings specified by the specific entity. In the case of an individual, profile data 302 includes, for example, a username, phone number, address, settings (such as notification and privacy settings), and an avatar representation (or a set of such avatar representations) selected by the user. A specific user can then selectively include one or more of these avatar representations within the content of messages transmitted via the interactive system 100 and on a map interface displayed to other users by the interactive client 104. The set of avatar representations may include “status avatars,” which present a graphical representation of a status or activity that the user can choose to transmit at a specific time.

[0076] In the case that the entity is a group, in addition to the group name, members and various settings of the associated group (such as notifications), the group profile data 302 may similarly include one or more avatar representations associated with the group.

[0077] Database 304 also stores enhancement data, such as overlays or filters, in enhancement table 312. The enhancement data is associated with and applied to the videos (whose data is stored in video table 314) and the images (whose data is stored in image table 316).

[0078] In some examples, filters are overlays displayed as superimposed on images or videos during presentation to the recipient user. Filters can be of various types, including filters selected by the user from a set of filters presented to the sending user by the interactive client 104 when the sending user composes a message. Other types of filters include geolocation filters (also known as geographic filters), which can be presented to the sending user based on geographic location. For example, the interactive client 104 may present geolocation filters specific to nearby or particular locations within the user interface based on geolocation information determined by the Global Positioning System (GPS) unit of the user system 102.

[0079] Another type of filter is a data filter, which can be selectively presented to the sending user by the interactive client 104 based on other inputs or information collected by the user system 102 during the message creation process. Examples of data filters include the current temperature at a specific location, the sending user's current travel speed, the battery life of the user system 102, or the current time.

[0080] Other augmented data that can be stored in image table 316 includes augmented reality content items (e.g., corresponding to an application lens or augmented reality experience). Augmented reality content items can be real-time effects and / or sounds that can be added to images or videos.

[0081] Collection table 318 stores data related to collections of messages and associated image, video, or audio data compiled into collections (e.g., stories or galleries). The creation of a specific collection can be initiated by a specific user (e.g., each user for whom records are maintained in entity table 308). A user can create a "personal story" in the form of a collection of content that has already been created and sent / broadcast by that user. For this purpose, the user interface of interactive client 104 may include user-selectable icons to allow the sending user to add specific content to his or her personal story.

[0082] Collections can also constitute "live stories," which are collections of content from multiple users created manually, automatically, or using a combination of manual and automatic technologies. For example, a "live story" can constitute a curated stream of user-submitted content from different locations and events. For instance, users with location services enabled on their client devices and who are at a common location event at a specific time can be presented with the option to contribute content to a specific live story via the user interface of interactive client 104. Users can be identified by interactive client 104 based on their location. The end result is a "live story" told from a community perspective.

[0083] Another type of content collection is called a "location story," which allows users of user system 102 located in a specific geographic location (e.g., within a college or university campus) to contribute to a specific collection. In some examples, contributions to a location story may employ a second level of verification to confirm that the end user belongs to a specific organization or other entity (e.g., is a student on a university campus).

[0084] As described above, video table 314 stores video data, which in some examples is associated with messages whose records are maintained within message table 306. Similarly, image table 316 stores image data associated with messages whose message data is stored in entity table 308. Entity table 308 can associate various enhancements from enhancement table 312 with various images and videos stored in image table 316 and video table 314.

[0085] Data communication architecture

[0086] Figure 4 This is a schematic diagram illustrating the structure of message 400 according to some examples, generated by interactive client 104 for transmission to another interactive client 104 via interactive server 124. The content of a particular message 400 is used to populate message table 306 within database 304 accessible by interactive server 124. Similarly, the content of message 400 is stored in memory as "in-transit" or "in-flight" data for user system 102 or interactive server 124. Message 400 is shown to include the following example components:

[0087] Message Identifier 402: A unique identifier that identifies message 400.

[0088] Message text payload 404: The text to be generated by the user via the user interface of user system 102 and included in message 400.

[0089] Message image payload 406: Image data captured by the camera device component of user system 102 or obtained from the storage component of user system 102 and included in message 400. The image data for the sent or received message 400 can be stored in image table 316.

[0090] Message video payload 408: Video data captured by the camera device component or obtained from the storage component of the user system 102 and included in message 400. The video data for the sent or received message 400 can be stored in image table 316.

[0091] Message audio payload 410: Audio data captured by the microphone or obtained from the storage component of the user system 102 and included in message 400.

[0092] Message enhancement data 412: This represents enhancement data (e.g., filters, labels, or other annotations or enhancements) to be applied to the message image payload 406, message video payload 408, or message audio payload 410 of message 400. Enhancement data for the sent or received message 400 can be stored in enhancement table 312.

[0093] Message duration parameter 414: A parameter value, in seconds, indicating the amount of time that the content of the message (e.g., message image payload 406, message video payload 408, message audio payload 410) will be presented to the user via the interactive client 104 or that is accessible to the user.

[0094] Message geolocation parameter 416: Geographic location data (e.g., latitude and longitude coordinates) associated with the message's content payload. Multiple message geolocation parameter 416 values ​​may be included in the payload, each of which is associated with a content item included in the content (e.g., a specific image within the message image payload 406 or a specific video within the message video payload 408).

[0095] Message Story Identifier 418: An identifier value that identifies one or more sets of content (e.g., “story” identified in set table 318) associated with a specific content item in the message image payload 406 of message 400. For example, multiple images within the message image payload 406 may each be associated with multiple sets of content using their respective identifier values.

[0096] Message Tag 420: Each message 400 can be labeled with multiple tags, each of which indicates the subject of the content included in the message payload. For example, in the case where a specific image depicts an animal (e.g., a lion) is included in the message image payload 406, a tag value can be included within the message tag 420 indicating the relevant animal. Tag values ​​can be manually generated based on user input, or can be automatically generated using, for example, image recognition.

[0097] Message sender identifier 422: An identifier (e.g., message sending system identifier, email address, or device identifier) ​​indicating the user of the user system 102 on which message 400 is generated and from which message 400 is sent.

[0098] Message receiver identifier 424: An identifier (e.g., message sending and receiving system identifier, email address, or device identifier) ​​indicating the user of the user system 102 to which message 400 is addressed.

[0099] The content (e.g., values) of each component of message 400 can be pointers to locations in tables where content data values ​​are stored. For example, image values ​​in message image payload 406 can be pointers to locations (or their addresses) within image table 316. Similarly, values ​​in message video payload 408 can point to data stored in image table 316, values ​​in message enhancement data 412 can point to data stored in enhancement table 312, values ​​in message story identifier 418 can point to data stored in set table 318, and values ​​in message sender identifier 422 and message receiver identifier 424 can point to user records stored in entity table 308.

[0100] Web browser experience

[0101] Figure 5 This is a flowchart illustrating the operation of a method for rendering a live view of special effects via a webpage, based on some examples. The process begins when a user, using a web browser application, makes an HTTP request to a webpage hosted by a web server providing the special effects services. Therefore, at operation 502, the web server receives the HTTP request from the web browser. At operation 504, the web server responds to the web browser by sending data representing the webpage, whereby the webpage, when rendered by the web browser, will display selectable special effects. In this case, each special effect may be associated with code hosted by the server, which causes the web browser to render the effect.

[0102] Next, at operation 506, the web server receives a request for a specific effect from the web browser, for example, because the user selects an effect from a selection of effects presented to the user. At operation 508, the web server responds by sending computer code (e.g., a script) to the web browser, which, when processed by the web browser, will cause the web browser to perform various operations to render the effect. For example, when the web browser processes the script received from the web server, it will access the camera of the client device to obtain a video stream from the camera. The web browser will then apply the effect to the live video stream according to the instructions specified in the script, resulting in a live view being rendered in the effects window of the webpage presented by the web browser. The web browser will also present a button or other GUI element that, when selected, will cause the capture of a media content item with the effect applied—e.g., a still image or video clip. The web browser will also present one or more buttons or GUI elements that the user can interact with to save or share the media content item. Further description and illustration of these options are described below in conjunction with a description of several accompanying figures illustrating an example user interface.

[0103] Figure 6 This is a user interface diagram illustrating an example of a webpage (e.g., a login page) 600 for selecting effects, consistent with some implementations. Webpage 600 displays several filtering or selection criteria 602, including options labeled "Recommended for You," "Trending," "Face," "World," "Music," and "Live." These categories and selection criteria 602 allow users to browse effects tailored to their interests, view popular effects, find effects to enhance faces or environments, and explore effects with music or live effects. For example, by selecting a specific GUI element (e.g., a button), available effects are selected or filtered by the selection criteria associated with that button, and the selected effects are then presented to the user via webpage 600.

[0104] In addition, several user-selectable graphics are displayed, each representing a different effect that can be previewed: "big lips" 604, "crying" 606, "smiling" 608, and "parrot" 610. Finally, a button 612 labeled "View more effects" is shown, which, when selected, will present additional effect options to the user.

[0105] This landing page 600 provides users with an intuitive and engaging way to explore the library of effects available through the web interface. Effects are organized into logical categories to suit various use cases. Users can click on any effect thumbnail or graphic to view a demo and try it out using their live camera feed. Additional effects can be added with a simple tap or mouse click. This allows informal users to easily discover interesting effects directly through their web browser without needing to install any additional software.

[0106] Figure 7 A user interface diagram is shown, illustrating a webpage 700 used for a specific effect. When webpage 700 is first rendered, an animation can be displayed in effects window 702, showing an example of the effect in operation. A button 704 labeled "Try the Parrot Effect" is displayed; when this button is selected, the following process is invoked, allowing the user to view a live view of the effect applied to an image captured in real-time from their device's camera.

[0107] Webpage 700 also includes a "Open in mobile app" button 706, which provides the user with the option to switch to an app-based experience if the user has the native mobile app installed. A profile image and the name "@JohnD" indicate an example social media user who has provided or posted the effect. Various additional effects are presented along with a "Find more effects" button 708, which, when selected, allows the user to view additional effects. In some implementations, additional effects can be selected for presentation on any particular effects page based on specific selection criteria. For example, additional effects can be related to the main effect, for instance, due to the user selecting the main effect (e.g., ...). Figure 7 (The parrot effect in the game) These additional effects are often chosen as well.

[0108] The Effects700 website allows users to preview specific effects (such as the "parrot" effect), easily try them out in real-time using their camera devices, and share the results through integrated social features. The option to open the effect or find more effects in the mobile app provides a seamless cross-platform experience. Users can enjoy the parrot effect entirely through their web browser without needing to install any additional software or register an account.

[0109] Figure 8 An example of a browser dialog box 802 prompting a user to authorize access to their device's microphone is shown. When webpage 700 attempts to access the microphone to enable a selected effect, the web browser detects this action and opens the dialog box. The dialog box states, "effect.app.com wants to use your microphone" and presents the option to "Allow" or "Block" access.

[0110] This dialog box is provided by the underlying operating system and allows users to control the access granted to websites to protected hardware, such as microphones. If the user selects "Allow," the webpage's JavaScript code gains access to live audio input, which can be processed and enhanced based on effects. If the user selects "Block," the website will block access to the microphone.

[0111] Requesting permission and authorization from users provides important privacy and security protections. Users are informed of what data the webpage is attempting to access and can choose to allow or deny access accordingly. Browsers and operating systems mediate and control access to hardware components (such as microphones), rather than unconditionally granting access to websites.

[0112] Figure 9 An example of browser dialog box 902 is shown, prompting the user to authorize access to their device's camera. When webpage 700 attempts to access the camera to enable a selected effect, the browser detects this and opens dialog box 902. The dialog box 902 states "effect.app.com wants to use your camera" and presents options to "Allow" or "Block" access.

[0113] This dialog box 902 is provided by the underlying operating system and allows users to control the access granted to websites to protected hardware, such as camera devices. If the user selects "Allow," the webpage's JavaScript code gains access to live video input, which can be processed and enhanced based on effects. If the user selects "Block," the website will be blocked from accessing the camera device.

[0114] Requesting permission and authorization from users provides important privacy and security protections. Users are informed of what data the webpage is attempting to access and can choose to allow or deny access accordingly. Browsers and operating systems mediate and control access to hardware components (such as camera devices), rather than unconditionally granting website access.

[0115] Figure 10 An example of dialog box 1002 is shown, prompting the user to agree to the terms of service and privacy policy before accessing effects. Dialog box 1002 states, "By using Effects, you confirm that you have read our privacy policy and agree to the terms of service." It further explains, "Some effects use information about your face, hands, and voice to work."

[0116] Dialog box 1002 prompts users to learn more, and if they agree, they can click the "I agree" button to continue. This ensures users understand how their biometric data may be used and provides an explicit option to join the consent process. Requesting this confirmation and consent to the policy provides transparency regarding data use and collection. Users are informed about how Effects uses personal information (such as facial and voice data). The system obtains the user's explicit consent before accessing any biometric or sensitive user data. This increases user awareness and trust in Effects' privacy practices.

[0117] Figure 11 An example of a live view in Effects window 702 is shown, displaying effects applied in real-time to the camera feed. This provides users with a preview of the effects on themselves, allowing them to see how the effects will enhance their appearance before capturing photos or videos. Button 1102 is presented within the Effects window displaying the live view, providing users with options for generating media content—for example, a short press of button 1102 captures a still photo, while a long press records a video clip. This allows users to easily create media content with applied effects.

[0118] The live preview shown in Effects window 702 builds excitement and engagement by demonstrating the effect running on the user's own camera feed. Integrated capture options allow for instant creation of works with effects for saving and sharing without the need to install a separate app or switch between services. Everything from effects selection to media creation and sharing can be done directly within the web interface.

[0119] Figure 12 An example interface 700 is shown, allowing users to save and share media content after creating it using special effects. Webpage 700 includes a "Send to Chat" option 1202, which directly shares images or video clips to a connected messaging platform. Alternatively, a "Download App" button 1204 allows users to download a mobile app for the special effects service. For example, if the user is currently using a mobile computing device (e.g., a smartphone), the "Download App" button 1204 can directly link to the download page of the mobile app in the app store of the operating system provider. Alternatively, if the user is currently using another device (e.g., a laptop or desktop computer), selecting the "Download App" button 1204 can result in a webpage displaying a QR code, which, when scanned with the mobile device's camera, will result in a link to the download page of the mobile app in the app store on the mobile device. Although in Figure 12 It is not shown in the image, but another button could be presented to allow users to save media content locally or in a cloud-based file service.

[0120] Integrated sharing and saving capabilities allow users to fully utilize their work created with effects without switching between multiple apps or services. They can download images or video clips to their camera's album or share them to connected social media accounts and messaging platforms with minimal effort. Streamlining the content creation and sharing process improves the overall user experience. Users can seamlessly complete everything from effect selection to media capture to sharing within a single web interface. This provides an unhindered workflow for creating and distributing compelling work using interactive effects.

[0121] Machine architecture

[0122] Figure 13 This is a schematic representation of machine 1300, within which instructions 1302 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause machine 1300 to perform any or more of the methods discussed herein. For example, instructions 1302 can cause machine 1300 to perform any or more of the methods described herein. Instructions 1302 transform a general, unprogrammed machine 1300 into a specific machine 1300 that is programmed to perform the described and illustrated functions in the described manner. Machine 1300 can operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 1300 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1300 may include, but is not limited to, server computers, client computers, personal computers (PCs), tablets, laptops, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, networked appliances, network routers, network switches, bridges, or any machine capable of sequentially or otherwise executing instructions 1302 that specify actions to be taken by machine 1300. Furthermore, while a single machine 1300 is shown, the term "machine" should also be considered as a collection of machines that individually or jointly execute instructions 1302 to perform any one or more of the methods discussed herein. For example, machine 1300 may include user system 102 or any of a plurality of server devices forming part of interactive server system 110. In some examples, machine 1300 may also include both client and server systems, wherein certain operations of a particular method or algorithm are performed on the server side and certain operations of said particular method or algorithm are performed on the client side.

[0123] Machine 1300 may include a processor 1304, a memory 1306, and an input / output (I / O) unit 1308 that can be configured to communicate with each other via a bus 1310. In the example, processor 1304 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processors 1312 and 1314 that execute instruction 1302. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although Figure 13 Multiple processors 1304 are shown, but machine 1300 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0124] Memory 1306 includes main memory 1316, static memory 1318, and storage cells 1320, all of which are accessible by processor 1304 via bus 1310. Main memory 1306, static memory 1318, and storage cells 1320 store instructions 1302 embodying any one or more of the methods or functions described herein. Instructions 1302 may also reside wholly or partially within main memory 1316, static memory 1318, machine-readable medium 1322 within storage cell 1320, at least one processor of processor 1304 (e.g., the processor's cache memory), or any suitable combination thereof during execution by machine 1300.

[0125] I / O component 1308 may include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurement results, etc. The specific I / O component 1308 included in a particular machine will depend on the type of machine. For example, a portable machine, such as a mobile phone, may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It will be understood that I / O component 1308 may include... Figure 13Many other components are not shown. In various examples, I / O component 1308 may include user output component 1324 and user input component 1326. User output component 1324 may include visual components (e.g., a display such as a plasma display panel (PDP), light-emitting diode (LED) display, liquid crystal display (LCD), projector, or cathode ray tube (CRT) display), acoustic components (e.g., a speaker), tactile components (e.g., a vibrating motor, a resistive mechanism), other signal generators, etc. User input component 1326 may include alphanumeric input components (e.g., a keyboard, a touchscreen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, touchpad, trackball, joystick, motion sensor, or other pointing instrument), tactile input components (e.g., a physical button, a touchscreen or other tactile input component that provides touch gestures or the position and force of a touch), audio input components (e.g., a microphone), etc.

[0126] In another example, I / O component 1308 may include biometric component 1328, motion component 1330, environmental component 1332, or position component 1334, as well as various other components. For example, biometric component 1328 includes components for detecting expressions (e.g., hand gestures, facial expressions, vocal expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and recognizing people (e.g., speech recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Biometric component may include a brain-computer interface (BMI) system that allows communication between the brain and external devices or machines. This can be achieved by recording brain activity data, converting that data into a format that can be understood by a computer, and then using the resulting signals to control the device or machine.

[0127] Examples of BMI technology types include:

[0128] BMI based on electroencephalography (EEG) uses electrodes placed on the scalp to record electrical activity in the brain.

[0129] Invasive BMI, which uses electrodes surgically implanted into the brain.

[0130] Optogenetics BMI uses light to control the activity of specific nerve cells in the brain.

[0131] Any biometric data collected by the biometrics component is captured and stored only with user approval and is deleted upon user request. Furthermore, such biometric data may be used for very limited purposes, such as authentication. To ensure the restricted and authorized use of biometric information and other personally identifiable information (PII), access to this data is limited to authorized personnel (if access to the data occurs). Any use of biometric data may be strictly limited to authentication purposes, and the data may not be shared or sold to any third party without the user's explicit consent. In addition, appropriate technical and organizational measures are implemented to ensure the security and confidentiality of this sensitive information.

[0132] The moving part 1330 includes an acceleration sensor part (e.g., an accelerometer), a gravity sensor part, and a rotation sensor part (e.g., a gyroscope).

[0133] Environmental component 1332 includes, for example, one or more camera devices (with still image / photograph and video capabilities), illuminance sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers for detecting ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), acoustic sensor components (e.g., one or more microphones for detecting background noise), proximity sensor components (e.g., infrared sensors for detecting nearby objects), gas sensors (e.g., gas detection sensors for detecting hazardous gas concentrations or measuring pollutants in the atmosphere for safety purposes), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.

[0134] Regarding the camera device, the user system 102 may have a camera device system, which includes, for example, a front-facing camera on the front surface of the user system 102 and a rear-facing camera on the rear surface of the user system 102. The front-facing camera may be used, for example, to capture still images and videos (e.g., “selfies”) of the user of the user system 102, and then enhance these still images and videos with enhancement data (e.g., filters) as described above. The rear-facing camera may be used, for example, to capture still images and videos in a more conventional camera mode, which are similarly enhanced with enhancement data. In addition to the front-facing and rear-facing cameras, the user system 102 may also include a 360° camera for capturing 360° photos and videos.

[0135] Furthermore, the camera system of user system 102 may include dual rear cameras (e.g., a main camera and a depth sensing camera) located on the front and rear sides of user system 102, or even triple, quadruple, or quintuple rear camera configurations. These multi-camera systems may include, for example, wide-angle cameras, ultra-wide-angle cameras, telephoto cameras, macro cameras, and depth sensors.

[0136] The position component 1334 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure from which altitude can be obtained), a direction sensor component (e.g., a magnetometer), etc.

[0137] Communication can be implemented using a wide variety of technologies. I / O component 1308 also includes a communication component 1336 operable to couple machine 1300 to network 1338 or device 1340 via a suitable coupling or connection. For example, communication component 1336 may include a network interface component or another suitable device that interfaces with network 1338. In further examples, communication component 1336 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components that provide communication via other modalities. Device 1340 may be another machine or any peripheral device from a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0138] Furthermore, communication component 1336 may detect identifiers or include components operable to detect identifiers. For example, communication component 1336 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, such as Quick Response (QR) codes, Aztec codes, data matrices, and data symbols). TM The communication component 1336 can be used to obtain various information, such as optical sensors for multidimensional barcodes (MaxiCode, PDF417, UltraCode, UCC RSS-2D barcodes, and other optical codes) or acoustic detection components (e.g., microphones for identifying audio signals of the markers). Furthermore, various information can be obtained via the communication component 1336, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detection of NFC beacon signals indicating a specific location, etc.

[0139] Various memories (e.g., main memory 1316, static memory 1318, and memory of processor 1304) and storage units 1320 may store one or more sets of instructions and data structures (e.g., software) used or embodied by any one or more of the methods or functions described herein. These instructions (e.g., instruction 1302) cause various operations to implement the disclosed examples when executed by processor 1304.

[0140] Instruction 1302 can be sent or received over network 1338 via a transmission medium using a network interface device (e.g., a network interface component included in communication component 1336) and using any of several known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instruction 1302 can be sent or received via a transmission medium through a coupling to device 1340 (e.g., a peer-to-peer coupling).

[0141] Software Architecture

[0142] Figure 14 This is a block diagram 1400 illustrating a software architecture 1402 that can be installed on any one or more of the devices described herein. The software architecture 1402 is supported by hardware of a machine 1404, including, for example, a processor 1406, memory 1408, and I / O components 1410. In this example, the software architecture 1402 can be conceptualized as a stack of layers, each providing specific functionality. The software architecture 1402 includes layers such as an operating system 1412, libraries 1414, frameworks 1416, and applications 1418. Operationally, application 1418 invokes API call 1420 via the software stack and receives message 1422 in response to API call 1420.

[0143] Operating system 1412 manages hardware resources and provides public services. Operating system 1412 includes, for example, a kernel 1424, services 1426, and drivers 1428. Kernel 1424 serves as an abstraction layer between hardware and other software layers. For example, kernel 1424 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. Services 1426 can provide other public services to other software layers. Drivers 1428 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 1428 may include display drivers, camera drivers, BLUETOOTH® or BLUETOOTH® low-power drivers, flash memory drivers, serial communication drivers (e.g., USB drivers), Wi-Fi® drivers, audio drivers, power management drivers, etc.

[0144] Library 1414 provides common low-level infrastructure used by application 1418. Library 1414 may include system library 1430 (e.g., the C standard library), which provides functions such as memory allocation, string manipulation, and mathematical functions. Furthermore, library 1414 may include API library 1432, such as media libraries (e.g., libraries for supporting the rendering and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Picture Experts Group (JPEG or JPG), or Portable Web Graphics (PNG)), graphics libraries (e.g., OpenGL frameworks for rendering graphic content on a display in two-dimensional (2D) and three-dimensional (3D) formats), database libraries (e.g., SQLite, which provides various relational database functions), network libraries (e.g., WebKit, which provides web browsing functionality), and so on. Library 714 may also include a wide variety of other libraries 1434 to provide many other APIs to application 1418.

[0145] Framework 1416 provides common high-level infrastructure for use by application 1418. For example, framework 1416 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. Framework 1416 can provide a wide range of other APIs that can be used by application 1418, some of which may be specific to a particular operating system or platform.

[0146] In the example, application 1418 may include home application 1436, contact application 1438, browser application 1440, book reader application 1442, location application 1444, media application 1446, messaging application 1448, game application 1450, and various other categories of applications, such as third-party application 1452. Application 1418 is a program that performs the functions defined in the program. One or more applications 1418 can be created using various programming languages, such as object-oriented programming languages ​​(e.g., Objective-C, Java, or C++) or procedural programming languages ​​(e.g., C or assembly language). In a particular example, third-party application 1452 (e.g., using Android by an entity other than the vendor of a particular platform). TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TMMobile software running on the Windows® Phone mobile operating system or another mobile operating system. In this example, third-party application 1452 can invoke API call 1420 provided by operating system 1412 to assist the functions described herein.

[0147] Example

[0148] Example 1 is a computing device including: a processor; and a memory storing instructions that, when executed by the processor, cause the computing device to perform operations including: displaying a webpage provided by a server by a web browser application executing on the computing device, the webpage including a series of special effects; receiving a user selection of the special effects from a series of choices by the web browser; in response to the user selection, the web browser executing a script provided by the server to: access a camera device of the client computing device; apply the selected special effects to a video stream received from the camera device; render the video stream with the applied special effects in a display window within the displayed webpage, thereby providing a live view of the special effects; while providing the live view of the special effects, the web browser receiving user capture input; in response to receiving the user capture input, capturing image data or video data from the live view of the special effects; displaying sharing options for sharing the captured image data or video data within the webpage; receiving a user selection of the sharing options by the web browser; and in response to receiving the user selection of the sharing options, the web browser executing instructions to determine a native sharing application available on the client computing device and programmatically invoking the native sharing application to share the captured image data or video data.

[0149] In Example 2, the theme of Example 1 includes, wherein the series of effects includes multiple selectable effects, and wherein the operation further includes: displaying a user interface element within the webpage, and when the user interface element is selected, causing the webpage to be refreshed to display an effect different from the selected effect.

[0150] In Example 3, the subject matter of Examples 1 and 2 includes, wherein the sharing options include multiple selectable sharing options, and wherein programmatically invoking the native sharing application includes: determining the type of captured image data or video data; determining the native sharing application associated with the type of captured image data or video data; and programmatically invoking the determined native sharing application to share the captured image data or video data.

[0151] In Example 4, the subject matter of Examples 1 to 3 includes, wherein the operations further include: displaying download options within a webpage for downloading captured image or video data; receiving a user selection of the download options by a web browser; in response to receiving the user selection of the download options: determining whether the client computing device is a mobile device or a desktop device; when the computing device is determined to be a desktop device, displaying a code for scanning by the mobile device's camera; and when the computing device is determined to be a mobile device, programmatically invoking the app store on the mobile device to install a native application associated with the webpage.

[0152] In Example 5, the subject matter of Examples 1 to 4 includes, wherein the operation further includes: displaying a preview option within a webpage for previewing captured image or video data; receiving a user selection of the preview option from a web browser; and, in response to receiving the user selection of the preview option, displaying a preview of the captured image or video data within the webpage.

[0153] In Example 6, the theme of Example 5 includes, where the display preview includes displaying captured image or video data with effects applied.

[0154] In Example 7, the topics of Examples 1 through 6 include, where the script is JavaScript code configured to access the camera device of the client computing device via a getUserMedia API call to the browser of the client computing device.

[0155] Example 8 is a computer-implemented method comprising: displaying a webpage provided by a server on a computing device via a web browser application, wherein the webpage includes a series of special effects; receiving a user selection of a special effect from the series of special effects via the web browser; in response to the user selection, executing a script provided by the server to perform the following actions: accessing a camera device of the computing device; applying the selected special effect to a video stream received from the camera device; rendering the video stream with the applied special effect in a display window within the webpage to provide a live view of the special effect; receiving user capture input via the web browser while providing the live view of the special effect; capturing image data or video data from the live view of the special effect in response to the user capture input; displaying sharing options for the captured image data or video data within the webpage; receiving a user selection of the sharing options via the web browser; and in response to the user selection of the sharing options, executing instructions to determine a native sharing application available on the computing device and programmatically invoking the native sharing application to share the captured image data or video data.

[0156] In Example 9, the subject of Example 8 includes displaying user interface elements within a webpage, whereby the webpage is refreshed when a user selects an element to display an effect different from the previously selected effect, wherein the selection of effects includes multiple selectable effects.

[0157] In Example 10, the topics of Examples 8 and 9 include: determining the type of captured image or video data; when the sharing options include multiple selectable sharing options, determining the native sharing application associated with the type of captured image or video data; and programmatically invoking the determined native sharing application to share the captured image or video data.

[0158] In Example 11, the topics of Examples 8 through 10 include displaying download options within a webpage for downloading captured image or video data; receiving a user selection of the download options via a web browser; and in response to receiving the user selection of the download options: determining whether the computing device is a mobile device or a desktop device; displaying a code for scanning by the mobile device's camera when the computing device is determined to be a desktop device; and programmatically invoking the app store on the mobile device to install a native application associated with the webpage when the computing device is determined to be a mobile device.

[0159] In Example 12, the topics of Examples 8 to 11 include displaying preview options within a webpage for previewing captured image or video data; receiving user selections of preview options via a web browser; and displaying a preview of the captured image or video data within the webpage in response to receiving user selections of preview options.

[0160] In Example 13, the subject of Example 12 includes displaying captured image or video data with effects applied.

[0161] In Example 14, the topics of Examples 8 through 13 include executing JavaScript code configured to access the camera device of a computing device via a call to the getUserMedia API of a web browser.

[0162] Example 15 is a server comprising: one or more processors; a memory storing instructions that, when executed by the processors, cause the server to perform operations including: receiving an HTTP request from a client device; in response to the HTTP request, sending a webpage to the client device for display in a web browser executed on the client device, the webpage including a series of special effects; receiving from the web browser on the client device a request for a script associated with an effect selected from the series of special effects; in response to receiving the request, sending the script to the web browser, wherein the script causes the web browser to: access a camera device of the client device; apply the selected special effects to a video stream from the camera device; render the video stream with the applied special effects in the webpage, providing a live view of the special effects; receive user capture input; in response to the user capture input, capturing image data or video data of the live view; displaying sharing options for sharing the captured image data or video data; receiving a user selection of the sharing options; and in response to the user selection of the sharing options, invoking a native sharing application on the client device to share the captured image data or video data.

[0163] In Example 16, the subject of Example 15 includes a selection of effects comprising multiple selectable effects, and wherein the operation further includes: receiving a request for a different script associated with a different effect selected from the selection of effects; and in response to receiving the request, sending the different script to a web browser, wherein the different script causes the web browser to apply different effects.

[0164] In Example 17, the subject matter of Examples 15 and 16 includes operations that further include: receiving captured image data or video data from a web browser; storing the received image data or video data; generating a link or identifier for accessing the stored image data or video data; and sending the link or identifier to the web browser.

[0165] In Example 18, the subject matter of Examples 15 to 17 includes operations that further include: providing an interface within a webpage for a user to preview a live view of an effect before capturing image or video data; receiving user input from a web browser on a client device indicating a desire to preview the effect; and, in response to the user input, enabling a preview mode on the web browser, wherein the user can interact with the live view of the effect without capturing image or video data.

[0166] In Example 19, the subject of Example 18 includes, wherein the operations further include: detecting user gestures performed during preview mode via a script; and modifying the live view of the effect to include interactive elements that respond to the user gestures in response to the detected user gestures.

[0167] In Example 20, the subject matter of Examples 15 to 19 includes, wherein the operations further include: receiving a request for additional information about the selected effect from a web browser on a client device; retrieving the additional information from a database communicating with a server; and sending the additional information to the web browser for display to the user, wherein the additional information includes at least one of the following: a description of the effect, instructions for use, or user-generated content featuring the effect.

[0168] Example 21 is at least one machine-readable medium including instructions that, when executed by a processing circuitry system, cause the processing circuitry system to perform operations for implementing any one of Examples 1 to 20.

[0169] Example 22 is an apparatus that includes means for implementing any one of Examples 1 to 20.

[0170] Example 23 is a system that implements any one of Examples 1 through 20.

[0171] Example 24 is a method for implementing any one of Examples 1 through 20.

[0172] Glossary

[0173] For example, "carrier signal" refers to any intangible medium or other intangible medium capable of storing, encoding, or carrying machine-executable instructions, including digital or analog communication signals. Instructions can be sent or received over a network using a transmission medium via a network interface device.

[0174] For example, a "client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, portable digital assistants (PDAs), smartphones, tablet computers, ultrabooks, netbooks, laptop computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0175] "Communications network" refers to one or more parts of a network, such as an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a Common Old-Style Telephone Service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, a network or a part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, coupling can enable any data transmission technology of various types, such as single-carrier radio transmission technology (1xRTT), evolved data optimization (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate GSM evolution (EDGE) technology, the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other long-distance protocols, or other data transmission technologies.

[0176] A “component” refers to a logical or physical entity having boundaries defined by functional or subroutine calls, branch points, APIs, or other technologies that provide partitioning or modularity for a particular processing or control function. A component can be combined with other components via its interface to perform machine processing. A component can be a packaged functional hardware unit designed to be used with other components, and part of a program that typically performs a related function. A component can constitute a software component (e.g., code embodied on a machine-readable medium) or a hardware component. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various examples, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., an application or application portion) to operate to perform certain operations described herein. A "hardware component" can be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic permanently configured to perform certain operations. A hardware component may be a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component may also include programmable logic or a circuit system temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific part of a machine) uniquely tailored to perform the configured function, and is no longer a general-purpose processor. It will be understood that the decision to implement a hardware component mechanically in a temporarily configured (e.g., software-configured) circuit system or in a dedicated and permanently configured circuit system may be driven by cost and time considerations. Therefore, the phrase "hardware component" (or...) "Hardware-implemented components" should be understood to encompass tangible entities that are physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain way or perform certain operations described herein. Consider examples of hardware components being temporarily configured (e.g., programmed), without needing to configure or instantiate each of the hardware components at any given time. For example, in cases where the hardware components include a general-purpose processor that becomes a dedicated processor through software configuration, the general-purpose processor can be configured at different times as (e.g., including different hardware components) different dedicated processors. The software accordingly configures one or more specific processors to constitute a particular hardware component, for example, at one time and different hardware components at different times. Hardware components can provide information to and receive information from other hardware components. Therefore, the described hardware components can be considered communicatively coupled.In the presence of multiple hardware components, communication can be achieved through signal transmission between or among two or more hardware components (e.g., via appropriate circuitry and buses). In examples where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, through the storage and retrieval of information in a storage structure accessible to the multiple hardware components. For example, a hardware component can perform an operation and store the output of that operation in a storage device communicatively coupled to it. Another hardware component can then access the storage device at a later time to retrieve and process the stored output. Hardware components can also initiate communication with input or output devices and be able to operate on resources (e.g., collections of information). The various operations of the example methods described herein can be performed at least in part by one or more processors, which are temporarily (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented at least in part by processors, where one or more specific processors are examples of hardware. For example, at least some of the operations of the methods can be executed by one or more processors or processor-implemented components. Furthermore, one or more processors can also operate to support the execution of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS). For example, at least some of the operations can be executed by a group of computers (as an example of machines including processors), where these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of some operations can be distributed across processors, not residing within a single machine, but deployed across multiple machines. In some examples, the processor or processor-implemented component may reside in a single geographic location (e.g., in a home environment, office environment, or server cluster). In other examples, the processor or processor-implemented component may be distributed across multiple geographic locations.

[0177] "Computer-readable storage medium" refers to both, for example, machine storage media and transmission media. Therefore, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" refer to the same thing and can be used interchangeably in this disclosure.

[0178] A "brief message" is a message that is accessible for a limited time, such as a short period of time. Brief messages can be text, images, videos, etc. The access time for a brief message can be set by the message sender. Alternatively, this access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is temporary.

[0179] "Machine storage medium" refers to, for example, a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. This term should be accordingly considered to include, but is not limited to, solid-state memory, as well as optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The terms "machine storage medium," "device storage medium," and "computer storage medium" refer to the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium," "computer storage medium," and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium."

[0180] "Non-transitory computer-readable storage medium" means, for example, a tangible medium capable of storing, encoding, or carrying instructions that can be executed by a machine.

[0181] "Signal medium" means, for example, any intangible medium capable of storing, encoding, or carrying machine-executable instructions and including digital or analog communication signals, or other intangible media that facilitates the transmission of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal whose characteristics are set or altered in a manner that encodes information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0182] "User equipment" means, for example, a device that is accessed, controlled, or owned by a user and that the user interacts with to perform actions or interactions (including interactions with other users or computer systems).

Claims

1. A computing device, comprising: processor; The memory stores instructions that, when executed by the processor, cause the computing device to perform operations, including: A webpage provided by a server is displayed by a web browser application running on the computing device; the webpage includes a series of special effects. The web browser receives the user's selection of an effect from the series of effects; In response to the user's selection, the web browser executes a script provided by the server to: The camera device accessing the computing device; Apply the selected special effects to the video stream received from the camera device; The video stream with the applied effects is rendered in a display window within the displayed webpage, thereby providing a live view of the effects; While providing a real-time view of the special effects, the web browser receives captured user input; In response to receiving the user-captured input, image data or video data is captured from the real-time view of the effect; The webpage displays sharing options for sharing captured image or video data. The web browser receives the user's selection of the sharing option; and In response to receiving a user selection of the sharing option, the web browser executes instructions to determine the native sharing applications available on the client computing device and The native sharing application is invoked programmatically to share the captured image or video data.

2. The computing device according to claim 1, wherein, The series of special effects includes multiple selectable effects, and the operation further includes: User interface elements are displayed within the webpage. When a user interface element is selected, the webpage is refreshed to display an effect different from the selected effect.

3. The computing device according to claim 1, wherein, The sharing options include multiple selectable sharing options, and wherein programmatically invoking the native sharing application includes: Determine the type of captured image or video data; Identify the native sharing application associated with the type of captured image or video data; and Programmatically invoke a designated native sharing application to share captured image or video data.

4. The computing device according to claim 1, wherein, The operation also includes: The webpage displays download options for downloading the captured image or video data. The web browser receives the user's selection of the download options; In response to receiving a user selection of the download option: Determine whether the client computing device is a mobile device or a desktop device; When the computing device is identified as a desktop device, a code is displayed for scanning by the mobile device's camera; and When the computing device is identified as a mobile device, the app store on the mobile device is programmatically invoked to install the native application associated with the webpage.

5. The computing device according to claim 1, wherein, The operation also includes: The webpage displays preview options for previewing the captured image or video data. The web browser receives the user's selection of the preview options; In response to receiving a user selection of the preview option, a preview of the captured image or video data is displayed within the webpage.

6. The computing device according to claim 5, wherein, Displaying the preview includes showing captured image or video data with the effect applied.

7. The computing device according to claim 1, wherein, The script is JavaScript code configured to access the camera device of the client computing device via a getUserMedia API call to the browser of the client computing device.

8. A computer-implemented method, comprising: A webpage provided by a server is displayed on a computing device via a web browser application, wherein the webpage includes a series of special effects; The web browser receives the user's selection of special effects from the series of special effects; In response to the user's selection, execute the script provided by the server to perform the following operations: The camera device accessing the computing device; Apply the selected special effects to the video stream received from the camera device; The video stream with the applied effects is rendered in a display window within the webpage to provide a live view of the effects; While providing a real-time view of the special effects, the web browser receives captured user input. In response to the user-captured input, image data or video data is captured from the real-time view of the effect; The webpage displays options for sharing the captured image or video data. Receive user selections of the sharing options through the web browser; and In response to a user selection of the sharing option, instructions are executed to determine the native sharing application available on the computing device and to programmatically invoke the native sharing application to share the captured image or video data.

9. The computer-implemented method according to claim 8, further comprising: The webpage displays user interface elements, which, when selected by the user, cause the webpage to refresh to display effects different from the previously selected effects. The series of effects includes multiple selectable effects.

10. The computer-implemented method according to claim 8, further comprising: Determine the type of captured image or video data; When the sharing options include multiple selectable sharing options, determine the native sharing application associated with the type of the captured image or video data; and Programmatically invoke a designated native sharing application to share captured image or video data.

11. The computer-implemented method according to claim 8, further comprising: The webpage displays download options for downloading the captured image or video data. The web browser receives the user's selection of the download options; as well as In response to receiving a user selection of the download option: Determine whether the computing device is a mobile device or a desktop device; When the computing device is identified as a desktop device, a code is displayed for scanning by the mobile device's camera. as well as When the computing device is identified as a mobile device, the app store on the mobile device is programmatically invoked to install the native application associated with the webpage.

12. The computer-implemented method according to claim 8, further comprising: The webpage displays preview options for previewing the captured image or video data. The web browser receives user selections for the preview options. as well as In response to receiving a user selection of the preview option, a preview of the captured image or video data is displayed within the webpage.

13. The computer-implemented method according to claim 12, further comprising: Display captured image or video data with the effect applied.

14. The computer-implemented method according to claim 8, further comprising: Execute JavaScript code configured to access the camera device of the computing device via a call to the getUserMedia API of the web browser.

15. A server, comprising: One or more processors; A memory storing instructions that, when executed by the processor, cause the server to perform operations, including: Receive HTTP requests from the client device; In response to the HTTP request, a webpage is sent to the client device for display in a web browser running on the client device, the webpage including a series of special effects; a request for a script associated with a special effect selected from the series of special effects is received from the web browser on the client device; In response to receiving the request, the script is sent to the web browser, wherein the script causes the web browser to: Access the camera device of the client device; Apply the selected special effects to the video stream from the camera device; Render the video stream with applied effects in the webpage, providing a real-time view of the effects; Receive captured user input; In response to the user capture input, capture image data or video data of the real-time view; Displays sharing options for sharing captured image or video data; Receive user selection of the sharing option; and In response to a user selection of the sharing option, the native sharing application on the client device is invoked to share the captured image or video data.

16. The server according to claim 15, wherein, The selection of special effects includes multiple selectable effects, and the operation further includes: Receive requests for different scripts associated with different effects selected from the series of effects; In response to receiving the request, the different scripts are sent to the web browser, wherein the different scripts cause the web browser to apply the different effects.

17. The server according to claim 15, wherein, The operation also includes: Receive captured image or video data from the web browser; Store the received image or video data; Generate links or identifiers for accessing stored image or video data; and Send the link or the identifier to the web browser.

18. The server according to claim 15, wherein, The operation also includes: The webpage provides an interface for the user to preview the special effects in real time before capturing the image data or video data; Receive user input indicating a desire to preview the special effect from the web browser on the client device; and In response to the user input, a preview mode is enabled on the web browser, wherein the user is able to interact with a live view of the effect without capturing the image data or the video data.

19. The server according to claim 18, wherein, The operation also includes: The script detects user gestures performed during the preview mode. In response to detecting the user gesture, the live view of the effect is modified to include interactive elements that respond to the user gesture.

20. The server according to claim 15, wherein, The operation also includes: Receive a request for additional information about the selected effect from the web browser on the client device; Retrieve the additional information from the database communicating with the server; and The additional information is sent to the web browser for display to the user, wherein the additional information includes at least one of the following: a description of the effect, instructions for use, or user-generated content featuring the effect.

Images