Computer-implemented methods, computing systems, and readable media

Abstract

A computer implementation is disclosed. The method includes: receiving video data from a real-time media stream; acquiring audience reaction data associated with the real-time media stream while streaming the real-time media stream, the audience reaction data indicating at least the amount of audience engagement activity related to the video content of the real-time media stream; identifying an event of interest in the real-time media stream based on determining that the rate of change of the amount of audience engagement activity exceeds a threshold level; and automatically initiating one or more defined actions in response to identifying the event of interest.

Description

Technical Field

[0001] This disclosure relates to media streaming technology, and more specifically to systems and methods for controlling the transmission of real-time media streams. Background Technology

[0002] Live streaming is a popular form of broadcasting content to online viewers. Live streaming services can be used to record and broadcast a wide variety of content, such as social media, interactive games, and sports coverage. Typical live streaming systems distribute content to viewers in a one-to-many model—the broadcaster creates a single stream of media content and transmits it to multiple viewers. Attached Figure Description

[0003] Embodiments will be described by way of example only with reference to the accompanying drawings, in which:

[0004] Figure 1A A sample system for processing real-time media streams is demonstrated, which includes a streaming media management engine;

[0005] Figure 1B It is configured for implementation Figure 1A A block diagram of an e-commerce platform based on an example embodiment of a streaming media management engine;

[0006] Figure 2 This is a block diagram of an e-commerce platform according to an example embodiment;

[0007] Figure 3 This is an example of an administrator's homepage based on the example embodiment;

[0008] Figure 4 An example method for processing audience reaction data associated with live media streams is illustrated in the form of a flowchart;

[0009] Figure 5 An example method for notifying broadcasters of events of interest in a live media stream is shown in the form of a flowchart;

[0010] Figure 6 An example method for selectively offering digital asset recommendations to viewers of live media streams is illustrated in the form of a flowchart;

[0011] Figure 7 An example method for generating digital assets containing real-time media streaming data is shown in the form of a flowchart;

[0012] Figure 8 An example method for obtaining broadcaster approval for generated digital assets is shown in the form of a flowchart;

[0013] Figure 9An example method for generating editable media objects based on data associated with multiple events of interest in a live media stream is shown in flowchart form;

[0014] Figure 10 An example method for providing modified media data for a real-time media stream based on a detected trigger action initiated by the broadcaster is illustrated in the form of a flowchart.

[0015] Figure 11 A flowchart illustrates an example method for providing digital asset recommendations alongside live media streaming; and

[0016] Figure 12 An example method for providing broadcasters with suggested actions regarding live media streams is illustrated in flowchart form. Summary of the Invention

[0017] Accordingly, a computer-implemented method, a computing system, and a computer program are provided, as described in detail in the appended claims. Detailed Implementation

[0018] Live streaming is a popular form of distributing media content to a large audience. Live streaming services can be used to record and broadcast a wide variety of content, such as social media, interactive games, and news reports. Other examples of live streaming include scheduled events such as concerts, sporting events, and product promotions. Live streaming can be initiated spontaneously by a broadcaster or according to a defined schedule. (The terms “broadcaster,” “streamer,” “stream creator,” and “host” are used interchangeably in this disclosure to refer to the entity broadcasting content via a live media stream.) Live streaming involves one-way broadcasting of content and typically requires source media (e.g., cameras, audio interfaces, etc.), encoders for digitizing the content, a media publishing entity, and a content distribution network for distributing the content to viewers.

[0019] Various platforms that support live streaming services enable viewers to interact with the live stream. Specifically, they allow viewers to interact with the broadcaster and / or other viewers. For example, a live streaming platform can provide a chat room user interface in which one or more viewers can participate. Viewers can communicate with each other or with the broadcaster during the live stream by typing comments, emoticons, etc. Viewers can also indicate their reaction to the live stream content by, for example, selecting one or more user interface elements (e.g., emotion icons) corresponding to their emotional response to the streaming content.

[0020] Conventional live streaming solutions do not provide broadcasters with sufficient support in controlling the delivery of the stream. During a live stream, broadcasters are often busy creating content and may not be able to closely review stream analytics, audience reactions, etc. Therefore, broadcasters may often lack signals or feedback regarding whether viewers are actually engaging with the live media stream. In many cases, broadcasters can employ incentives (e.g., subscriptions, giveaways, etc.) to create content that responds to viewer reactions and emotions. Specifically, it is desirable to provide broadcasters with content management tools that leverage audience reaction data from the live media stream.

[0021] On one hand, this application discloses a computer-implemented method. The method includes: receiving video data from a real-time media stream; acquiring audience reaction data associated with the real-time media stream while streaming the real-time media stream, the audience reaction data indicating at least the amount of audience engagement activity related to the video content of the real-time media stream; identifying an event of interest in the real-time media stream based on determining that the rate of change of the amount of audience engagement activity exceeds a threshold level; and automatically initiating one or more defined actions in response to identifying the event of interest.

[0022] In some implementations, audience response data may include user input obtained via a computing device associated with a viewer of the live media stream.

[0023] In some implementations, user input may include, for example, at least one of text input associated with a live media stream or selection of a defined user interface element.

[0024] In some implementations, the amount of audience participation can be determined based on the amount of user input.

[0025] In some implementations, acquiring audience reaction data may include determining that the audience reaction data is non-negative reaction data.

[0026] In some implementations, obtaining business data may include obtaining product preference data from e-commerce accounts associated with viewers.

[0027] In some implementations, obtaining audience response data may include: receiving audience response input; filtering the audience response input to exclude negative audience response input to generate the audience response data; and determining the amount of audience participation in the activity based on the audience response data.

[0028] In some implementations, automatically initiating the action of the one or more defined actions may include: generating recommendations for product discounts related to the defined product; and providing the generated recommendations to at least a subset of viewers of the live media stream.

[0029] In some implementations, generating recommendations may further include determining that the event of interest is relevant to the defined product.

[0030] In some implementations, automatically initiating the one or more defined actions may include providing one or more digital assets associated with the live media stream to at least one of: a subset of the viewers of the live media stream; or the stream creator associated with the live media stream.

[0031] In some implementations, automatically initiating one or more defined actions may include prompting at least a subset of viewers of the live media stream to obtain input related to the live media stream.

[0032] On the other hand, this application discloses a computing system. The computing system includes a processor and a memory storing computer-executable instructions that, when executed, cause the processor to: receive video data from a real-time media stream; acquire, while streaming the real-time media stream, audience reaction data associated with the real-time media stream, the audience reaction data indicating at least the amount of audience engagement activity related to the video content of the real-time media stream; identify an event of interest in the real-time media stream based on determining that the rate of change of the amount of audience engagement activity exceeds a threshold level; and automatically initiate one or more defined actions in response to identifying the event of interest.

[0033] On the other hand, this application discloses a computer-implemented method. The method includes: receiving media data from a real-time media stream; acquiring audience reaction data associated with the real-time media stream; identifying events of interest in the real-time media stream based on the audience reaction data, wherein the time of the events of interest precedes the time of the audience reaction data; obtaining a segment of at least one of audio or video data from the real-time media stream associated with the time of the events of interest; generating a digital asset containing the segment; and providing the digital asset to at least one viewer of the real-time media stream.

[0034] In some implementations, the segment may include at least one of audio capture data or video frame capture data from media data that is time-associated with the event of interest.

[0035] In some implementations, the digital asset may include product recommendations, which may include products for which at least one of audio capture data or video frame capture data is applicable.

[0036] In some implementations, the product may include clothing, household goods, or promotional items.

[0037] In some implementations, the digital asset may include an electronic product containing at least one of audio capture data or video frame capture data.

[0038] In some implementations, electronic products may include customizable Graphics Interchange Format (GIF) images.

[0039] In some implementations, the digital asset may include multiple segments of audio or video data, each of which corresponds to a specific event of interest.

[0040] In some implementations, the segment may include a video clip from video data that is time-associated with an event of interest.

[0041] In some implementations, generating the digital asset may include generating display data associated with the digital asset, and providing the digital asset may include providing the generated display data as overlay content of the live media stream.

[0042] In some implementations, the method may further include providing instructions to the stream creator associated with the live media stream regarding the digital asset.

[0043] In some implementations, providing instructions to the stream creator regarding the digital asset may include prompting the stream creator to approve the digital asset before providing it to at least one viewer of the live media stream.

[0044] On the other hand, this application discloses a computing system. The computing system includes a processor and a memory storing computer-executable instructions that, when executed, cause the processor to: receive media data from a real-time media stream; acquire audience reaction data associated with the real-time media stream; identify an event of interest in the real-time media stream based on the audience reaction data, wherein the time of the event of interest precedes the time of the audience reaction data; acquire a segment of at least one of audio or video data from the real-time media stream associated with the time of the event of interest; generate a digital asset containing the segment; and provide the digital asset to at least one viewer of the real-time media stream.

[0045] On the other hand, this application discloses a computer-implemented method. The method includes: receiving media data from a real-time media stream; detecting a trigger associated with the media data of the real-time media stream; in response to detecting the trigger, generating at least one of audio overlay content or video overlay content associated with the trigger; and sending the at least one of the audio overlay content or video overlay content along with the real-time media stream to a viewer's device.

[0046] In some implementations, the trigger for detecting media data may include one or more defined keywords in the audio data of the real-time media stream.

[0047] In some implementations, the method may further include first determining the keywords of the one or more definitions.

[0048] In some implementations, determining the one or more defined keywords may include retrieving keyword data from memory that identifies the one or more defined keywords as specified by the creator.

[0049] In some implementations, determining the one or more defined keywords includes: detecting an acceleration of the viewer's reaction greater than a threshold during the display of the media stream, during a previous media stream involving the stream creator; identifying a trigger time within the media stream associated with the audience's reaction greater than the threshold acceleration; and identifying the spoken version of the one or more defined keywords at the trigger time.

[0050] In some implementations, detecting triggers associated with media data may include detecting a defined gesture in the video data of a real-time media stream.

[0051] In some implementations, the method may further include: identifying events of interest based on audience reaction data associated with the live media stream; and providing an indication of the identified events of interest and one or more suggestion words associated with the identified events of interest to a stream creator associated with the live media stream, and detecting triggers associated with the media data may include detecting at least one of the suggestion words associated with the identified events of interest in the audio data of the live media stream.

[0052] In some implementations, the detection trigger may include determining that the content of the real-time media stream is related to a specific product, and generating at least one of audio overlay content or video overlay content may include generating media content related to the specific product.

[0053] In some implementations, determining that the content of the live media stream is related to a specific product may include determining that the live media stream is associated with a product tag corresponding to that specific product.

[0054] In some implementations, determining that the content of the live media stream is related to a specific product may include detecting the specific product in the video data of the live media stream.

[0055] On the other hand, this application discloses a computing system. The computing system includes a processor and a memory storing computer-executable instructions that, when executed, cause the processor to: receive media data from a real-time media stream; detect a trigger associated with the media data of the real-time media stream; in response to detecting the trigger, generate at least one of audio overlay content or video overlay content associated with the trigger; and transmit the at least one of the audio overlay content or video overlay content along with the real-time media stream to a viewer's device.

[0056] On the other hand, this application discloses a non-transitory computer-readable medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform at least some of the operations of the methods described herein.

[0057] Other exemplary embodiments of this disclosure will be apparent to those skilled in the art upon review of the following detailed description in conjunction with the accompanying drawings.

[0058] In this application, the term "and / or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any single listed element, any sub-combination of these elements, or all of them, and does not necessarily exclude additional elements.

[0059] In this application, the phrase “...and at least one of ...” is intended to cover any one or more of the listed elements, including any single listed element, any sub-combination of these elements, or all of them, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0060] In this application, the term "product data" generally refers to data associated with products offered for sale on an e-commerce platform. Product data may include, but is not limited to, product specifications, product category, manufacturer information, pricing details, availability, inventory(s), expected delivery time, shipping costs, and tax and customs information. Some product data may include static information (e.g., manufacturer name, product size, etc.), while other product data may be modified by the merchant on the e-commerce platform. For example, a merchant may change the price of a product at any time. Specifically, a merchant may set a product price to a specific value and update the price as desired. Once a customer places an order for a product at a specific price, the merchant commits to the pricing; that is, the price of the product in the ordered order will not change. Product data that the merchant can control (e.g., change, update, etc.) will be referred to as variable product data. More specifically, variable product data refers to product data that can be changed automatically or by the decision of the merchant offering the product.

[0061] In this application, the term "e-commerce platform" broadly refers to a computerized system (or service, platform, etc.) that facilitates commercial transactions (i.e., buying and selling activities) conducted via a computer network (e.g., the Internet). For example, an e-commerce platform can be a standalone online store, a social network, a social media platform, etc. Customers can initiate transactions and any associated payment requests through an e-commerce platform, and the e-commerce platform can be equipped with transaction / payment processing components or delegate such processing activities to one or more third-party services. An e-commerce platform can be extended by connecting one or more additional sales channels, which represent platforms where products can be sold. In particular, a sales channel itself can be an e-commerce platform, such as Facebook Shops. TM Amazon TM wait.

[0062] Real-time media streaming

[0063] This application discloses solutions to some of the aforementioned technical limitations of conventional live streaming systems. The proposed system is designed to automatically identify events of interest in a live media stream based on audience reaction data. More specifically, the system processes audience reaction data associated with the live media stream and identifies events of interest based on changes in metrics measuring audience reaction. Upon identification of an event of interest, the system can automatically initiate one or more defined actions associated with the live media stream. In particular, the detection of events of interest allows broadcasters to have greater control over the content (i.e., media data) created for delivery to viewers.

[0064] Now for reference Figure 1A The diagram illustrates a sample system 300 for processing real-time media streams in block form. Figure 1A As shown, system 300 may include viewer device 320, broadcaster device 330, video broadcasting system 340, and network 125 connecting one or more components of system 300.

[0065] As shown, viewer device 320 and broadcaster device 330 communicate via network 125. In at least some embodiments, each viewer device 320 and broadcaster device 330 may be a computing device. Viewer device 320 and broadcaster device 330 may take many forms, including, for example, mobile communication devices (such as smartphones), tablet computers, wearable computers (such as head-mounted displays or smartwatches), laptop computers or desktop computers, or other types of computing devices.

[0066] Broadcaster device 330 is associated with a broadcaster. Specifically, broadcaster device 330 enables the broadcaster to initiate the streaming of media content to one or more viewers. In at least some embodiments, broadcaster device 330 may have a media streaming application 332 residing thereon. Media streaming application 332 may be a standalone application (e.g., a mobile app) or a web-based application. The broadcaster may launch media streaming application 332 on broadcaster device 330 and initiate a live media (e.g., audio, video, etc.) stream. The live video stream may be directly transmitted to viewer device 320. Alternatively, the live video stream may be transmitted to an intermediate video broadcasting system 340. In some embodiments, video broadcasting system 340 may be a social networking system, and media streaming application 332 may be a social networking application for gaining access to the social network. Broadcaster device 330 may communicate with the server of video broadcasting system 340 via media streaming application 332. Video broadcasting system 340 may then transmit the live video stream to viewer device 320. The media streaming application 332 may include various monitoring and management functions involved in generating a live video stream. For example, broadcasters can use the media streaming application 332 to control the transmission settings of the live video stream, manage viewer permissions, and monitor audience reactions.

[0067] Viewer device 320 is associated with a viewer of the live video stream. The viewer can access the live video stream using a media streaming application 322, which can be a standalone application or a web-based application. For example, a web browser, social networking application, media playback application, etc., can be used to watch the live video stream. Viewer device 320 can communicate directly with broadcaster device 330, or they can communicate with the server of video broadcasting system 340.

[0068] Video broadcasting system 340 provides a platform for sharing content via video data streams, including live video streams. Video broadcasting system 340 may include a server configured to receive and transmit live media streams. In at least some embodiments, video broadcasting system 340 may be a social networking system. Specifically, video broadcasting system 340 may be a computing system capable of hosting online social networks. Users can access social networks to broadcast content to other users or watch content streamed by other users. For example, video broadcasting system 340 may provide websites or software (e.g., social media apps) that enable users to initiate or watch live video streams. Video broadcasting system 340 receives digitally encoded data representing the live video stream from broadcaster device 330, and viewer device 320 accesses the server of video broadcasting system 340 to receive the transmission of encoded video stream data.

[0069] System 300 provides a streaming media management engine 310. The streaming media management engine 310 may be a software-implemented module containing processor-executable instructions that, when executed by one or more processors, cause the computing system to perform some of the processes and functions described herein. In some embodiments, the streaming media management engine 310 may be provided as a standalone service. In particular, the computing system may use the streaming media management engine 310 as a service to facilitate the processing of real-time video streams.

[0070] The streaming media management engine 310 is configured to receive audio and video data from a real-time video stream. Specifically, the streaming media management engine 310 can be communicatively connected to one or more broadcaster devices 330. For example, the broadcaster device 330 can send real-time video stream data directly to the streaming media management engine 310, or the real-time video stream data can be received at the streaming media management engine 310 via an intermediate system such as a video broadcasting system 340.

[0071] According to one or more disclosed embodiments, the streaming media management engine 310 can facilitate the customization of live video streams for individual viewers. For example, the streaming media management engine 310 can send a modified version of the original live video stream to the viewer's device. That is, the media data (e.g., audio, video, etc.) of the live video stream can be modified by the streaming media management engine 310 before being sent to the viewer. Alternatively, the streaming media management engine 310 can send the original stream and instructions on how to modify the stream on the client side (i.e., at the viewer's device) before presenting the stream to the viewer. For example, the streaming media management engine 310 can be configured to provide personalized overlay content sent to the viewer's device along with the original live video stream.

[0072] The streaming media management engine 310 includes a media stream processing module 312. The media stream processing module 312 performs operations for processing media data associated with live streaming. The media stream processing module 312 receives real-time video feeds from various sources (e.g., video mixers, broadcaster devices, etc.). The real-time video feeds can be in compressed or uncompressed formats. The media stream processing module 312 can provide the real-time video feeds to multiple video encoders that compress the real-time video feeds using one or more codecs (e.g., MPEG-2, H.264, etc.).

[0073] The media stream processing module 312 can perform analysis of media content associated with the live video feed. In some embodiments, the media stream processing module 312 can perform object detection in the live video stream. Specifically, the media stream processing module 312 can perform real-time detection of objects (e.g., people, physical objects, etc.) and associated features and actions based on analysis of audio and / or video data from the live video stream. For example, the media stream processing module 312 can be configured to detect the posture and spoken keywords of subjects appearing in the live video stream.

[0074] While real-time video feeds have been discussed above, this process can also be performed in the context of other types of real-time media content. For example, media stream processing module 312 can perform analysis on real-time audio feeds or real-time text sessions to detect / identify objects or extract information related to objects (e.g., physical objects, people, etc.). Taking real-time audio feeds as an example, this can be achieved by performing audio signal analysis to parse the audio signal in real time. It should be noted that, depending on the signal type of the real-time media content, any suitable engineering techniques or technologies can be used to analyze various real-time media content or feed sessions in real time for object detection.

[0075] In some embodiments, the media streaming module 312 can capture segments of audio or video data from a live video stream. For example, the media streaming module 312 can identify significant portions of the live video stream and capture audio and / or video segments associated with those identified portions. The segments to be captured can be determined based on input from the broadcaster's device and / or the viewer's device (e.g., timestamps indicating highlights in the live video stream) or based on various defined rules for media capture, some of which will be described in more detail below.

[0076] The streaming media management engine 310 also includes a media generation module 314. The media generation module 314 is configured to generate audio or image / video data as overlay content for a real-time video stream. For example, the media generation module 314 can generate replacement graphics or audio that can be used to overlay at least a portion of the real-time video stream. Specifically, the overlay content generated by the media generation module can be presented along with the original stream of the real-time video stream.

[0077] The streaming media management engine 310 also includes an audience engagement processing module 316. The audience engagement processing module 316 is configured to collect audience engagement data, which may include viewer statistics (e.g., number of viewers, number of featured items added to a shopping list, viewer demographics) and audience reaction data (e.g., type, frequency, timing, and duration of reactions to the content). Audience reaction data may include, for example, analyses related to viewers' emotional reactions to the content of the live video stream. Emotional reactions may include expressions of viewer response to the content, such as "like," "praise," "love," "care," "sadness," "anger," "curiosity," etc.

[0078] The streaming media management engine 310, viewer device 320, broadcaster device 330, and video broadcasting system 340 may be geographically different locations. In other words, viewer device 320 may be located away from one or more of the following: streaming media management engine 310, broadcaster device 330, and video broadcasting system 340. As mentioned above, viewer device 320, broadcaster device 330, streaming media management engine 310, and video broadcasting system 340 may be computing systems.

[0079] Network 125 is a computer network. In some embodiments, network 125 may be, for example, an interconnected network that may be formed by one or more interconnected computer networks. For example, network 125 may be or may include Ethernet, Asynchronous Transfer Mode (ATM) network, wireless network, etc.

[0080] In some example embodiments, the streaming media management engine 310 can be integrated as a component of an e-commerce platform. That is, the e-commerce platform can be configured to implement example embodiments of the streaming media management engine 310. More specifically, the subject matter of this application—including the example methods disclosed herein for controlling the transmission of real-time media streams—can be used in the specific context of e-commerce.

[0081] refer to Figure 1BThe figure illustrates an example embodiment of an e-commerce platform 105 implementing a streaming media management engine 310. Viewer devices 320 and broadcaster devices 330 can be communicatively connected to the e-commerce platform 105. In at least some embodiments, viewer devices 320 and broadcaster devices 330 can be associated with accounts on the e-commerce platform 105. More specifically, viewer devices 320 and broadcaster devices 330 can be associated with entities (e.g., individuals) that have accounts associated with the e-commerce platform 105. For example, one or more viewer devices 320 and broadcaster devices 330 can be associated with customers (e.g., customers with e-commerce accounts) or merchants who have one or more online stores on the e-commerce platform 105. The e-commerce platform 105 can, for example, store indications of the association between viewer devices / broadcaster devices and merchants or customers on the e-commerce platform in data facility 134.

[0082] In at least some embodiments, the e-commerce platform 105 can provide processing facilities for streaming media. The e-commerce platform 105 can be used to provide viewer-specific, customized content for viewers of the live video stream. More specifically, components of the e-commerce platform 105 can be configured to provide customized streams containing product variant information tailored to individual viewers of the live video stream.

[0083] E-commerce platform 105 includes a business management engine 136, a streaming media management engine 310, data facilities 134, and data storage 302 for streaming media-related analytics. The business management engine 136 can be configured to handle various operations related to e-commerce accounts associated with e-commerce platform 105. For example, the business management engine 136 can be configured to obtain e-commerce account information for various entities (e.g., merchants, customers, etc.) and historical account data (such as transaction event data, browsing history data, etc.) for selected e-commerce accounts. Specifically, the business management engine 136 can obtain account information of e-commerce accounts associated with e-commerce platform 105 for viewers and / or broadcasters of the live video stream.

[0084] In at least some embodiments, the business management engine 136 can determine preferred product variations and recommendations for selected viewers of the live video stream, which can be used to customize the live video stream for each viewer. For example, the business management engine 136 can determine discounts, special offers, incentives, etc., to be offered to selected viewers of the live video stream based on account information associated with the viewer's e-commerce account. In some embodiments, the business management engine 136 can coordinate with the streaming media management engine 310 to control viewer access to events such as discounts and special offers that can be offered as part of a customized live video stream. Additionally, the business management engine 136 can manage the connection between viewers' streaming / social network accounts and their e-commerce accounts.

[0085] The functionalities described in this article can be used in business to provide an improved customer or buyer experience. E-commerce platform 105 can implement functionalities for any of a variety of different applications, and examples of these applications are described in this article. While Figure 1B The streaming media management engine 310 is shown as a standalone component of the e-commerce platform 105, but this is merely an example. The engine may also, or alternatively, be provided by another component residing within or outside the e-commerce platform 105. In some embodiments, one or more applications associated with the e-commerce platform 105 may provide the engine to implement the functions described herein, making the functions available to customers and / or merchants. Furthermore, in some embodiments, the business management engine 136 may provide this engine. However, the location of the streaming media management engine 310 may be implementation-specific. In some embodiments, the streaming media management engine 310 may be provided at least partially by the e-commerce platform as a core function of the e-commerce platform, or as an application or service supported by or communicating with the e-commerce platform. Alternatively, the streaming media management engine 310 may be implemented as a standalone service to clients such as customer devices or merchant devices. Additionally, at least a portion of such an engine may be implemented in merchant devices and / or customer devices. For example, the customer device may store and run the engine locally as a software application.

[0086] The streaming media management engine 310 is configured to implement at least some of the functions described herein. While the embodiments described below can be implemented in association with an e-commerce platform (such as, but not limited to, e-commerce platform 105), the embodiments described below are not limited to e-commerce platforms.

[0087] In some embodiments, the streaming media management engine 310 may allow the association of a live video stream with an e-commerce account associated with the e-commerce platform 105. For example, the streaming media management engine 310 may determine that a broadcaster of the live video stream is associated with a merchant on the e-commerce platform 105. The broadcaster may be a merchant (e.g., a gamer selling game-related merchandise, a social media influencer selling branded products, etc.), the broadcaster may specify the merchants for which it streams content (e.g., a social media influencer showcasing products sponsored by a merchant), or the merchant may specify the broadcasters authorized to showcase their products in the live video stream. The streaming media management engine 310 may associate the live video stream with a merchant. As another example, the streaming media management engine 310 may determine that one or more viewers of the live video stream are associated with customer accounts on the e-commerce platform 105. The streaming media management engine 310 may associate the live video stream with the e-commerce accounts of those customers watching the live video stream.

[0088] In at least some embodiments, the media generation module 314 collaborates with the commerce management engine 136 to generate overlay content for the live video stream. Specifically, the media generation module 314 may obtain e-commerce account data of viewers of the live video stream via the commerce management engine 136 and generate overlay content based on the account data to provide a personalized version of the live video stream.

[0089] Data facility 134 may store data collected by e-commerce platform 105 based on interactions between merchants and customers and e-commerce platform 105. For example, merchants provide data through their online sales activities. Examples of merchant data include, but are not limited to, merchant identification information, product data of products offered for sale, online store settings, geographic regions of sales activities, historical sales data, and inventory locations. Customer data, or data based on interactions between customers and potential buyers and e-commerce platform 105, may also be collected and stored in data facility 134. Such customer data is obtained based on input received via customer devices associated with customers and / or potential buyers. For example, historical transaction event data, including details of customer purchase transactions on e-commerce platform 105, may be recorded, and such transaction event data can be considered customer data. Such transaction event data may indicate product identifiers, purchase date / time, final selling price, buyer information (including the customer's geographic region), and payment method details, etc. Other data regarding the use of e-commerce platform 105 by merchants and customers (or potential buyers) may also be collected and stored in data facility 134.

[0090] Data facility 134 may include customer preference data of customers of e-commerce platform 105. For example, data facility 134 may store account information, order history, browsing history, etc., for each customer with an account associated with e-commerce platform 105. Data facility 134 may also store wish list data and shopping cart content data of one or more virtual shopping carts for multiple e-commerce accounts.

[0091] Now for reference Figure 4 The diagram illustrates, in flowchart form, an example method 400 for processing audience reaction data associated with a real-time media stream. Method 400 can be implemented by a computing system (e.g., [missing information]) that performs media stream processing. Figure 1A The streaming media management engine (310) is executed. As detailed above, the streaming media management engine can be a service provided inside or outside the e-commerce platform to facilitate the integration of real-time video streaming with e-commerce activities.

[0092] In operation 402, the streaming media management engine receives video data from a live media stream. The live video stream may be sent by a computing device associated with a broadcaster. In some embodiments, the video data may be transmitted directly from the broadcaster's device to the streaming media management engine. Alternatively, the streaming media management engine may receive video data from a video broadcasting system (such as a server of an online social network). The live video stream may be broadcast by users of the social network, and the video data may be sent from the social network server to be delivered to viewer devices associated with other users of the social network.

[0093] The streaming management engine receives video data before delivering the live video stream to viewers. That is, for one or more viewers of a live video stream—i.e., viewers requesting access to the live video stream—the streaming management engine can process the video data of the live video stream before delivering the streaming content to the viewer. Specifically, the streaming management engine is configured to receive and process the raw video data of the live video stream in real time and deliver the modified video data to the viewer.

[0094] In operation 404, the streaming management engine acquires audience reaction data associated with the live media stream while streaming it. The audience reaction data indicates at least the amount of audience engagement activity related to the video content of the live media stream. Various metrics can be used to measure the amount of audience engagement, such as the number of viewer reactions (e.g., selected emotion icons, typed responses, etc.), the number of viewers watching, the peak number of concurrent viewers, the number of unique viewers, the average amount of time a viewer spends watching the live stream (i.e., minutes, seconds), etc. In at least some embodiments, the streaming management engine can determine that the audience reaction data is non-negative reaction data. That is, the audience reaction data may only include positive reactions (e.g., likes, love, etc.).

[0095] In at least some embodiments, audience reaction data may include user input obtained from a viewer's device via a live media stream. For example, user input may be at least one of text input associated with a live media stream or selection of a defined user interface element.

[0096] In operation 406, the streaming media management engine identifies events of interest in the live media stream. Events of interest are identified based on determining that the rate of change in the amount of viewer engagement exceeds a threshold level. In at least some embodiments, the streaming media management engine may determine a baseline “velocity” of viewer engagement. For example, this “velocity” may represent the average (or median, etc.) of a metric of viewer engagement over a defined time period (e.g., a 1-minute span of the live video stream). The rate of change in the quantity can be monitored to detect in-stream events that result in a substantial response from the viewing viewer. That is, an event of interest may be an in-stream event that occurs immediately prior to the time at which an acceleration in viewer engagement is detected.

[0097] In some embodiments, an event of interest can be defined as an in-stream action that occurs within a predefined time window prior to a detected acceleration. For example, an event of interest can be an in-stream event whose start time falls within a defined time window prior to a detected acceleration. In some embodiments, an event of interest can be defined as an event whose start time corresponds to an inflection point in the rate of change of audience engagement. For example, an event of interest can be an in-stream event whose start time corresponds to the start point of an acceleration in audience engagement.

[0098] In at least some embodiments, the amount of audience participation can be determined based on the amount of user input received via the viewer's device (e.g., typed responses or activation of user interface elements).

[0099] In response to identifying an event of interest, in operation 408, the streaming media management engine automatically initiates one or more defined actions. For example, in some embodiments, the streaming media management engine may generate recommendations for product discounts related to a defined product. These recommendations may be generated based on determining that the event of interest is related to the defined product. The generated recommendations may be offered to at least a subset of viewers of the live media stream. For example, the generated recommendations may be offered to one or more active participants in the live video stream (e.g., viewers providing responses). In some embodiments, the streaming media management engine may offer digital assets associated with the live media stream to viewers of the live media stream and / or broadcasters associated with the live media stream. For example, the streaming media management engine may offer stream-related rewards (e.g., badges) to selected viewers.

[0100] Now for reference Figure 5The diagram illustrates, in flowchart form, an example method 500 for detecting events of interest in a real-time media stream and notifying the broadcaster of these events. Method 500 can be implemented by a computing system (e.g., [missing information]) that performs media stream processing. Figure 1A The streaming media management engine 310) executes the operation. The operation of method 500 can be executed as a supplement to or alternative to one or more operations of method 400.

[0101] In operation 502, the streaming management engine receives video data from the live media stream. In operation 504, the streaming management engine obtains values ​​for defined attributes associated with the live media stream. In particular, these attributes may relate to the amount of audience engagement or their reaction to the live media stream content. Reactions may include, for example, emotional responses to the live media stream (e.g., likes, love, etc.) and comments (and more generally, typed responses).

[0102] In Operation 506, the streaming management engine determines a first threshold level based on the values ​​of defined attributes. The threshold level represents the value at which one or more current values ​​of a defined attribute are compared to determine whether an event of interest has occurred in the live media stream. The definition of the threshold level can depend on several factors, such as the type of stream and historical changes in the second-order values ​​of audience engagement metrics.

[0103] In operation 508, the streaming management engine compares the rate of change of audience reaction data to a first threshold level. In other words, the streaming management engine can compare the acceleration of reactions associated with the live media stream, such as positive emotional responses, to a first threshold level. For example, an acceleration of positive responses exceeding a known threshold could indicate anchored engagement of the live media stream audience with events of interest.

[0104] In operation 510, the streaming media management engine identifies events of interest in the live media stream based on this comparison. Specifically, the streaming media management engine determines whether the acceleration of a positive response to the live media stream exceeds a first threshold level. If the acceleration does indeed exceed the first threshold level, it can be determined that an event of interest has occurred in the live media stream.

[0105] If, in operation 512, the streaming management engine determines that the detected event of interest is one of a pre-selected set of events, the streaming management engine returns data monitoring audience reactions and compares relevant metrics to defined thresholds (operation 508). Traditionally, the broadcaster pre-selects events of interest and may even act out or script these events to elicit specific audience reactions. If the identified event of interest is such a pre-selected event, neither the streaming management engine nor the broadcaster may take any action.

[0106] On the other hand, if the detected event of interest is not one of the pre-selected events, then in operation 514, the streaming media management engine can send an indication of the detected event to the broadcaster. Therefore, the streaming media management engine allows for the capture of events of interest beyond those that the broadcaster has pre-predicted or is already known to be of interest.

[0107] Now for reference Figure 6 The diagram illustrates, in flowchart form, an example method 600 for selectively providing digital asset recommendations to viewers of live media streams. For example, in the context of e-commerce, a broadcaster might be able to identify, according to method 600, specific points in time when product-related recommendations (e.g., discounts, special offers, etc.) should be provided to a selected subset of live media stream viewers. Method 600 can be implemented by a computing system (e.g., [missing information]) that performs media stream processing. Figure 1A The streaming media management engine 310) executes the operation. The operation of method 600 can be executed as a supplement to or alternative to one or more operations of methods 400 and 500.

[0108] In operation 602, the streaming management engine obtains audience reaction input associated with the live media stream. Audience reaction input may include, for example, typed text, selections of user interfaces (e.g., likes, emotion icons), and spoken words. In operation 604, the streaming management engine filters the audience reaction input to generate audience reaction data. Specifically, the streaming management engine may filter audience reaction input to exclude negative audience reaction input, thereby generating audience reaction data.

[0109] In at least some embodiments, negative audience reactions may include reactions that are not expected to be counted as part of the overall audience reaction count. When tracking audience reactions, it is desirable to detect activities that could be considered spam or otherwise affect the accuracy of analyses associated with audience engagement. For example, actions designed to trigger certain incentives for viewers (such as consecutive or repeated clicks to trigger discount recommendations) may affect the accuracy of audience engagement data. Such known activities can be filtered, for example, by detecting the frequency of viewer engagement or by looking at the engagement rate performed by the viewer (or associated e-commerce account).

[0110] In operation 606, the streaming management engine identifies events of interest in the live media stream based on filtered audience response data. The amount of audience engagement can be determined based on the audience response data. In operation 608, the streaming management engine automatically provides digital asset recommendations to selected viewers of the live media stream. For example, discounts or special offers can be automatically offered to a defined number of viewers of the live media stream or to those viewers who perform a specific action explicitly requesting a recommendation.

[0111] In operation 610, the streaming management engine processes viewer-initiated actions in response to digital asset recommendations based on one or more defined rules. For example, recommendations for discounts, special offers, etc., may be offered only during specific time windows. In some cases, the time window for claiming recommendations / rewards may be dynamically changed to avoid predictable outcomes. In some embodiments, the streaming management engine may prompt at least a subset of viewers of the live media stream for input related to the live media stream. This input may be a specific type of input (e.g., a typed answer to a question), which can be used to determine whether to reject a viewer's request for a recommendation. In some cases, the streaming management engine may perform secondary analysis to identify frequent participants and cross-type groups to understand viewer streaming consumption clustering data and utilize this data to provide "tickets" that can be used to obtain specific rewards, discounts, special offers, etc.

[0112] Now for reference Figure 7 The diagram illustrates, in flowchart form, an example method 700 for generating digital assets containing media data in the form of a real-time media stream. Method 700 can be implemented by a computing system (e.g., [missing information]) that performs media streaming processing. Figure 1A The streaming media management engine (310) is executed.

[0113] In operation 702, the streaming media management engine receives media data from the live media stream. In operation 704, the streaming media management engine obtains audience reaction data associated with the live media stream. Operations 702 and 704 can be performed in a similar manner to operations 402 and 404 of method 400.

[0114] In Operation 706, the streaming media management engine identifies events of interest in the live media stream based on viewer response data. The time associated with the event of interest precedes the time associated with the viewer response data; that is, the viewer response follows the event of interest.

[0115] In operation 708, the streaming media management engine obtains a segment of at least one of audio or video data from the real-time media stream associated with the time of the event of interest. The time associated with the detected event of interest may refer to the start time, end time, or a defined time window of the event of interest. In at least some embodiments, the segment may include at least one of audio capture data or video frame capture data from the media data associated with the time of the event of interest. For example, the segment may include a video frame (i.e., an image) or video clip from the video data associated with the time of the event of interest.

[0116] In operation 710, the streaming media management engine generates a digital asset containing the segment. In some embodiments, the digital asset may include a product recommendation, which includes a product for which at least one of the audio capture data or video frame capture data is applicable. For example, the product may be one of clothing, household goods, or promotional items, and the digital asset may be a recommendation for such a product (the printed image of the frame capture is applicable to the product).

[0117] In some embodiments, digital assets may include electronic artifacts containing at least one of audio capture data or video frame capture data. For example, the electronic artifact may be or include a customizable Graphics Interchange Format (GIF) image. In some embodiments, the streaming media management engine may generate display data associated with the digital assets and provide the generated display data as overlay content for a live media stream.

[0118] In operation 712, the streaming management engine provides digital assets to at least one viewer of the live media stream. For example, digital assets can be provided to a subset of viewers participating in the live media stream (e.g., via user interface elements corresponding to emotional responses, typed responses, etc.).

[0119] Now for reference Figure 8 The diagram illustrates, in flowchart form, an example method 800 for obtaining broadcaster approval regarding generated digital assets. Method 800 can be implemented by a computing system (e.g., [missing information]) that performs media streaming processing. Figure 1A The streaming media management engine 310) executes the operation. The operation of method 800 can be executed as a supplement to or alternative to one or more operations of method 700.

[0120] In operation 802, the streaming media management engine obtains a segment of audio or video data from the real-time media stream associated with the detected event of interest. According to the example embodiment described above, the event of interest could be, for example, an event associated with an acceleration of a positive response to the real-time media stream.

[0121] In Operation 804, the streaming management engine identifies the product to be offered to viewers of the live media stream. This product can be one of a pre-selected set of goods by the merchant associated with the live media stream. For example, the product could be clothing, household goods, or promotional items.

[0122] In operation 806, the streaming media management engine generates digital assets associated with the segment and the identified product. For example, the digital asset could be a recommendation of a discount or special offer on a product, presented as a graphical representation (e.g., a screenshot) of the captured video frame. In some embodiments, the product can be processed using one or more filters to alter the characteristics of the graphical representation (e.g., cartoonization, desaturation, adding badges, etc.).

[0123] In operation 808, the streaming management engine sends instructions to the broadcaster regarding the generated digital assets. In some embodiments, the streaming management engine may prompt the broadcaster to approve the digital assets before providing them to viewers of the live media stream. For example, the broadcaster may be asked to confirm providing the generated digital assets to selected viewers of the live media stream.

[0124] In operation 810, the streaming management engine receives approval for the generated digital asset from the broadcaster, and in response, in operation 812, the streaming management engine provides the digital asset to at least one viewer of the live media stream.

[0125] Now for reference Figure 9 The diagram illustrates, in flowchart form, an example method 900 for generating editable media objects based on data associated with multiple events of interest in a real-time media stream. Method 900 can be implemented by a computing system (e.g., [missing information]) that performs media stream processing. Figure 1A The streaming media management engine 310) executes the operation. The operation of method 900 can be performed as a supplement to or alternative to one or more operations of methods 700 and 800.

[0126] In Operation 902, the streaming media management engine identifies multiple events of interest (ROIs) in the live media stream based on viewer response data. These ROIs could be, for example, those events in the live media stream associated with an acceleration of positive responses exceeding a defined threshold level for the stream. Such ROIs can be considered highlights of the live media stream. Each of these ROIs can be associated with a different time period during the live media stream (e.g., start time, end time, or time window).

[0127] In operation 904, for each identified event of interest, the streaming media management engine obtains a segment of audio and / or video data from the real-time media stream associated with the time of the event of interest. For example, one or more video frames, audio files, and / or video clips associated with the event of interest may be obtained.

[0128] In operation 906, the streaming media management engine automatically generates an editable media object based on segments of combined audio and / or video data. In some embodiments, the streaming media management engine can stitch together captured segments associated with an event of interest to generate a single video-on-demand (e.g., a "highlight" video). For example, segments can be stitched together chronologically based on their associated times. Then, in operation 908, the editable media object can be provided to the broadcaster.

[0129] Now for reference Figure 10The diagram illustrates, in flowchart form, an example method 1000 for providing modified media data for a real-time media stream based on the detection of a triggering action initiated by a broadcaster. Method 1000 can be implemented by a computing system (e.g., [missing information]) that performs media stream processing. Figure 1A The streaming media management engine (310) is executed.

[0130] In operation 1002, the streaming media management engine receives media data (e.g., audio, video, etc.) from the real-time media stream. In operation 1004, the streaming media management engine detects triggers associated with the media data of the real-time media stream. Various triggers initiated by the broadcaster can be associated with automated actions that can be performed in relation to the real-time media stream.

[0131] In at least some embodiments, the trigger for detecting media data may include detecting one or more defined keywords in the audio data of the live media stream. The streaming management engine may first determine the one or more defined keywords to be detected in the audio data processed by the streaming management engine. In some embodiments, the streaming management engine may retrieve broadcaster-specified keyword data identifying one or more defined keywords from a storage associated with an e-commerce platform.

[0132] Alternatively, the streaming management engine can determine the defined keywords based on previous media streams involving the broadcaster. Specifically, the streaming management engine can detect an acceleration greater than a threshold in the viewer's reaction during the display of the previous media stream. The streaming management engine identifies the trigger time within the media stream associated with the acceleration of the viewer's reaction greater than the threshold, and identifies the utterance of one or more defined keywords at the trigger time.

[0133] In some embodiments, detecting triggers associated with media data may include detecting defined gestures in the video data of a live media stream. For example, a streaming management engine may perform real-time gesture recognition (e.g., gestures) to identify one of a set of gestures defined by the broadcaster as being associated with automated actions related to the live media stream.

[0134] In response to the detection of a trigger, in operation 1006, the streaming media management engine generates at least one of audio overlay content or video overlay content associated with the trigger. In some embodiments, the overlay content may include one or more recommendations of one or more digital assets that can be sent along with the live video stream. For example, the digital asset may be a graphical representation of a digital coupon, ticket, etc., for accessing a discount or special offer associated with a specific product. In operation 1008, the streaming media management engine sends at least one of the audio overlay content or video overlay content along with the live media stream to one or more viewer devices.

[0135] In some embodiments, detecting triggers associated with media data may include detecting product-specific content in a live media stream. For example, if a stream is associated with a product tag (e.g., a hash tag) corresponding to a specific product, the streaming management engine may determine that the stream's content is related to that specific product. As another example, the streaming management engine may determine that the stream's content is related to that product when a specific product is detected in the video data of the live media stream.

[0136] Now for reference Figure 11 The diagram illustrates, in flowchart form, an example method 1100 for providing digital asset recommendations along with live media streaming. Method 1100 can be implemented by a computing system that performs media streaming processing (e.g., Figure 1A The streaming media management engine 310) executes the operation. The operation of method 1100 can be executed as a supplement to or alternative to one or more operations of method 1000.

[0137] In operation 1102, the streaming management engine detects one or more keywords in the audio data of the live media stream. For example, the streaming management engine can analyze a broadcaster's speech to determine whether the broadcaster uttered one or more defined keywords in the live media stream.

[0138] In operation 1104, the streaming media management engine identifies at least one product associated with one or more detected keywords. The product can be identified based on, for example, a mapping defined by keywords specified by the broadcaster to a set of goods expected to be provided to selected viewers of the live media stream.

[0139] In operation 1106, the streaming management engine generates a digital asset recommendation associated with the at least one product. Before distributing the recommendation to one or more viewers of the live media stream, it may be provided to the broadcaster for consideration and approval. In operation 1108, the streaming management engine receives approval for the digital asset recommendation from the broadcaster. Then, in operation 1110, the digital asset recommendation may be sent along with the live media stream to the viewer's device.

[0140] Now for reference Figure 12 The diagram illustrates, in flowchart form, an example method 1200 for providing broadcasters with suggested actions regarding live media streams. Method 1200 can be implemented by a computing system that performs media stream processing (e.g., Figure 1A The streaming media management engine 310) executes the operation. The operation of method 1200 can be performed as a supplement to or alternative to one or more operations of methods 1000 and 1100.

[0141] In operation 1202, the streaming media management engine detects events of interest associated with the live media stream. In some embodiments, events of interest can be detected based on audience reaction data associated with the live media stream. For example, an event of interest could be an in-stream event that occurs immediately prior to the detection of an acceleration in viewer engagement.

[0142] In operation 1204, the streaming media management engine determines suggested action prompts associated with the identified event of interest. Action prompts represent guidance provided to the broadcaster during live media streaming to confirm one or more automated actions related to the live media stream. Action prompts may be, for example, keywords associated with the event of interest, the live media stream, and / or one or more automated actions.

[0143] In operation 1206, the streaming media management engine provides broadcasters with indications of identified events of interest, along with suggested action prompts associated with those events. This indication can be provided, for example, via the graphical user interface (or another application, user interface, etc.) of the media streaming application the broadcaster uses for their live media stream.

[0144] In operation 1208, the streaming media management engine detects at least one triggering action associated with a suggested action cue in the audio and / or video data of the live media stream. For example, the streaming media management engine may detect at least one suggested keyword associated with an identified event of interest in the audio data of the live media stream. For example, suggested keywords may be identified from the broadcaster's speech in the audio data.

[0145] In operation 1210, the streaming media management engine generates overlay content for the live media stream. This overlay content can correspond to an automated action that uses the detection of a triggering action as confirmation. For example, the overlay content can be a graphical representation of a digital asset associated with a product offered for sale to one or more viewers of the live media stream.

[0146] Example e-commerce platform

[0147] Although not required, in some embodiments, the methods disclosed herein may be performed on or in connection with an e-commerce platform. An example of an e-commerce platform will now be described.

[0148] Figure 2 An e-commerce platform 100 according to one embodiment is shown. The e-commerce platform 100 may be a reference. Figure 1BAn example of an e-commerce platform 105 is described. E-commerce platform 100 can be used to provide merchants' products and services to customers. While this disclosure envisions the use of devices, systems, and processes to purchase products and services, for simplicity, the description herein will refer to products. All references to products in this disclosure should also be understood as references to products and / or services, including, for example, physical products, digital content (e.g., music, video, games), software, tickets, subscriptions, services to be offered, etc.

[0149] While this disclosure assumes throughout that 'merchants' and 'customers' may refer to more than just individuals, for simplicity, the descriptions herein generally refer to the merchants and customers themselves. All references to merchants and customers in this disclosure should also be understood as references to individuals, groups, companies, corporations, computing entities, etc., and may represent for-profit or non-profit exchanges of products. Furthermore, while this disclosure consistently refers to "merchants" and "customers" and describes their roles, the e-commerce platform 100 should be understood more generally as supporting users within an e-commerce environment. References to merchants and customers in this disclosure should also be understood as references to users, such as merchant users (e.g., sellers, retailers, wholesalers, or product providers), customer users (e.g., buyers, purchasing agents, consumers, or product users), potential users (e.g., users browsing but not yet committed to a purchase, users evaluating the e-commerce platform 100 for potential marketing and sales of products), service provider users (e.g., transportation providers 112, financial providers, etc.), corporate or enterprise users (e.g., company representatives purchasing, selling, or using products; enterprise users; customer relations or customer management agents, etc.), information technology users, and computing entity users (e.g., computing robots used for purchasing, selling, or using products). Furthermore, it can be recognized that while in one context a given user can play a given role (e.g., as a merchant) and their associated device can be mentioned accordingly (e.g., as a merchant device), in another context the same person can play different roles (e.g., as a customer) and the same or another associated device can be mentioned accordingly (e.g., as a customer device). For example, a person can be a merchant of one type of product (e.g., shoes) and a customer / consumer of another type of product (e.g., groceries). In another example, a person can be both a consumer and a merchant of the same type of product. In a particular example, a merchant engaged in the trading of a particular category of goods can act as a customer of the same category of goods when placing an order with a wholesaler (wholesaler acting as merchant).

[0150] E-commerce platform 100 provides merchants with online services / facilities to manage their businesses. The facilities described herein are shown as part of platform 100, but may also be configured, wholly or partially, to be separate from platform 100 as independent services. Furthermore, in some embodiments, such facilities are additionally or alternatively provided by one or more providers / entities.

[0151] exist Figure 2 In the example, the facility is deployed via a machine, service, or engine that executes computer software, modules, program code, and / or instructions on one or more processors, which may be part of platform 100 or external to the platform as described above. Merchants may use e-commerce platform 100 to implement or manage business with customers, such as through online stores 138, applications 142A-B, channels 110A-B, and / or through point-of-sale (POS) devices 152 at physical locations (e.g., physical storefronts or other locations, such as through self-service terminals, terminals, readers, printers, 3D printers, etc.). Merchants may use e-commerce platform 100 as their sole business presence with customers, or in conjunction with other merchant business facilities, such as through physical stores (e.g., brick-and-mortar retail stores), merchant-external websites 104 (e.g., commercial internet websites or other internet or network properties or assets separate from e-commerce platform 100 and supported or represented by the merchant), applications 142B, etc. However, even these 'other' merchant facilities can be integrated with or communicate with the e-commerce platform 100. For example, a POS device 152 in a merchant's physical store can be linked to the e-commerce platform 100, or a merchant's external website 104 can be linked to the e-commerce platform 100, for example, by linking the content of the merchant's external website 104 to the 'buy button' of the online store 138.

[0152] Online store 138 can represent a multi-tenant facility comprising multiple virtual storefronts. In an embodiment, a merchant can configure and / or manage one or more storefronts in online store 138, for example, via merchant device 102 (e.g., computer, laptop, mobile computing device, etc.), and offer products to customers through various channels 110A-B (e.g., online store 138; applications 142A-B; physical storefronts, via POS device 152; e-marketplaces, such as via e-purchase buttons integrated into websites or social media channels, such as on social networks, social media pages, social media messaging systems). Merchants can sell across channels 110A-B and then manage their sales through e-commerce platform 100, where channel 110A can be offered as a facility or service internal or external to e-commerce platform 100. Alternatively or concurrently, merchants can sell in their physical retail stores, at pop-up stores, through wholesale, by telephone, etc., and then manage their sales through e-commerce platform 100. Merchants can employ all or any combination of these operating modes. It is worth noting that, by employing multiple and / or specific combinations of these models, merchants may increase the probability and / or volume of sales. In this disclosure, the terms “online store” and “storefront” may be used synonymously to refer to the online e-commerce services offered by a merchant through e-commerce platform 100, wherein online store 138 may refer to a collection of storefronts supported by e-commerce platform 100 (e.g., for one or more merchants), or to a single merchant’s storefront (e.g., the merchant’s online store).

[0153] In some embodiments, customers may interact with platform 100 via customer device 150 (e.g., computer, laptop computer, mobile computing device, etc.), POS device 152 (e.g., retail equipment, self-service terminal, automated (self-service) checkout system, etc.) and / or any other business interface device known in the art. E-commerce platform 100 enables merchants to contact customers via online store 138, via applications 142A-B, via POS device 152 at a physical location (e.g., a merchant's storefront or other location), via electronic communication facility 129, etc., to provide a system for contacting customers and facilitating merchant services for real or virtual paths available for contacting and interacting with customers.

[0154] In some embodiments, and as further described herein, the e-commerce platform 100 may be implemented via a processing facility. Such a processing facility may include a processor and memory. The processor may be a hardware processor. The memory may be and / or may include transient memory such as random access memory (RAM), and / or non-transitory memory such as non-transitory computer-readable media (e.g., persistent storage devices, such as magnetic storage devices). The processing facility may (e.g., in memory) store a set of instructions that, when executed, cause the e-commerce platform 100 to perform the e-commerce functions and supporting functions described herein. The processing facility may be one or more of, or a part of, a server, client, network infrastructure, mobile computing platform, cloud computing platform, fixed computing platform, and / or other computing platform, and may provide electronic connectivity and communication between components of the e-commerce platform 100, merchant device 102, payment gateway 106, applications 142A-B, channels 110A-B, transportation provider 112, customer device 150, POS device 152, etc. In some embodiments, the processing facility may be or may include one or more such computing devices working collaboratively. For example, multiple collaborative computing devices may act as / provide the processing facility. E-commerce platform 100 can be implemented as or used in the following ways: cloud computing services, Software as a Service (SaaS), Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Desktop as a Service (DaaS), Managed Software as a Service (MSaaS), Mobile Backend as a Service (MBaaS), Information Technology Management as a Service (ITMaaS), etc. For example, the underlying software implementing the facility described herein (e.g., online store 138) can be provided as a service and centrally hosted (e.g., then accessed by users via a web browser or other application, and / or via client device 150, POS device 152, etc.). In some embodiments, elements of e-commerce platform 100 can be implemented to operate and / or integrate with various other platforms and operating systems.

[0155] In some embodiments, the facilities of e-commerce platform 100 (e.g., online store 138) may provide content to client device 150, for example, via network 420 (using data 134) connected to e-commerce platform 100. For instance, online store 138 may provide or send content in response to a request for data 134 from client device 150, wherein a browser (or other application) connects to online store 138 via network 420 using a network communication protocol (e.g., Internet Protocol). The content may be written in a machine-readable language and may include Hypertext Markup Language (HTML), template languages, JavaScript, and / or any combination thereof.

[0156] In some embodiments, online store 138 may be or may include instances of services that provide content to client devices and allow clients to browse and purchase various available products (e.g., adding products to a cart, purchasing via a buy button, etc.). Merchants can also customize the look and feel of their website through a theme system, where they can choose and change the appearance and feel of their online store 138 by displaying the same underlying product and business data within the product information of the online store. It is possible that the theme can be further customized through a theme editor (i.e., a design interface that allows users to flexibly customize the design of their website). Alternatively or alternatively, the theme may be customized using theme-specific settings that can change aspects of a given theme (e.g., specific colors, fonts, and pre-built layout schemes). In some implementations, the online store may implement a content management system for website content. Merchants may employ such a content management system when creating blog posts or static pages and publishing them to their online store 138 (e.g., via blogs, articles, login pages, etc.) and configuring navigation menus. Merchants can upload images (e.g., products), videos, content, data, etc., to e-commerce platform 100, for example, for system storage (e.g., storage as data 134). In some embodiments, e-commerce platform 100 can provide functions for manipulating such images and content, such as functions for resizing images, associating images with products, adding text and associating text with images, adding images for new product variations, protecting images, etc.

[0157] As described herein, e-commerce platform 100 can provide product sales and marketing services to merchants through various channels 110A-B, including, for example, online stores 138, applications 142A-B, and physical POS devices 152. E-commerce platform 100 may additionally or alternatively include business support services 116, administrators 114, warehouse management systems, etc., associated with operating an online business (e.g., providing domain registration services 118 associated with their online stores, payment services 120 to facilitate transactions with customers, shipping services 122 to provide customers with shipping options for purchased products, fulfillment services for managing inventory, risk and insurance services 124 associated with product protection and liability, merchant invoices, etc.). Service 116 may be provided via e-commerce platform 100 or in connection with external facilities, such as payment gateways 106 for payment processing, shipping providers 112 for expediting product transportation, etc.

[0158] In some embodiments, the e-commerce platform 100 may be configured with a transportation service 122 (e.g., through the e-commerce platform's transportation facilities or through a third-party carrier) to provide merchants and / or their customers with various transportation-related information, such as shipping tags or freight information, real-time delivery updates, tracking, etc.

[0159] Figure 3 A non-limiting embodiment of the homepage of administrator 114 is depicted. Administrator 114 may be referred to as a management console and / or administrator console. Administrator 114 may display information about daily tasks, recent store activity, and subsequent steps that the merchant can take to build their business. In some embodiments, the merchant may log in to administrator 114 via merchant device 102 (e.g., a desktop computer or mobile device) and manage various aspects of their online store 138, such as viewing recent visits or order activity of online store 138, updating the online store 138 catalog, managing orders, etc. In some embodiments, the merchant may be able to use a sidebar (e.g., Figure 3 The sidebar shown provides access to different sections of Administrator 114. These sections may include various interfaces for accessing and managing core aspects of the merchant's business, including orders, products, customers, available reports, and discounts. Administrator 114 may additionally or alternatively include interfaces for managing the store's sales channels, including online store 138, multiple mobile applications (mobile apps) that customers can use to access the store, POS devices, and / or purchase buttons. Administrator 114 may additionally or alternatively include interfaces for managing applications (apps) installed on the merchant's account; and settings applied to the merchant's online store 138 and account. Merchants can use the search bar to find products, pages, or other information in their store.

[0160] More detailed business and visitor information for a merchant's online store 138 can be viewed through reports or metrics. Reports may include, for example, traffic reports, behavior reports, customer reports, financial reports, marketing reports, sales reports, product reports, and customized reports. Merchants may be able to view sales data from different channels 110A-B over different time periods (e.g., days, weeks, months, etc.), for example, by using drop-down menus. Data overviews can also be provided for merchants who want to view more detailed sales and engagement data for their store. Activity feeds can be provided in the metrics section of the homepage to show an overview of activity on a merchant's account. For example, by clicking the "View all recent activity" data panel button, a merchant may be able to see a longer period of recent activity feeds on their account. The homepage can display notifications about a merchant's online store 138 based on, for example, account status, growth, recent customer activity, order updates, etc. Notifications can be provided to help merchants navigate through workflows configured for online store 138, such as payment workflows, order fulfillment workflows, order archiving workflows, return workflows, etc.

[0161] E-commerce platform 100 can provide communication facilities 129 and associated merchant interfaces to provide electronic communication and marketing, such as using electronic messaging facilities to collect and analyze communication interactions between merchants, customers, merchant devices 102, customer devices 150, POS devices 152, etc., to aggregate and analyze communications, thereby improving sales conversion rates. For example, a customer may have product-related questions, which may lead to a conversation between the customer and the merchant (or an automation processor-based agent / chatbot representing the merchant). In this case, communication facilities 129 are configured to request an automated response from the customer and / or provide the merchant with suggestions on how to respond, for example, to increase the probability of a sale.

[0162] E-commerce platform 100 can provide financial facilities for secure financial transactions with customers, such as through a secure card server environment. For example, in a Payment Card Industry Data (PCI) environment (e.g., a card server), e-commerce platform 100 can store credit card information for purposes such as financial auditing, invoicing merchants, and executing Automated Clearing House (ACH) transfers between e-commerce platform 100 and merchants' bank accounts. Financial facilities can also provide financial support to merchants and buyers, such as through lending (e.g., loans, cash advances, etc.) and providing insurance. In some embodiments, online store 138 can support multiple independently managed storefronts and process large volumes of transaction data daily for various products and services. Transaction data may include: any customer information (e.g., contact information, billing information, shipping information, return / refund information, discount / offer information, payment information) or online store events or information (e.g., page views, product search information (search keywords, click events), product reviews, abandoned purchases) instructing customers, customer accounts, or transactions made by customers, and other transaction information related to business conducted through e-commerce platform 100. In some embodiments, the e-commerce platform 100 may store the data in data facility 134. (See again...) Figure 2 In some embodiments, the e-commerce platform 100 may include a business management engine 136, which may be configured to execute various workflows for tasks related to product, inventory, customers, orders, suppliers, reporting, finance, risk, and fraud, or for content management. In some embodiments, additional functionality may be provided, alternatively, through applications 142A-B to provide greater flexibility and customization to accommodate the growing variety of online stores, POS devices, products, and / or services. Application 142A may be a component of the e-commerce platform 100, while application 142B may be provided or hosted as a third-party service outside the e-commerce platform 100. The business management engine 136 may be adapted to store-specific workflows and, in some embodiments, may be combined with administrator 114 and / or online store 138.

[0163] Implementing the functionality as applications 142A-B enables the business management engine 136 to remain responsive and reduce or avoid service degradation or more serious infrastructure failures.

[0164] While isolating online store data may be important for maintaining data privacy between online store 138 and merchants, there may also be reasons to collect and use cross-store data, for example, for order risk assessment systems or platform payment facilities, both of which require information from multiple online stores 138 to function well. In some embodiments, it may be preferable to move these components out of the business management engine 136 and into their own infrastructure within the e-commerce platform 100.

[0165] A platform payment facility is an example of a component that utilizes data from the business management engine 136 but is implemented as a separate component or service. The platform payment facility allows customers interacting with online stores 138 to have their payment information securely stored by the business management engine 136, enabling customers to enter their payment information only once. When customers visit different online stores 138, even if they have never been there before, the platform payment facility can retrieve their information to achieve faster and / or potentially less error-prone checkout (e.g., by avoiding situations where customers might mistype their information if they need to re-enter it). This can provide a cross-platform network effect, in which the e-commerce platform 100 becomes more useful to its merchants and buyers as more merchants and buyers join, for example, because more customers check out more frequently due to ease of use in customer purchasing. To maximize the effect of this network, a given customer's payment information can be available and globally accessible across multiple online stores 138.

[0166] For functionality not included in the business management engine 136, applications 142A-B provide a way to add features to the e-commerce platform 100 or individual online stores 138. For example, applications 142A-B may be able to access and modify data on a merchant's online store 138, perform tasks through an administrator 114, implement new flows for the merchant through a user interface (which is rendered, for example, via an extension / API), etc. Merchants may be able to discover and install applications 142A-B through application search, recommendation, and support 128. In some embodiments, the business management engine 136, applications 142A-B, and administrator 114 may be developed to work together. For example, application extension points may be built within the business management engine 136, which applications 142A and 142B can access through interfaces 140B and 140A to provide additional functionality, and these application extension points can be presented to merchants in the user interface of administrator 114.

[0167] In some embodiments, applications 142A-B can provide functionality to merchants through interfaces 140A-B, such as applications 142A-B being able to present transaction data to merchants (e.g., App: "Engineer, present my app data in the mobile app or admin 114"), and / or the business management engine 136 being able to request applications to perform tasks as needed (Engineer: "App, give me the local tax calculation for this checkout").

[0168] Applications 142A-B can connect to the Business Management Engine 136 via interfaces 140A-B (e.g., via REST (Representative State Transfer) and / or GraphQL APIs) to expose functionality and / or data available within the Business Management Engine 136 to the applications. For example, the e-commerce platform 100 can provide interfaces 140A-B for the APIs of applications 142A-B, allowing these applications to connect to products and services outside of platform 100. The flexibility provided by the use of applications and APIs (e.g., for application development) enables the e-commerce platform 100 to better adapt to new and unique needs of merchants or address specific use cases without constantly modifying the Business Management Engine 136. For example, transportation service 122 can integrate with the Business Management Engine 136 via a transportation or carrier services API, allowing the e-commerce platform 100 to provide transportation service functionality without directly affecting the code running in the Business Management Engine 136.

[0169] Depending on the implementation, applications 142A-B may use APIs to fetch data on demand (e.g., customer creation events, product change events, or order cancellation events) or push data as updates occur. A subscription model may be used to provide events to applications 142A-B when they occur, or to provide updates on the changed state of the business management engine 136. In some embodiments, when a change related to an update event subscription occurs, the business management engine 136 may publish a request, such as to a predefined callback URL. The body of this request may contain a description of the new state of the object and an action or event. Update event subscriptions may be created manually in the administrator facility or automatically (e.g., via API 140A-B). In some embodiments, update events may be queued and processed asynchronously with the state change that triggered the update event, which may result in update event notifications that are not distributed in real-time or near real-time.

[0170] In some embodiments, the e-commerce platform 100 may provide one or more of application search, recommendation, and support 128. Application search, recommendation, and support 128 may include: developer products and tools to assist in application development; application data dashboards (e.g., providing developers with a development interface, administrators with application management, merchants with application customization, etc.); facilities for installation and providing access to applications 142A-B (e.g., for public access, such as where standards must be met before installation, or for private use by merchants); application search to make it easy for merchants to search for applications 142A-B that meet the needs of their online store 138; application recommendations to provide merchants with suggestions on how they can improve the user experience through their online store 138; and so on. In some embodiments, applications 142A-B may be assigned application identifiers (IDs), such as for linking to applications (e.g., via API), searching for applications, making application recommendations, etc.

[0171] Applications 142A-B can be broadly grouped into three categories: customer-facing applications, merchant-facing applications, and integrated applications. Customer-facing applications 142A-B can include online stores 138 or channels 110A-B, which are places where merchants can list products for purchase (e.g., online stores, applications for flash sales (e.g., merchant products or opportunistic sales opportunities from third-party sources), mobile store applications, social media channels, applications for offering wholesale purchases, etc.). Merchant-facing applications 142A-B can include applications that allow merchants to manage their online stores 138 (e.g., through applications related to the web or website or mobile devices), operate their businesses (e.g., through applications related to POS devices), and develop their businesses (e.g., through applications related to transportation (e.g., drop shipping), using automated agents, developing and improving using process flows, etc.). Integrated applications can include applications that provide useful integrations for participating in business operations, such as transportation providers 112 and payment gateways 106.

[0172] In this way, e-commerce platform 100 can be configured to provide an online shopping experience through a flexible system architecture that enables merchants to connect with customers in a flexible and transparent manner. The example purchase workflow of this embodiment provides a better understanding of the typical customer experience. In this example purchase workflow, a customer browses a merchant's products on channels 110A-B, adds the products they intend to buy to their shopping cart, proceeds to checkout, and pays for the contents of the shopping cart, thus creating an order for the merchant. The merchant can then review and fulfill (or cancel) the order. The products are then shipped to the customer. If the customer is not satisfied, they may return the products to the merchant.

[0173] In the example embodiment, customers can browse a merchant's products through multiple different channels 110A-B (e.g., the merchant's online store 138; physical stores, via POS devices 152; e-marketplaces, via e-purchase buttons integrated into a website or social media channel). In some cases, channels 110A-B can be modeled as applications 142A-B. The showcase component in the business management engine 136 can be configured to create and manage product listings (e.g., using product data objects or models) to allow merchants to describe what they want to sell and where they sell it. The association between the product listing and the channels can be modeled as product postings and accessed via channel applications (e.g., via a product listing API). Products can have numerous attributes and / or characteristics (such as size and color) and numerous variations that expand the available options to specific combinations of all attributes, such as a green variation for an extra-small size or a blue variation for a large size. Products may have at least one variation created for products without any options (e.g., a "default variation"). To facilitate browsing and management, products can be grouped into sets, and products can be given product identifiers (e.g., item numbers (SKUs)). A collection of products can be constructed by manually categorizing products into one type (e.g., a custom collection), by building a set of rules for automatic categorization (e.g., a smart collection), or by other means. The product list can include 2D images, 3D images, or models that can be viewed through virtual reality or augmented reality interfaces.

[0174] In some embodiments, a shopping cart object is used to store or track products that a customer intends to purchase. A shopping cart object can be channel-specific and can consist of multiple cart order items, each tracking the quantity of a specific product variant. Since adding a product to a cart does not imply any commitment from the customer or merchant, and the expected lifespan of a cart may be on the order of minutes (rather than days), the shopping cart object / data representing the cart can be persisted to a temporary data store.

[0175] The customer then proceeds to checkout. The checkout object or page generated by the commerce management engine 136 can be configured to receive customer information to complete the order, such as customer contact information, billing information, and / or shipping details. If the customer enters their contact information but does not make a payment, the e-commerce platform 100 can (e.g., via an order cancellation component) send a message to the customer's device 150 to encourage the customer to complete the checkout. For these reasons, the checkout object may be much longer than the shopping cart object (hours or even days) and may therefore persist. The customer then pays for the contents of their shopping cart, thus creating an order for the merchant. In some embodiments, the commerce management engine 136 can be configured to communicate with various payment gateways and services (e.g., online payment systems, mobile payment systems, digital wallets, credit card gateways) via a payment processing component. Actual interaction with the payment gateway 106 can be provided through a card server environment. An order is created at the end of the checkout process. An order is a sales contract between the merchant and the customer, in which the merchant agrees to provide the goods and services listed on the order (e.g., order items, shipping items, etc.) and the customer agrees to provide payment (including tax). Once an order is created, an order confirmation notification can be sent to the customer, and an order placement notification can be sent to the merchant via the notification component. Inventory can be reserved to prevent overselling when payment processing begins (e.g., merchants can control this behavior using inventory strategies or configurations for each variant). Inventory reservations may have a short time span (a few minutes) and may need to be very fast and scalable to support flash sales or "drops," during which discounted, promotional, or limited stock of products is offered to buyers at specific locations and / or for a specific (usually short) time. If payment fails, the reservation is cancelled. After successful payment and order creation, the reservation is converted into a persistent (long-term) inventory commitment allocated to a specific location. The inventory component of the Business Management Engine 136 can record the storage location of variants and track the quantity of variants with inventory tracking enabled. This component can separate product variants (a customer-facing concept representing product listing templates) from inventory items (a merchant-facing concept representing items whose quantity and location are managed). The inventory level component can track the quantity available for sale, for order commitments, or for inflows from the inventory transfer component (e.g., suppliers).

[0176] Merchants can then review and fulfill (or cancel) orders. The review component of the Business Management Engine 136 implements the business process that merchants use to ensure an order is suitable for fulfillment before actually fulfilling it. Orders may be fraudulent, may require verification (e.g., ID checks), or may have payment methods that require merchants to wait to ensure they will receive their payments. Risks and recommendations can be continuously present in the order risk model. Order risks can be generated by fraud detection tools, submitted by third parties via an order risk API, etc. Before fulfillment, merchants may need to capture payment information (e.g., credit card information) or wait to receive payment information (e.g., via bank transfer, check, etc.) before marking the order as paid. Now, merchants can prepare the products to be delivered. In some embodiments, this business process can be implemented by the fulfillment component of the Business Management Engine 136. The fulfillment component can group order items of an order into logical fulfillment work units based on inventory location and fulfillment services. Merchants can review and adjust work units and trigger related fulfillment services, such as manual fulfillment services used when merchants pick products and pack them into boxes, purchase shipping labels and enter their tracking numbers (e.g., in merchant-managed locations), or simply mark items as fulfilled. Alternatively, API fulfillment services can trigger third-party applications or services to create fulfillment records for third-party fulfillment services. Other possibilities exist for fulfilling orders. If customers are dissatisfied, they can return (multiple) products to the merchant. The business process of merchants “canceling sales” can be implemented through a returns component. Returns can include various actions such as: restocking, where the sold products are actually returned to the business and can be resold; refunds, partially or fully refunding money received from customers; accounting adjustments, recording the refund amount (e.g., including whether there were any restocking costs, or whether the goods were not returned and remained with the customer); etc. Returns can represent a change to the sales contract (e.g., an order), and in this case, the e-commerce platform 100 can make merchants aware of compliance issues regarding legal obligations (e.g., taxes). In some embodiments, e-commerce platform 100 enables merchants to track sales contracts over time, for example through sales model components (e.g., a date-based append-only ledger that records sales-related events occurring on goods).

[0177] Implementation

[0178] The methods and systems described herein can be deployed, in part or in whole, by a machine that executes computer software, program code, and / or instructions on a processor. The processor can be part of a server, cloud server, client, network infrastructure, mobile computing platform, fixed computing platform, or other computing platform. The processor can be any kind of computing or processing device capable of executing program instructions, code, binary instructions, etc. The processor can be or include signal processors, digital processors, embedded processors, microprocessors, or any variant such as a coprocessor (mathematical coprocessor, graphics coprocessor, communication coprocessor, etc.) that can directly or indirectly facilitate the execution of program code or program instructions stored thereon. Furthermore, the processor can implement the execution of multiple programs, threads, and code. Threads can execute concurrently to enhance processor performance and facilitate simultaneous operation of applications. By implementation, the methods, program code, program instructions, etc., described herein can be implemented in one or more threads. A thread can cause other threads that may have been assigned an associated priority; the processor can execute these threads based on priority or based on any other order according to the instructions provided in the program code. The processor may include memory storing the methods, code, instructions, and programs as described herein and elsewhere. The processor can access, through an interface, a storage medium that can store methods, code, and instructions as described herein and elsewhere. The storage medium associated with the processor for storing methods, programs, code, program instructions, or other types of instructions executable by a computing or processing device may include, but is not limited to, one or more of CD-ROM, DVD, memory, hard disk, flash drive, RAM, ROM, cache, etc.

[0179] A processor may include one or more cores that can improve the speed and performance of a multiprocessor. In some embodiments, the process may be a dual-core processor, a quad-core processor, or other chip-level multiprocessors that combine two or more independent cores (called dies).

[0180] The methods and systems described herein can be deployed, in part or in whole, on a machine that executes computer software on a server, cloud server, client, firewall, gateway, hub, router, or other computer and / or networking hardware. The software program can be associated with a server, which may include a file server, print server, domain server, internet server, intranet server, and other variations such as a secondary server, primary server, distributed server, etc. The server may include one or more of the following: memory, processor, computer-readable medium, storage medium, ports (physical and virtual), communication devices, and interfaces capable of accessing other servers, clients, machines, and devices via wired or wireless media. The methods, programs, or code described herein and elsewhere can be executed by a server. Additionally, other devices required to perform the methods described in this application can be considered part of the infrastructure associated with the server.

[0181] The server can provide interfaces to other devices, including but not limited to clients, other servers, printers, database servers, print servers, file servers, communication servers, distributed servers, etc. Furthermore, this coupling and / or connection can facilitate remote execution of programs across a network. Without departing from the scope of this disclosure, networking some or all of these devices can facilitate parallel processing of programs or methods at one or more locations. Additionally, any device connected to the server via an interface can include at least one storage medium capable of storing methods, programs, code, and / or instructions. A central repository can provide program instructions to be executed on different devices. In this implementation, a remote repository can act as a storage medium for program code, instructions, and programs.

[0182] The software program can be associated with a client, which may include file clients, print clients, domain clients, internet clients, intranet clients, and other variations such as secondary clients, host clients, distributed clients, etc. The client may include one or more of the following: memory, processor, computer-readable medium, storage medium, port (physical and virtual), communication device, and interface capable of accessing other clients, servers, machines, and devices via wired or wireless media. The methods, programs, or code described herein and elsewhere can be executed by the client. Additionally, other devices required to perform the methods described in this application can be considered part of the infrastructure associated with the client.

[0183] The client can provide interfaces to other devices, including but not limited to servers, other clients, printers, database servers, print servers, file servers, communication servers, distributed servers, etc. Furthermore, this coupling and / or connection can facilitate remote execution of programs across a network. Without departing from the scope of this disclosure, networking some or all of these devices can facilitate parallel processing of programs or methods at one or more locations. Additionally, any device connected to the client via an interface can include at least one storage medium capable of storing methods, programs, applications, code, and / or instructions. A central repository can provide program instructions to be executed on different devices. In this implementation, a remote repository can act as a storage medium for program code, instructions, and programs.

[0184] The methods and systems described herein can be deployed, in whole or in part, through a network infrastructure. The network infrastructure may include components such as computing devices, servers, routers, hubs, firewalls, clients, personal computers, communication devices, routing devices, and other active and passive devices, modules, and / or components known in the art. Among other components, the computing and / or non-computing devices associated with the network infrastructure may include storage media such as flash memory, buffers, stacks, RAM, and ROM. The processes, methods, program code, and instructions described herein and elsewhere can be executed by one or more network infrastructure components.

[0185] The methods, program code, and instructions described herein and elsewhere can be implemented in various devices that can operate in wired or wireless networks. Examples of wireless networks include fourth-generation (4G) networks (e.g., Long Term Evolution (LTE)) or fifth-generation (5G) networks, as well as non-cellular networks such as wireless local area networks (WLANs). However, the principles described herein are equally applicable to other types of networks.

[0186] The operations, methods, program code, and instructions described herein and elsewhere can be implemented on or through mobile devices. Mobile devices may include navigation devices, cellular phones, mobile phones, mobile personal digital assistants, laptops, handheld computers, netbooks, pagers, e-book readers, music players, etc. Among other components, these devices may include storage media such as flash memory, buffers, RAM, ROM, etc., and one or more computing devices. The computing devices associated with the mobile device can be enabled to execute program code, methods, and instructions stored thereon. Alternatively, the mobile device may be configured to cooperate with other devices to execute instructions. The mobile device may communicate with a base station that interfaces with a server and is configured to execute program code. The mobile device may communicate on peer-to-peer networks, mesh networks, or other communication networks. Program code may be stored on storage media associated with the server and executed by a computing device embedded within the server. A base station may include computing devices and storage media. The storage device may store program code and instructions executed by the computing device associated with the base station.

[0187] Computer software, program code, and / or instructions can be stored and / or accessed on machine-readable media, which may include: computer components, devices, and recording media that retain digital data used for computation over a period of time; semiconductor storage devices known as random access memory (RAM); mass storage devices typically used for more persistent storage, such as optical discs, magnetic storage forms (e.g., hard disks, magnetic tapes, magnetic drums, cards, and other types); processor registers, cache memory, volatile memory, and non-volatile memory; optical storage devices such as CDs and DVDs; removable media such as flash memory (e.g., USB flash drives or keys), floppy disks, magnetic tapes, paper tapes, punched cards, stand-alone RAM disks, Zip drives, removable mass storage devices, offline storage, etc.; and other computer memories such as dynamic memory, static memory, read / write storage devices, variable storage devices, read-only memory, random access memory, sequential access memory, location-addressable memory, file-addressable memory, content-addressable memory, network-attached storage devices, storage area networks, barcodes, magnetic ink, etc.

[0188] The methods and systems described in this paper can transform entities and / or intangible things from one state to another. The methods and systems described in this paper can also transform data representing entities and / or intangible things from one state to another, such as transforming usage data into a standardized usage dataset.

[0189] The elements described and depicted herein, including those in the flowcharts and block diagrams in all the accompanying drawings, imply logical boundaries between elements. However, in accordance with software or hardware engineering practice, the depicted elements and their functions can be implemented on a machine with a processor capable of executing program instructions stored thereon as a monolithic software architecture, as a standalone software module, or as a module employing external routines, code, services, etc., or any combination thereof, and all such implementations are within the scope of this disclosure. Examples of such machines may include, but are not limited to, personal digital assistants, laptop computers, personal computers, mobile phones, other handheld computing devices, medical devices, wired or wireless communication devices, transducers, chips, calculators, satellites, tablet PCs, e-books, gadgets, electronic devices, devices with artificial intelligence, computing devices, networking devices, servers, routers, etc. Furthermore, the elements or any other logical components depicted in the flowcharts and block diagrams can be implemented on a machine capable of executing program instructions. Therefore, while the foregoing figures and descriptions illustrate various functional aspects of the disclosed system, the specific arrangement of the software used to implement these functional aspects should not be inferred from these descriptions unless explicitly stated or otherwise clearly apparent from the context. Similarly, it will be understood that the steps identified and described above are subject to change, and the order of these steps may be adapted to the specific application of the technology disclosed herein. All such changes and modifications are intended to fall within the scope of this disclosure. Thus, the depiction and / or description of the order of the steps should not be construed as requiring these steps to be performed in a particular order, unless required by the specific application, or expressly stated or otherwise clearly apparent from the context.

[0190] The methods and / or processes and their steps described above may be implemented in hardware, software, or any combination of hardware and software suitable for a particular application. Hardware may include general-purpose computers and / or special-purpose computing devices, or specific computing devices, or specific aspects or components of specific computing devices. These processes may be implemented in one or more microprocessors, microcontrollers, embedded microprocessors, programmable digital signal processors, or other programmable devices, and internal and / or external memory. The processor may additionally or alternatively be implemented in application-specific integrated circuits, programmable gate arrays, programmable array logic, or any other device or combination of devices that can be configured to process electronic signals. It will be further understood that one or more of these processes may be implemented as computer-executable code capable of executing on a machine-readable medium.

[0191] Computer executable code can be written in structured programming languages ​​such as C, object-oriented programming languages ​​such as C++, or any other high- or low-level programming languages ​​(including assembly languages, hardware description languages, and database programming languages ​​and techniques). These high- or low-level programming languages ​​can be stored, compiled, or interpreted to run on one of the above devices, as well as heterogeneous combinations of processors, processor architectures, or combinations of different hardware and software, or any other machine capable of executing program instructions.

[0192] Therefore, in one respect, each method and combination thereof described above can be implemented using computer-executable code that, when executed on one or more computing devices, performs the steps of the methods. In another respect, these methods can be implemented using a system that performs their steps and can be distributed across devices in various ways, or all functionality can be integrated into a dedicated, standalone device or other hardware. In yet another respect, means for performing the steps associated with the processes described above can include any of the aforementioned hardware and / or software. All such permutations and combinations are intended to fall within the scope of this disclosure.

[0193] This teaching can also be extended to one or more of the following numbered clauses:

[0194] 1. A computer-implemented method, comprising:

[0195] Receive video data from real-time media streams;

[0196] While streaming the live media stream, audience response data associated with the live media stream is obtained, which at least indicates the amount of audience engagement related to the video content of the live media stream;

[0197] Identifying events of interest in the real-time media stream based on determining that the rate of change in the amount of audience participation exceeds a threshold level; and

[0198] In response to the identification of the event of interest, one or more defined actions are automatically initiated.

[0199] 2. The method as described in Clause 1, wherein the audience response data includes user input obtained via a computing device associated with the viewer of the live media stream.

[0200] 3. The method as described in Clause 2, wherein the user input includes at least one of text input associated with the live media stream or selection of a defined user interface element.

[0201] 4. The method as described in Clause 2, wherein the amount of audience participation is determined based on the amount of user input.

[0202] 5. The method as described in Clause 1, wherein obtaining audience reaction data includes determining that the audience reaction data is non-negative reaction data.

[0203] 6. The method as described in Clause 1, wherein obtaining audience reaction data includes:

[0204] Receive audience feedback input;

[0205] Filtering the audience response input to exclude negative audience response input, thereby generating the audience response data; and

[0206] The amount of audience participation in the activity was determined based on the audience response data.

[0207] 7. The method as described in Clause 1, wherein automatically initiating the one or more defined actions includes providing one or more digital assets associated with the live media stream to at least one of the following:

[0208] A subset of the viewers of this live media stream; or

[0209] The stream creator associated with this live media stream.

[0210] 8. The method as described in Clause 1, wherein automatically initiating the one or more defined actions includes prompting at least a subset of the viewers of the live media stream to obtain input related to the live media stream.

[0211] 9. The method as described in Clause 1, wherein automatically initiating the one or more defined actions includes:

[0212] Generate recommendations for product discounts related to the defined products; and

[0213] The generated recommendations should be provided to at least a subset of the viewers of the live media stream.

[0214] 10. The method as described in Clause 9, wherein generating a recommendation further includes determining that the event of interest is relevant to the defined product.

[0215] 11. A computing system, comprising:

[0216] processor;

[0217] Memory that stores executable instructions for a computer, which, when executed by the processor, cause the processor to:

[0218] Receive video data from real-time media streams;

[0219] While streaming the live media stream, audience response data associated with the live media stream is obtained, which at least indicates the amount of audience engagement related to the video content of the live media stream;

[0220] Identifying events of interest in the real-time media stream based on determining that the rate of change in the amount of audience participation exceeds a threshold level; and

[0221] In response to the identification of the event of interest, one or more defined actions are automatically initiated.

[0222] 12. The computing system as described in Clause 11, wherein the audience reaction data includes user input obtained via a computing device associated with a viewer of the live media stream.

[0223] 13. The computing system as described in Clause 12, wherein the user input includes at least one of text input associated with the live media stream or selection of a defined user interface element.

[0224] 14. The computing system as described in Clause 12, wherein the amount of audience participation is determined based on the amount of user input.

[0225] 15. The computing system as described in Clause 11, wherein acquiring audience reaction data includes determining that the audience reaction data is non-negative reaction data.

[0226] 16. The computing system as described in Clause 11, wherein acquiring audience reaction data includes:

[0227] Receive audience feedback input;

[0228] Filtering the audience response input to exclude negative audience response input, thereby generating the audience response data; and

[0229] The amount of audience participation in the activity was determined based on the audience response data.

[0230] 17. The computing system as described in Clause 11, wherein automatically initiating the one or more defined actions includes providing one or more digital assets associated with the live media stream to at least one of the following:

[0231] A subset of the viewers of this live media stream; or

[0232] The stream creator associated with this live media stream.

[0233] 18. The computing system as described in Clause 11, wherein automatically initiating the actions defined therein includes: generating recommendations for product discounts related to the defined product; and

[0234] The generated recommendations should be provided to at least a subset of the viewers of the live media stream.

[0235] 19. The computing system as described in Clause 18, wherein generating a recommendation further includes determining that the event of interest is relevant to the defined product.

[0236] 20. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a processor, cause the processor to:

[0237] Receive video data from real-time media streams;

[0238] While streaming the live media stream, audience response data associated with the live media stream is obtained, which at least indicates the amount of audience engagement related to the video content of the live media stream;

[0239] Identifying events of interest in the real-time media stream based on determining that the rate of change in the amount of audience participation exceeds a threshold level; and

[0240] In response to the identification of the event of interest, one or more defined actions are automatically initiated.

Claims

1. A computer-implemented method, comprising: Receive video data from real-time media streams; While streaming the live media stream, audience response data associated with the live media stream is obtained, which at least indicates the amount of audience engagement related to the video content of the live media stream; Events of interest in the real-time media stream are identified by the rate of change of a metric that determines the amount of audience participation exceeding a threshold level; the rate of change of this metric indicates the acceleration of positive responses. as well as In response to the identification of the event of interest, one or more digital assets associated with the live media stream are provided to a subset of viewers who provide a response to the live media stream; The audience reaction data obtained includes: Receive audience feedback input; Filtering the audience response input to exclude negative audience response input, thereby generating the audience response data; and Determine the level of audience participation based on the audience response data; The event of interest is defined as an in-stream action that occurs within a predefined time window prior to the detected acceleration.

2. The method as described in claim 1, wherein, The audience response data includes user input obtained via computing devices associated with viewers of the live media stream.

3. The method as described in claim 2, wherein, The user input includes at least one of text input associated with the live media stream or selection of a defined user interface element.

4. The method of claim 2, wherein, The amount of audience participation is determined based on the amount of user input.

5. The method of claim 1, wherein, Obtaining audience reaction data includes determining whether the audience reaction data is non-negative.

6. The method of claim 1, further comprising providing one or more digital assets associated with the live media stream to the stream creator associated with the live media stream.

7. The method of claim 1, further comprising, in response to identifying the event of interest, prompting at least a subset of viewers of the live media stream to obtain input related to the live media stream.

8. The method of claim 1, wherein, Providing one or more digital assets includes: Generate recommendations for product discounts related to the defined products; and The generated recommendations should be provided to at least a subset of the viewers of the live media stream.

9. The method of claim 8, wherein, Generating recommendations further includes determining whether the event of interest is relevant to the defined product.

10. A computing system, comprising: processor; The memory stores computer-executable instructions that, when executed by the processor, enable the processor to: receive video data from a real-time media stream; While streaming the live media stream, audience response data associated with the live media stream is obtained, which at least indicates the amount of audience engagement related to the video content of the live media stream; Events of interest in the real-time media stream are identified by the rate of change of a metric that determines the amount of audience participation exceeding a threshold level; the rate of change of this metric indicates the acceleration of positive responses. as well as In response to the identification of the event of interest, one or more digital assets associated with the live media stream are provided to a subset of viewers who provide a response to the live media stream; The audience reaction data obtained includes: Receive audience feedback input; Filtering the audience response input to exclude negative audience response input, thereby generating the audience response data; and Determine the level of audience participation based on the audience response data; The event of interest is defined as an in-stream action that occurs within a predefined time window prior to the detected acceleration.

11. The computing system of claim 10, wherein, The audience response data includes user input obtained via computing devices associated with viewers of the live media stream.

12. The computing system of claim 11, wherein, The user input includes at least one of text input associated with the live media stream or selection of a defined user interface element.

13. The computing system of claim 11, wherein, The amount of audience participation is determined based on the amount of user input.

14. The computing system of claim 10, wherein, Obtaining audience reaction data includes determining whether the audience reaction data is non-negative.

15. The computing system of claim 10, wherein, When executed by the processor, the instruction further causes the processor to provide one or more digital assets associated with the live media stream to the stream creator associated with the live media stream.

16. The computing system of claim 10, wherein, Providing one or more digital assets includes: Generate recommendations for product discounts related to the defined products; and The generated recommendations should be provided to at least a subset of the viewers of the live media stream.

17. The computing system of claim 16, wherein, Generating recommendations further includes determining whether the event of interest is relevant to the defined product.

18. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a processor, cause the processor to: Receive video data from real-time media streams; While streaming the live media stream, audience response data associated with the live media stream is obtained, which at least indicates the amount of audience engagement related to the video content of the live media stream; Events of interest in the real-time media stream are identified by the rate of change of a metric that determines the amount of audience participation exceeding a threshold level; the rate of change of this metric indicates the acceleration of positive responses. as well as In response to the identification of the event of interest, one or more digital assets associated with the live media stream are provided to a subset of viewers who provide a response to the live media stream; in, Audience reaction data includes: Receive audience feedback input; Filtering the audience response input to exclude negative audience response input, thereby generating the audience response data; and Determine the level of audience participation based on the audience response data; The event of interest is defined as an in-stream action that occurs within a predefined time window prior to the detected acceleration.

Images