Terminals, devices, and methods

Abstract

This invention provides terminals, devices, and methods for live streaming. [Solution] In a live streaming system, a terminal including a circuit performs the steps of: receiving interaction data from viewers in a live streaming room; detecting a first condition from the interaction data; and, in response to the detection of the first condition, transmitting a first command for a first operation to a first device. This can motivate viewers to interact more with the live streamer, potentially increase the rewards for the live streamer, make the live streaming room more enjoyable, facilitate smoother interaction within the live streaming room, and thus potentially improve the user experience.

Description

Technical Field

[0001] This disclosure relates to information and communication technologies, and more particularly to terminals, devices, and methods in live streaming.

Background Art

[0002] Some applications and platforms provide live streaming services that allow live streamers and viewers to interact. Live streamers may perform performances to cheer up viewers, or viewers may donate or send gifts to support live streamers.

[0003] Interaction between viewers and live streamers plays an important role in a live streaming room. Patent Document 1 discloses a live-streamed concert in which a light stick lights up in response to a specific comment sent by a viewer.

[0004] However, current technologies cannot fully meet the needs of viewers and live streamers, which may lead to a decline in the user experience. Therefore, it is important to improve the user experience.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] One embodiment of this disclosure relates to a terminal including a circuit configured to receive interaction data from a viewer in a live streaming room, detect a first condition from the interaction data, and transmit a first command for a first operation to a first device in response to the detection of the first condition.

[0007] Another embodiment of the present disclosure relates to a device including a circuit, which is configured to perform the steps of: receiving a first command for a first action in response to the detection of a first condition from interaction data in a live streaming room; and triggering the first action based on the first command.

[0008] Another embodiment of the present disclosure is a method, which includes the steps of: receiving interaction data from a viewer in a live streaming room; detecting a first condition from the interaction data; and transmitting a first command for a first action to a first device in response to the detection of the first condition.

[0009] According to this disclosure, viewers can be motivated to interact more with live streamers, and live streamers may also receive increased compensation. Furthermore, it can make live streaming rooms more enjoyable and facilitate smoother interaction within them. Therefore, the user experience may be improved. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing the configuration of a live streaming system 1 based on some embodiments of the present disclosure. [Figure 2] This is a block diagram of a user terminal 20 based on some embodiments of the present disclosure. [Figure 3] This is a block diagram of server 10 based on some embodiments of the present disclosure. [Figure 4] This is a block diagram of a device 40 based on some embodiments of the present disclosure. [Figure 5] Figure 3 is a table showing an example data structure for the stream DB320. [Figure 6] Figure 3 is a table showing an example data structure for user DB322. [Figure 7]Figure 3 is a table showing an example data structure for the gift DB324. [Figure 8] Figure 2 is a table showing an example data structure for the terminal gift DB212. [Figure 9] Figure 2 shows a table illustrating the exemplary data structure of DB214. [Figure 10] This is an exemplary screen image of the live streaming room screen 600 displayed on the display of the live streamer's user terminal 20 or the viewer's user terminal 30. [Figure 11] This is an exemplary screen image of the live streaming room screen 600 displayed on the display of the live streamer's user terminal 20 or the viewer's user terminal 30. [Figure 12] This is an exemplary screen image of the live streaming room screen 600 displayed on the display of the live streamer's user terminal 20 or the viewer's user terminal 30. [Figure 13] This is an exemplary screen image of the live streaming room screen 600 displayed on the display of the live streamer's user terminal 20 or the viewer's user terminal 30. [Figure 14] This flowchart shows the steps involved in the application startup process on the user terminals 20 and 30. [Figure 15] This flowchart shows the steps involved in the application startup process on the user terminal 40. [Figure 16] This is an exemplary hardware configuration of an information processing device based on some embodiments of the present disclosure. [Modes for carrying out the invention]

[0011] In the following drawings, identical or similar components, members, procedures, or signals are denoted by the same reference numerals, and redundant explanations are omitted where appropriate. Furthermore, some components that are not essential to the description of each drawing are omitted.

[0012] Based on some embodiments of the present disclosure, the live streaming system 1 provides enhanced functions to smooth communication and interaction among users. More specifically, it is something that entertains viewers and live streamers in a technical way.

[0013] FIG. 1 shows a schematic diagram showing the configuration of the live streaming system 1 based on some embodiments of the present disclosure. The live streaming system 1 provides a live streaming service for real-time interaction between a live streamer (also called a liver, streamer, or distributor) LV who conducts the distribution and viewers (also called an audience) AU (AU1, AU2...). As shown in FIG. 1, the live streaming system 1 can include a server 10, a user terminal 20, and user terminals 30 (30a, 30b...). The user terminal 20 may be a live streamer, and the user terminal 30 may be a viewer. In some embodiments, the live streamer and the viewer may be called users. The server 10 can include one or more information processing devices connected via a network NW. The user terminals 20 and 30 may be, for example, portable terminals such as smartphones, tablets, notebook PCs, recorders, portable game machines, wearable terminals, or stationary computers such as desktop PCs. The server 10, the user terminal 20, and the user terminal 30 may be communicably connected by any type of wired or wireless network NW.

[0014] An app provider (not shown) that provides the live streaming system 1, the live streamer LV, and the viewer AU is involved. The live streamer LV can record content such as their own songs, talks, performances, game streaming, etc. on their user terminal 20 and upload it to the server 10 to become a person who distributes the content in real time. In some embodiments, the live streamer LV can communicate with the viewer AU via the live stream.

[0015] The application provider can provide a platform for content that is live-streamed on the server 10. In some embodiments, the application provider may be a media or manager that manages real-time communication between the live streamer (LV) and the viewer (AU). The viewer (AU) can access the platform via the user terminal 30 and select and watch the content they wish to view. The viewer (AU) can use the user terminal 30 to interact with the live streamer, such as by commenting on or supporting them. The live streamer providing the content can respond to such comments and support. The live streamer's response can be transmitted to the viewer (AU) via video and / or audio. Thus, mutual communication between the live streamer and the viewer can be achieved.

[0016] As used herein, "live distribution (live streaming)" can refer to data transmission that enables a viewer AU to substantially play and view content recorded by a live distributor LV using the user terminal 20 via the user terminal 30. In some embodiments, "live distribution (live streaming)" may also refer to the distribution realized by the above-described data transmission. The live distribution can be realized by known technologies such as HTTP live distribution, CMAF (Common Media Application Format), WebRTC (Web Real-Time Communications), RTMP (Real-Time Messaging Protocol), MPEG DASH, etc. The live distribution may further include an embodiment in which the viewer AU can play or view the content with a specific delay while the live distributor is recording the content. It is desirable that the degree of the delay be small enough for at least the live distributor LV and the viewer AU to communicate with each other. However, live distribution is different from so-called on-demand distribution. More specifically, the on-demand distribution may refer to storing all the data of the recorded content in a server and providing the data from the server to the user at a random timing in response to a user request.

[0017] In this specification, “distribution data” may refer to data including image data and audio data. More specifically, the image data (which may also be called video data) may be generated by the image capture function of the user terminals 20 and 30. The audio data (which may also be called audio data) may be generated by the audio input function of the user terminals 20 and 30. The distribution data may be played back on the user terminals 20 and 30 to allow the user to view content related to the user. In some embodiments, processing such as compression, expansion, encoding, decoding, and transcoding is envisioned to change the format, size, and specifications of the data between the time the distribution data is generated on the live streamer's user terminal and the time it is played back on the viewer's user terminal. Before and after such processing, the content (video and audio) is substantially unchanged, and for this reason, in the current embodiments of this disclosure, the distribution data before processing and the distribution data after processing are described as being the same. In other words, when the streaming data generated by the user terminal of the live streamer is played back on the viewer's user terminal via the server 10, the streaming data generated by the user terminal of the live streamer, the streaming data that has passed through the server 10, and the streaming data received and played back by the viewer's user terminal are all the same streaming data.

[0018] As shown in Figure 1, the live streamer LV provides a live stream. The user terminal 20 of the live streamer generates streaming data by recording the streamer's video and / or audio, and transmits it to the server 10 via the network NW. At the same time, the user terminal 20 can display the video VD on its display and check the streaming content of the live streamer LV.

[0019] Viewers AU1 and AU2 of user terminals 30a and 30b, which request the platform to provide the live stream from the live streamer, receive the distribution data corresponding to the live stream via the network NW, play the received distribution data to display the video VD1 and VD2 on a display, and output audio from speakers or the like. The video VD1 and VD2 displayed on user terminals 30a and 30b are substantially the same as the video recorded by the user terminal of the live streamer LV, and the audio output from user terminals 30a and 30b is substantially the same as the audio recorded by the user terminal of the live streamer LV.

[0020] The recording on the user terminal 20 of the live streamer may occur simultaneously with the playback of the streamed data on the user terminals 30a and 30b of the viewers AU1 and AU2. When viewer AU1 inputs a comment regarding the content of the live streamer LV into its user terminal 30a, the server 10 displays the comment in real time on the live streamer's user terminal 20, and also on the user terminals 30a and 30b of viewers AU1 and AU2, respectively. If the live streamer LV responds to the comment, the response is output as text, image, video, or audio from the user terminals 30a and 30b of viewers AU1 and AU2, thereby enabling communication between the live streamer LV and viewers AU1 and AU2. Thus, the live streaming system can realize two-way live streaming.

[0021] Figure 2 is a block diagram showing the functions and configuration of the user terminal 20 shown in Figure 1, based on an embodiment of the present disclosure. The user terminal 30 has the same functions and configuration as the user terminal 20. The blocks depicted in the block diagrams of this specification are implemented by hardware such as a computer CPU or mechanical parts, and software such as a computer program that represents a functional block performed by the cooperation of these elements. Therefore, it will be understood by those skilled in the art that functional blocks can be implemented in various ways through combinations of hardware and software.

[0022] The live streamer (LV) and viewer (AU) can download and install the live streaming application (live streaming app) described herein from a download site to their user terminals 20 and 30 via the network (NW). Alternatively, the live streaming app may be pre-installed on the user terminals 20 and 30. By performing live streaming on the user terminals 20 and 30, they can communicate with the server 10 via the network (NW) and implement multiple functions. These functions, implemented by the execution of the live streaming app by the user terminals 20 and 30 (more specifically, by the processor such as the CPU), are described below as functions of the user terminals 20 and 30. These functions are basically functions that the live streaming app enables the user terminals 20 and 30 to implement. In some embodiments, these functions may be implemented by sending them from the server 10 to the web browser of the user terminals 20 and 30 via the network (NW) and executing them by a computer program in the web browser. This computer program may be written in a programming language such as HTML (Hyper Text Markup Language).

[0023] The user terminal 20 includes a streaming unit 100 and a viewing unit 200. In some embodiments, the streaming unit 100 is configured to record the user's audio and / or video data and generate distribution data to be sent to the server 10. The viewing unit 200 is configured to receive and play the distribution data from the server 10. In some embodiments, the user can activate the streaming unit 100 when broadcasting or when viewing the distribution. In some embodiments, the user terminal activating the streaming unit 100 may be referred to as the live broadcaster or the user terminal generating the distribution data. The user terminal activating the viewing unit 200 may be referred to as the viewer or the user terminal playing the distribution data.

[0024] The streaming unit 100 may include a video control unit 102, an audio control unit 104, a distribution unit 106, and a UI control unit 108. The video control unit 102 may be connected to a camera (not shown), and the video is controlled by the camera. The video control unit 102 can acquire video data from the camera. The audio control unit 104 may be connected to a microphone (not shown), and the audio is controlled by the microphone. The audio control unit 104 can acquire audio data from the microphone.

[0025] The distribution unit 106 receives distribution data, which includes video data from the video control unit 102 and audio data from the audio control unit 104, and transmits it to the server 10 via the network NW. In some embodiments, the distribution unit 106 transmits the distribution data in real time. That is, the generation of the distribution data from the video control unit 102 and the audio control unit 104 and the distribution by the distribution unit 106 are performed simultaneously.

[0026] The UI control unit 108 controls the UI of the live streamer. The UI control unit 108 is connected to a display (not shown) and is configured to generate the streaming data for the recipient to whom the streaming unit 106 transmits, plays back, and displays on the display. The UI control unit 108 is configured to display an object to be manipulated or an object to receive instructions on the display and to accept tap input from the live streamer.

[0027] The viewing unit 200 may include a UI control unit 202, a rendering unit 204, an input transmission unit 206, a processing unit 210, a terminal gift DB 212, and a configuration DB 214. The viewing unit 200 is configured to receive distribution data from the server 10 via a network NW. The UI control unit 202 controls the viewer's UI. The UI control unit 202 is connected to a display (not shown) and / or a speaker (not shown) and is configured to display video on the display and output audio from the speaker by playing the distribution data. In some embodiments, outputting video on the display and outputting audio from the speaker may be referred to as "playing the distribution data". The UI control unit 202 is connected to an input unit such as a touch panel, keyboard, or display and can acquire input from the user.

[0028] The rendering unit 204 may be configured to render the data distributed from the server 10 and a frame image. The frame image may include user interface objects for receiving user input, comments entered by viewers, and data received from the server 10. The input transmission unit 206 is configured to receive the user input from the UI control unit 202 and transmit it to the server 10 via the network NW.

[0029] In some embodiments, the user input may be clicking on an object on the user's terminal screen, such as selecting a live stream, entering a comment, sending a gift, following or unfollowing a user, voting in an event, or playing a game. For example, the input transmission unit 206 may generate gift information and send it to the server 10 via the Internet network when the viewer's terminal clicks on a gift object on the screen to send a gift to the live streamer.

[0030] The processing unit 210 is configured to process interaction data within the live streaming room. This interaction data may include any parameters possible within the live streaming room, such as comments from viewers, the number of gifts from each viewer or all viewers individually or in total, votes, purchases, game scores, and PK status. The processing unit 210 may also detect specific conditions from the interaction data. These conditions may include, for example, a specific number of gifts from each viewer or all viewers to the live streamer, sales figures, revenue from each or all products, wins and losses in penalty shootouts, and the number of points received or ranking achievements by individuals or groups.

[0031] In some embodiments, the processing unit 210 may trigger devices around the live streamer to perform actions. These devices may be, for example, wearable devices such as watches, earrings, necklaces, or patches. They may also be devices near the live streamer, such as aroma diffusers, music boxes, fans, dolls, chairs, desks, glasses, drones, robots, or disco balls. In some embodiments, the devices may be pre-manufactured or existing devices that can be connected to trigger specific physical effects, such as light, scent, electricity, or movement. In some embodiments, pre-manufactured additional components may be added to existing devices to form the devices. For example, an electric patch may be added to a chair or watch to form an electric chair or watch.

[0032] In some embodiments, the processing unit 210 may trigger the device in response to the detection of specific conditions. The means for triggering the device may be wired or wireless communication technology, such as NFC, Bluetooth, Wi-Fi, Zigbee, or Z-Wave. In some embodiments, the terminal and the device may incorporate wireless communication equipment for data transmission and device control. For example, the processing unit 210 may send a signal to an aroma diffuser, triggering it to release aroma, in response to a viewer sending a perfume gift object, or to a live streamer receiving more than one million points.

[0033] In some embodiments, the condition may be greater than or less than a specific parameter, such as points received or sales revenue. In some embodiments, the condition may be a dynamic combination of the conditions described above. For example, the condition may be two thresholds: receiving 500,000 points or more and receiving 1,000,000 points or more. As another example, the condition may be that the number of viewers is 1,000 or more, less than 1,000, and then 1,000 or more again. Here, the condition may be the number of viewers in the live streaming room, and the number being greater than or equal to, less than, and then greater than or equal to, is an example of the combination of the conditions described above.

[0034] In some embodiments, the operation of the device in response to the fulfillment of the conditions may be the same or different depending on the conditions. For example, if the first condition is met, the lamp may light up, and if the second condition is met, the lamp may relight, repeatedly light on and off, light up brighter, or trigger another device. In some embodiments, the conditions and the operation of the device may be flexibly changed and combined as needed.

[0035] In some embodiments, the processing unit 210 may be installed in the terminal as described above, or it may be installed inside a server 10 or the like. More specifically, the detection of specific conditions and the triggering of the device may be performed from the server 10 or from the terminal. The server 10 may trigger the device to operate directly, or it may operate indirectly through the terminal. In some embodiments, the device may detect the location of the terminal by receiving location information from the terminal or any other possible positioning technology or image detection technology. In some embodiments, the device may operate based on the location of the terminal, etc. For example, a music box may play music facing the direction of the terminal, or a drone carrying a gift may fly towards the location of the live streamer's smartphone or the like.

[0036] Figure 3 is a block diagram of the server 10 based on some embodiments of the present disclosure. The server 10 may include a distribution information unit 302, a relay unit 304, a processing unit 306, a stream DB 320, a user DB 322, and a gift DB 324.

[0037] The distribution information unit 302 receives a live streaming request from the live streamer's user terminal 20 via the network NW. Upon receiving the request, the distribution information unit 302 registers the live streaming information in the stream DB 320. In some embodiments, the live streaming information may consist of the stream ID of the live streaming and / or the live streamer ID of the live streamer corresponding to the live streaming.

[0038] When the distribution information unit 302 receives a request for information about the live stream from the viewer via the network NW to the viewing unit 200 of the user terminal 30, it refers to the stream DB 320 and generates a list of available live streams.

[0039] Subsequently, the distribution information unit 302 transmits the list to the user terminal 30 via the network NW. The UI control unit 202 of the user terminal 30 generates a live streaming selection screen based on the list and displays the list on the user terminal 30's display.

[0040] When the input transmission unit 206 of the user terminal 30 receives a live stream selection by the viewer on the live stream selection screen, it generates a distribution request including the stream ID of the selected live stream and sends it to the server 10 via the network. The distribution information unit 302 can then begin providing the live stream specified by the stream ID in the distribution request to the user terminal 30. The distribution information unit 302 can update the stream DB 320 and add the viewer ID of the viewer on the user terminal 30 to the live streamer ID of the stream ID.

[0041] The relay unit 304 can relay the transmission of the live stream from the live streamer's user terminal 20 to the viewer's user terminal 30 in the live stream initiated by the distribution information unit 302. The relay unit 304 can receive a signal indicating user input from the viewer from the input transmission unit 206 while the distribution data is being played back. The signal indicating user input may be an object designation signal indicating the designation of an object displayed on the user terminal 30's screen. The object designation signal may include the viewer's viewer ID, the live streamer ID of the live streamer broadcasting the live stream the viewer is watching, and the object ID designated by the object. If the object is a gift, the object ID may be a gift ID, etc. Similarly, the relay unit 304 can receive a signal indicating user input from the live streamer, such as the object designation signal, from the streaming unit 100 of the user terminal 20 while the distribution data is being played back.

[0042] The processing unit 306 is configured to process data between the viewer and the live streamer within the live streaming room. In some embodiments, the processing unit 306 may function similarly to the processing unit 210 in the terminal. In some embodiments, the processing unit 306 may handle data storage, updating, etc., on a server 10 or the like.

[0043] In some embodiments, the server 10 may further include a gift information unit (not shown) and a gift processing unit (not shown). The gift information unit provides gift information to the corresponding broadcaster and viewer. The gift information unit sends gift information for the user terminal to the broadcaster in response to a request from the user terminal. The gift information unit obtains effect data from the gift DB corresponding to the gift ID included in the gift information signal received by the relay unit 304. The gift information unit sends the obtained effect data to other user terminals 20, 30 as a response to the gift transmission request signal.

[0044] The gift processing unit updates the user database 322 to update the points of the broadcaster and the viewer according to the points of the gift identified by the gift ID included in the gift redemption signal. Specifically, the gift processing unit refers to the gift database and identifies the points to be awarded for the gift ID included in the received gift redemption signal. The gift processing unit then updates the user database 322 and adds the determined points to the points of the broadcaster ID included in the gift redemption signal. In some embodiments, the gift processing unit may update the user database 322 and subtract the determined points from the points of the viewer ID included in the gift redemption signal.

[0045] Figure 4 is a block diagram of the device 40 based on some embodiments of the present disclosure. The device 40 may include a receiving unit 402, a processing unit 406, a transmitting unit 410, and the like. The receiving unit 402 is configured to receive signals from a user terminal or server 10. In some embodiments, the signals may include commands to trigger the operation of the device 40, etc. In some embodiments, the signals may further include information about the user terminal from the user terminal or server 10, etc.

[0046] The processing unit 406 is configured to perform the operation of the device based on the signal from the user terminal or server 10. In some embodiments, the operation may include physical effects that can be perceived by the live streamer or the like. The transmitting unit 410 is configured to transmit any possible or necessary information to the user terminal or server 10 or the like. For example, when the operation of the device 40 is completed, the transmitting unit 410 may transmit an operation completion signal to the user terminal or server 10 or the like.

[0047] Figure 5 is a table showing an exemplary data structure of the Stream DB 320 in Figure 3. The Stream DB 320 holds information about the live stream currently in progress. The Stream DB 320 stores, in association with each other, a Stream ID for identifying the live stream on the live streaming platform provided by the Live Streaming System 1, a Live Streamer ID for identifying the live stream provider, and a Viewer ID for identifying the viewers of the live stream.

[0048] Figure 6 is a table showing an exemplary data structure of the user DB322 in Figure 3. The user DB322 holds information about users. The user DB322 stores, in relation to each other, a user ID to identify the user, points to identify the points accumulated by the user, a level to identify the user's level, and a status to identify the user's status. The points are electronic value that circulates within the live streaming platform. The level may be an indicator of the amount of activity or engagement of the user on the live streaming platform. The status may be the user's identity or membership status on the live streaming platform.

[0049] Figure 7 shows a table illustrating the exemplary data structure of the Gift DB324 in Figure 3. The Gift DB324 holds information about gifts that viewers can use during live streaming. The Gift DB324 stores, in association with each other, a Gift ID to identify the gift, the amount of points that viewers spend to send the gift to the streamer, an Event ID to identify the event to which the gift corresponds, and gift effect data.

[0050] Viewers can send gifts to broadcasters by paying equivalent value points for their desired gifts while watching the live stream. Such equivalent value point payments can be made by appropriate electronic means; for example, viewers can pay equivalent value points to the administrator. Alternatively, bank transfers or credit card payments may be used. The relationship between awarded points and equivalent value points can be set at the administrator's discretion.

[0051] A gift is electronic data with the following characteristics: - It can be purchased in exchange for points (explained in detail later), or it may be given away for free. - Viewers can send gifts to the streamer. - Sending gifts to streamers is referred to as using gifts or throwing gifts. - Some gifts can be used immediately upon purchase, while others can be used later at any time by the purchaser or recipient after the gift has been given. - When a viewer sends a gift to a streamer, the streamer is awarded points corresponding to the gift, and an effect related to the gift is applied. For example, the effect corresponding to the gift is displayed on the live stream screen.

[0052] The effect in question is a visual, auditory, or tactile effect (e.g., vibration) that characterizes the gift, or a combination thereof. Examples of visual effects include animation, images, and flashing lights. Examples of auditory effects include sound effects and voices. The effect data is data that enables such effects to be implemented on the user terminal 20, and the user terminal 20 implements such effects by processing the effect data. Since the technology for implementing the effect data is publicly known, a detailed explanation thereof is omitted below.

[0053] Figure 8 is a table showing an exemplary data structure of the terminal gift DB 212 in Figure 2. The terminal gift DB 212 is configured to store the gift information of the user terminal 20 or 30. The terminal gift DB 212 may include a gift ID, an icon ID, an effect data ID, and a last updated date. The gift ID may identify the gift. The icon ID may be the icon data of the gift object. The effect data ID may identify the data that implements the effect corresponding to the gift. The last updated date may identify the last updated date of the gift.

[0054] In some embodiments, the viewer may download the gift in advance, or download the gift when sending it. In some embodiments, the gift information may be downloaded and stored in the terminal gift DB 212 in advance. In some embodiments, the terminal gift DB 212 may include a gift list corresponding to the gift list provided by the server 10. In some embodiments, the gift list in the terminal gift DB 212 may be synchronized with the server 10 periodically or by user operation.

[0055] Figure 9 is a table showing an exemplary data structure of the configuration DB214 in Figure 2. The configuration DB214 holds information about the settings of the conditions related to the operation of the device. The configuration DB214 stores, in association with each other, a configuration ID to identify the setting, conditions that identify the conditions that trigger the operation of the device, a device ID that identifies the device to be triggered, and an operation that identifies the operation of the device.

[0056] In some embodiments, such settings may be generated manually by the live streamer or viewer, or automatically by the server 10, etc. For example, the live streamer may set the music box to operate when they receive 1 million points or more, or the server 10 may set the neon lights to turn on and off in response to the number of received points reaching 1,000, 1 million, 1 billion or more.

[0057] In some embodiments, the operation of the device may be any visual, auditory, olfactory, gustatory, or tactile sensation that the device can generate, such as light, color, vibration, shape, distance, sound, volume, pitch, rhythm, smell, taste (sweet, sour, etc.), temperature, texture, pressure, or pain on the skin. For example, the operation of the device may be such that a fan blows air towards a live streamer.

[0058] Figures 10 to 13 are illustrative screen images of a live streaming room screen 600 displayed on the display of the live streamer's user terminal 20 or the viewer's user terminal 30. When a viewer selects a live streaming room and enters it, the live streamer's live streaming room screen 600 may be displayed on the display. The live streaming room screen 600 may include a live streamer information object 602, a live streamer image 604, a message zone 606, a message input box 608, a gift object 610, a shared object 612, etc.

[0059] The live streamer information object 602 displays information about the streamer. The live streamer image zone 604 displays an image or video of the streamer, which may be obtained by playing video data. The message zone 606 displays messages from viewers, the streamer, or the server 10. The message input zone 608 indicates an input zone for the streamer or viewer to enter messages, etc. The gift object 610 provides the user with the ability to send gifts to the live streamer, and the share object 612 provides the user with the ability to share the live streaming room. In some embodiments, the UI and related functions within the live streaming room may be flexibly designed.

[0060] In some embodiments, the live streamer may broadcast from any possible location. Figure 10 shows an example of a location within the live streamer's room. The live streamer may apply makeup, wear a necklace 620, earrings 622, and a watch 624, and dress up for the broadcast. In some embodiments, the live streamer's room may contain everyday necessities, small appliances, and household gadgets such as an aroma diffuser 626, a smart speaker 628, a doll 630, and an electric fan 632.

[0061] The live streamer may perform acts to support the viewers, and the viewers may send gifts to support the live streamer. In some embodiments, when the number of points received by the live streamer reaches a threshold such as 1 million, the operation of the device may be triggered, as shown on the right side of Figure 10. For example, if the device is a necklace 620 or earrings 622, the necklace 620 or earrings 622 may light up, repeatedly light up and turn off, etc. In some embodiments, if the device is a wristwatch 624, the operation may be the generation of a low-frequency current. Here, in order to realize this function, an electrode patch may be added separately to the wristwatch 624, or the wristwatch 624 may be manufactured with the electrode patch already installed.

[0062] In some embodiments, if the device is an aroma diffuser 626 or a smart speaker 628, the action may be the release of a pleasant scent or a beautiful melody in the room or directed towards the live streamer. If the device is a doll 630, the action may be waving or moving its arms. If the device is an electric fan 632, the action may be the activation of the fan, generating airflow in the room or directed towards the live streamer. In some embodiments, the device may be an existing device, a pre-manufactured device, or an existing device with pre-manufactured elements added.

[0063] In some embodiments, the pre-fabricated element may produce the same, similar, or different physical effects as existing devices. For example, an LED patch may be added to the smart speaker 628 to light up the appearance of the smart speaker. In some embodiments, the parameter of the condition may be any parameter possible within the live streaming room, such as points sent, points received, number of purchases, sales revenue, votes, viewing time, broadcast time, or game score. For example, the operation of the device may be triggered when the broadcast time exceeds one hour or when the live streamer achieves a high score in a game.

[0064] In some embodiments, virtual effects may be added to the location of or near the device on the screen 600, in addition to the physical effects produced by the device, to generate virtual reality or augmented reality effects. For example, a virtual effect 634 of airflow may be displayed between the electric fan 632 and the live streamer to form or emphasize the movement of the airflow from the electric fan 632. As another example, a virtual effect in the shape of the doll 630 may be added to the doll 630 to change the doll's color and facial expression, making the doll's movements more interesting.

[0065] In some embodiments, the live streamer may engage in group calls or player killing (PK) with other live streamers and perform multi-user interactions. Figure 11 shows an example of PK with other live streamers within a live streaming room. In some embodiments, the conditions may be any possible trigger point within the live streaming room, such as the start or end of PK, winning or losing a PK, or the difference in PK scores reaching a threshold.

[0066] As shown on the left side of Figure 11, one live streamer may start a player-killing (PK) with another live streamer, and viewers may send gifts to support their favorite live streamer. In some embodiments, when the PK time ends and one live streamer has received more gifts than the other live streamer, it may be understood that the live streamer has defeated the other live streamer and won the PK. As shown on the right side of Figure 11, a low-frequency current 638 may be generated as a penalty on the other live streamer's chair 636. Here, the chair may be a chair that has been pre-manufactured to achieve this function or an existing chair with electrode patches attached. According to this embodiment, it is possible to bring more fun and excitement to the PK participants and improve the user experience.

[0067] In some embodiments, the disclosure may be applied to offline events. Figure 12 shows an example of an offline event. In an offline event, many chairs and tables may be provided, and a live streamer may be seated and broadcasting, calling on viewers to support them in winning the event. On stage, there may be a host who introduces the event and controls its flow. The conditions may be any possible trigger point during the event, such as exceeding a threshold in points received or making it into the top 10 on the final scoreboard. The operation of the device may also be any possible operation or device in the offline event, such as lighting up desks, chairs, glasses, etc.

[0068] In some embodiments, the device may be a drone 640 or a robot 642, as shown in Figure 12. In some embodiments, the device may receive information about the user terminal, such as the location of the live streamer's user terminal. The location may be absolute coordinates, relative coordinates, or the distance from the device to the user terminal, and the detection of the location may be achieved by positioning technology.

[0069] In some embodiments, the operation may be performed based on information from the user terminal. For example, as shown in Figure 12, the operation may be a drone 640 or a walking robot 642 carrying the prize 644 flying toward the location of the user terminal in order to deliver the prize 644 to the live streamer. In some embodiments, information about the prize 644 may be included in the signal from the user terminal to the device so that the drone 640 or robot 642 can select the correct prize 644 and deliver it to the live streamer.

[0070] In some embodiments, the device may include a face recognition unit (not shown) or the like. For example, if the live streamer wants to thank a viewer for their support, the live streamer may send the viewer a specific virtual gift. This specific virtual gift may trigger the operation of the drone 640 or robot 642. When the specific virtual gift is sent via the user terminal, the user terminal may send the trigger for the operation and the viewer's information (such as a displayable photo of the viewer) to the drone 640 or robot 642. The drone 640 or robot 642 may bring a physical gift, detect the viewer's face there using the face recognition unit, and send the physical gift to the viewer.

[0071] In some embodiments, the condition may be a parameter within the individual live streamer or a competition group. In some embodiments, if live streamers sitting at the same table in an event belong to the same group, the total points received by the live streamers in that group may be the condition that triggers the operation of the device. For example, if the total points received by the group at table 646 reach a threshold, table 646 may be lit up to celebrate the group's achievement at table 646.

[0072] In some embodiments, the disclosure may apply to a live commerce event. Figure 13 shows an example of a live commerce event. In the live commerce event, the live streamer may introduce a product 520 on platform 652 and encourage viewers in the live streaming room to purchase the product. The product may be, for example, cups or cosmetics in various types, sizes, and colors, or a smartphone.

[0073] In some embodiments, the conditions may be any trigger point possible during the live commerce event, such as the number of sales for each type of product or for all products, the sales revenue for each type of product or for all products, the number of viewers in the live streaming room, or the number of likes from viewers. In some embodiments, the device may be the product itself, a platform for displaying the product, a decorative ball, or any other device possible during the live commerce event.

[0074] As shown in Figure 13, the product 650 may be a cup, and the cup may include different colors and patterns, such as white with a smiley face, gray and black. Each type of cup may be displayed on a separate desk. In some embodiments, if a particular type of product 650, such as a "gray cup," is sold out, it means that the number of gray cups sold has reached the inventory quantity, and the desk light may change from on to off to indicate that the product is sold out. Here, this condition may also be that the number of products sold is equal to the inventory quantity, and the operation of the device may be such as the desk light changing from on to off.

[0075] In some embodiments, the condition may be sales revenue, and the operation of the device may be equipment within the live commerce event. As shown in Figure 13, if the total purchase amount from viewers exceeds a threshold such as 1 million, the kusudama 654 device may be triggered to open and a celebratory banner may be displayed. According to this embodiment, special physical effects may encourage viewer donations and purchases, and may help the live streamer maintain the atmosphere in the live streaming room and receive more revenue from viewers.

[0076] Figure 14 is a flowchart showing the steps of the application startup process on the user terminals 20 and 30. In some embodiments, the live streamer may start a live stream, and viewers may enter the live streaming room and enjoy interacting with the live streamer. In some embodiments, the live streamer may receive the interaction data from the viewers via the live streamer's user terminal (S502).

[0077] In some embodiments, the interaction data may be any parameters possible within the live streaming room, such as sending gifts, voting, purchasing, following, or liking. In some embodiments, the processing unit 210 may detect specific conditions from the interaction data (S504). In some embodiments, if no specific conditions are met from the interaction data ("No" in S504), the live streamer may continue to interact with the viewer and receive the interaction data (S502).

[0078] In some embodiments, if certain conditions are met from the interaction data ("yes" in S504), the processing unit 210 may determine the device and the corresponding operation of the device (S506). In some embodiments, the device and the corresponding operation may be stored in a database or lookup table so that the processing unit 210 can retrieve the information. Once the device and the corresponding operation are retrieved and determined in light of the conditions, the processing unit 210 may send a signal to the device to trigger the operation (S512).

[0079] In some embodiments, the operation of the device may be triggered simply, or it may be triggered based on further information from the user terminal or server 10, etc. For example, the electric fan device may be triggered to simply blow air, or it may be triggered to blow air towards the location of the user terminal or the live streamer, etc.

[0080] In some embodiments, the processing unit 210 may determine whether or not information about the user terminal is necessary (S508). In some embodiments, the information about the user terminal may be any information available on the user terminal. For example, the information about the user terminal may include the physical location inside and outside the live streaming room in the application, temperature, delivery data, interaction data, and information related to the live streamer or viewers in the live streaming room.

[0081] In some embodiments, if the user terminal information is not needed ("No" in S508), the processing unit 210 may simply send a signal to the device to trigger the operation (S512). For example, if the operation of the device is to make a doll wave its hand, the signal may simply be sent to a pre-manufactured doll to perform the doll's action.

[0082] In some embodiments, if information about the user terminal is required (Yes in S508), the processing unit 210 may receive information about the user terminal from a local database or server 10, etc. (S510). Upon receiving such information, the processing unit 210 may transmit a signal along with the information to trigger an action in the device (S512). For example, the action and the device may release perfume from the aroma diffuser toward the live streamer, and the aroma diffuser may be triggered based on the location of the user terminal, etc.

[0083] Figure 15 is a flowchart showing the steps of the application startup process in the device 40. In some embodiments, the device may receive a signal that triggers operation in the device (S552). In some embodiments, the device may be any existing or pre-manufactured item worn by or near the live streamer during the live stream.

[0084] In some embodiments, if the information of the user terminal is not needed ("No" in S544), the processing unit 406 may simply perform the operation on the device (S558). For example, the smart speaker may play music or the LED light may turn on or off. In some embodiments, if the information of the user terminal is needed ("Yes" in S544), the processing unit 406 may further perform the operation on the device based on the information of the user terminal (S556). For example, the electric fan may blow air towards the live streamer or the LED light may shine light towards the live streamer.

[0085] In some embodiments, the device may receive any possible information from the user terminal or its vicinity, such as the live streamer. For example, the information may be the temperature of the smartphone or the body temperature of the live streamer, and the device may be operating as an electric fan 630. In some embodiments, the settings may be flexibly determined according to the actual needs.

[0086] In some embodiments, the conditions and the operation of the device may be determined based on the type of virtual item. For example, if a viewer sends a virtual gift of a fan, the electric fan may be triggered to blow air. In some embodiments, instead of triggering the device, the device may be triggered to turn off in response when a condition is detected from the interaction data. For example, if a viewer sends a virtual gift of a "power outage," the electric fan may be triggered to turn off and stop generating wind directed towards the live streamer.

[0087] In some embodiments, the condition may be triggered by a specific event, such as a virtual gift in a lucky bag winning the top prize. Here, "lucky bag" may be called a virtual gift sent to a live streamer for a specified number of points, and the live streamer may receive points equal to or higher / lower than the specified number. For example, a virtual gift in a lucky bag worth 10,000 points may have different potential values, such as 1 million, 20,000, 5,000, 100, or 1. If the lucky bag wins the top prize of 1 million, the operation of a device such as a smart speaker that plays music may be triggered.

[0088] In some embodiments, the conditions and the operation of the device may be related to more than one live streamer. For example, if multiple live streamers are individually playing penalty shootouts (PK) against each other, the crowns or other symbols above each streamer's head may light up when they win a PK. According to this embodiment, other live streamers can easily see who won the PK, who they will be playing against, etc. In some embodiments, different conditions may correspond to different types of operation on the device. For example, the condition of receiving 10,000 or 1,000,000 points may trigger a white light or a yellow light.

[0089] In some embodiments, different conditions may trigger the same, but different, operation in the device, such as differing characteristics, attributes, intensity, or genre. For example, the operation may be a light-up, but it may differ in form, such as white light or yellow light, strong light or weak light, flashing or steady light. In some embodiments, different conditions may trigger different operations in the device. In some embodiments, the same or different conditions may trigger the same or different operations in the same or different devices. In some embodiments, the combination of conditions, operations, and devices may be flexibly determined as needed.

[0090] In some embodiments, the processing unit 212 may detect whether or not a virtual image of the device is displayed on the screen 600. For example, if the device is a doll 630, the processing unit 212 may detect whether or not the doll 630 is displayed on the screen of the live streamer's user terminal, as shown in Figure 10. In some embodiments, if the device is not displayed on the screen 600, a message indicating this may be sent to or displayed as a pop-up on the user terminal. The live streamer may then adjust the user terminal or move the device so that it is displayed on the screen 600. According to this embodiment, when such action of the device is triggered, viewers in the live streaming room may become aware of the action, potentially improving the user experience.

[0091] In some embodiments, the information received by the device from the user's terminal may not only be used to perform the operation but may also be displayed for reference by other users. For example, if a live streamer receives 1 million points or wins an event, their name may be displayed on a large screen on the stage, or a smart speaker may be triggered to announce the live streamer's achievements.

[0092] According to this disclosure, viewers can be motivated to interact more with live streamers, and live streamers may also receive increased compensation. Furthermore, it can make live streaming rooms more enjoyable and facilitate smoother interaction within them. Therefore, the user experience may be improved.

[0093] Figure 16 is a schematic block diagram of computer hardware for executing a system configuration and processing based on certain embodiments of the present disclosure. The information processing device 900 in Figure 16 is configured, for example, to realize the server 10 and the user terminals 20 and 30, respectively, based on certain embodiments of the present disclosure.

[0094] The information processing device 900 includes a CPU 901, read-only memory (ROM) 903, and random access memory (RAM) 905. Furthermore, the information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input unit 915, an output unit 917, a storage unit 919, a drive 921, a connection port 925, and a communication unit 929. The information processing device 900 may also include an imaging device (not shown), such as a camera. The CPU 901 is an example of a hardware configuration for realizing the functions realized by the components described herein. The functions described herein may also be realized by circuits programmed to realize such functions. Circuits programmed to realize the functions described herein include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a general-purpose processor, an application-specific processor, an integrated circuit, an ASIC (Application Specific Integrated Circuit), and / or a combination thereof. In this specification, a unit that implements a specific function may be implemented as a circuit programmed to implement that function. This unit includes, but is not limited to, a streaming unit 100, a video control unit 102, an audio control unit 104, a distribution unit 106, a UI control unit 108, a viewing unit 200, a UI control unit 202, a rendering unit 204, an input transmission unit 206, a processing unit 210, a terminal gift DB 212, a setting DB 214, a distribution information unit 302, a relay unit 304, a processing unit 306, a stream DB 320, a user DB 322, a gift DB 324, a device 40, a receiving unit 402, a processing unit 406, a transmission unit 410, and the like.

[0095] The CPU 901 functions as an arithmetic processing unit and control unit, and controls the overall operation or a part of the operation of the information processing unit 900 according to various programs recorded in the ROM 903, RAM 905, storage unit 919, or removable recording medium 923. For example, the CPU 901 controls the overall operation of each functional unit included in the server 10 and the user terminals 20 and 30 in the above-described embodiment. The ROM 903 stores programs and operating parameters used by the CPU 901. The RAM 905 transiently stores programs used when the CPU 901 is executed and parameters that change as appropriate when executing those programs. The CPU 901, ROM 903, and RAM 905 are connected to each other via a host bus 907, which is composed of internal buses such as a CPU bus. The host bus 907 is connected to an external bus 911, such as a peripheral component interconnect / interface (PCI) bus, via a bridge 909.

[0096] The input unit 915 is a device operated by the user, such as a mouse, keyboard, touch panel, button, switch, or lever. The input unit 915 may also be a device that converts physical quantities into electrical signals, such as an audio sensor (microphone, etc.), acceleration sensor, tilt sensor, infrared sensor, depth sensor, temperature sensor, or humidity sensor. The input unit 915 may also be a remote control device that uses infrared rays or other types of radio waves. Alternatively, the input unit 915 may be an external connection terminal 927, such as a mobile phone, that is compatible with the operation of the information processing device 900. The input unit 915 includes an input control circuit that generates an input signal based on information input from the user and outputs the generated input signal to the CPU 901. The user inputs various data and gives instructions to the information processing device 900 for processing operations by operating the input unit 915.

[0097] The output unit 917 includes a device that can visually or audibly notify the user of the acquired information. The output unit 917 may be, for example, a display device such as an LCD, PDP, or OLED, an audio output device such as a speaker or headphones, or a printer. The output unit 917 outputs the results obtained by the processing performed by the information processing device 900 in the form of text, images or other visuals, or audio or other sounds.

[0098] The storage unit 919 is a data storage device and is an example of a storage unit of the information processing device 900. The storage unit 919 includes, for example, magnetic storage devices such as hard disk drives (HDDs), semiconductor storage devices, optical storage devices, magneto-optical storage devices, etc. The storage unit 919 stores programs and various data executed by the CPU 901, as well as various data acquired from external sources.

[0099] The drive 921 is a reader / writer for removable recording media 923 such as magnetic disks, optical disks, magneto-optical disks, and semiconductor memory, and is built into or attached to the information processing device 900. The drive 921 reads information recorded on the installed removable recording media 923 and outputs it to the RAM 905. The drive 921 also writes data to the installed removable recording media 923.

[0100] The connection port 925 is a port used to directly connect equipment to the information processing device 900. The connection port 925 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. The connection port 925 may also be an RS-232C port, an optical audio terminal, an HDMI (High-Definition Multimedia Interface®) port, etc. By connecting an external connection terminal 927 to the connection port 925, various types of data can be exchanged between the information processing device 900 and the external connection terminal 927.

[0101] The communication unit 929 is, for example, a communication interface that includes a communication device for connecting to a communication network NW. The communication unit 929 may also be, for example, a communication card for a wired or wireless local area network (LAN), Bluetooth®, or wireless USB (WUSB).

[0102] The communication unit 929 may be, for example, a router for optical communication, a router for ADSL (asymmetric digital subscriber line), or a modem for various types of communication. For example, the communication unit 929 uses a predetermined protocol such as TCP / IP to send and receive signals over the Internet or to other communication devices. The communication network NW to which the communication unit 929 is connected is a network established by a wired or wireless connection. The communication network NW may be, for example, the Internet, a home LAN, infrared communication, radio communication, or satellite communication.

[0103] The imaging device (not shown) is a device that captures real space and generates an image using an image sensor such as a CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor) and various components such as a lens for controlling the imaging of a subject on the image sensor. The imaging device may capture still images or moving images.

[0104] The live streaming system 1 of this disclosure has been described above with reference to embodiments. The embodiments described above are for illustrative purposes only. Rather, it is readily apparent to those skilled in the art that various combinations and modifications of the components and processes of the embodiments can be made, and these are also included within the technical scope of this disclosure.

[0105] The processes described herein, particularly those described using flowcharts, may be modified by omitting some of the processes, adding processes not explicitly included in the processes, and / or rearranging the order of the processes. Such modifications, including omissions, additions, and rearrangements, are still included within the scope of this disclosure, provided they do not deviate from the essence of this disclosure.

[0106] In some embodiments, at least a portion of the functions performed by the server 10 may be performed by an entity other than the server 10, for example, by the user terminal 20 or 30. In some embodiments, at least a portion of the functions performed by the user terminal 20 or 30 may be performed by an entity other than the user terminal 20 or 30, for example, by the server 10. In some embodiments, the rendering of frame images may be performed by the user terminal, such as a viewer, server, or live streamer.

[0107] Furthermore, the system or method described in the above embodiments may be provided as a non-transient computer-readable storage device such as a solid-state memory device, optical disk memory device, or magnetic disk memory device, or as a computer program product. Alternatively, the program may be downloaded from a server via the Internet.

[0108] While the technical content and features of this disclosure have been described above, any person with ordinary skill in the art to which this disclosure pertains can make many further modifications and alterations without departing from the teachings and disclosures of this disclosure. Therefore, the scope of this disclosure is not limited to the embodiments already disclosed, but is also included in the scope of any subsequent claims, including other modifications and alterations that do not depart from this disclosure. [Explanation of symbols]

[0109] 1. Live Streaming System 10 servers 20 User Terminals 100 Streaming Units 102 Video Control Unit 104 Audio Control Unit 106 distribution units 108 UI Control Unit 200 viewing units 202 UI Control Unit 204 rendering units 206 Input Units 210 Processing Units 212 Terminal Gift Database 214 Configuration DB 30, 30a, 30b User terminals 302 Distribution Information Unit 304 Relay Unit 306 Processing Units 320 Stream DB 322 User DB 324 Gift DB 40 equipment 402 Receiving Unit 406 Processing Units 410 Transmitter Unit 600 screens 602 Object 604 images 606 Message Zone 608 Message Input Box 610 Gift Objects 612 Shared Objects 900 Information Processing Equipment 901 CPU 903 ROM 905 RAM 907 Host Bus 909 Bridge 911 External bus 913 Interface 915 Input Unit 917 Output Unit 919 Storage Units 921 Drive 923 Removable recording media 925 connection ports 927 External connection terminal 929 Communication Unit LS Live Streaming LV Live Streamer NW Network SP specific part AU1, AU2 viewers S502~S512, S552~S558 process VD, VD1, VD2 video

Claims

1. A terminal including a circuit, wherein the circuit is The process of receiving interaction data from viewers in the live streaming room, A step of detecting a first condition from the interaction data, A step of transmitting a first command for a first operation to a first device in response to the detection of a first condition, A terminal characterized by being configured to perform a certain action.

2. A step of detecting a second condition from the aforementioned interaction data, The process involves transmitting a second command for a second operation to a second device in response to the detection of a second condition, The terminal according to claim 1, further comprising the first operation being the same as or different from the second operation, and the first device being the same as or different from the second device.

3. A step of detecting a second condition from the aforementioned interaction data, A step of transmitting a second command for a second operation to the first device, The terminal according to claim 1, further comprising the first operation being the same as the second operation, but with different characteristics, attributes, intensity, or genre.

4. The first condition is that the gifts or purchase revenue received within the live streaming room exceeds a threshold, or that a specific event occurs during the interaction of multiple users. The first device is a wearable device worn by the user of the terminal or a device located near the user. The terminal according to claim 1, characterized in that

5. A step of detecting the virtual position of the first device on the screen of the terminal, A step of adding a virtual effect on or near the virtual position of the first device on the screen, The terminal according to claim 1, further comprising:

6. The process includes transmitting information related to the terminal to the first device, The first operation of the first device is performed based on the information of the terminal, The information further includes the physical location of the terminal. The terminal according to claim 1, characterized in that

7. A step of detecting whether or not the virtual image of the first device is displayed on the screen of the terminal, The steps include: displaying a message on the terminal in response to the failure to detect the virtual image; The terminal according to claim 1, further comprising:

8. A device including a circuit, wherein the circuit is The process includes receiving a first command for a first operation in response to the detection of a first condition from interaction data within a live streaming room, A step of triggering the first operation based on the first instruction, A device characterized by being configured to perform the following.

9. The process of receiving information about the terminal, Based on the aforementioned information, the process involves triggering the first operation, The apparatus according to claim 8, further comprising the information including the physical location of the terminal.

10. It is a method, The process of receiving interaction data from viewers in the live streaming room, A step of detecting a first condition from the interaction data, A step of transmitting a first command for a first operation to a first device in response to the detection of a first condition, A method characterized by including

Images