Information processing device, information processing method, and information processing program

Abstract

To provide more appropriate reactions, and to generate and provide content with reactions added. [Solution] The information processing device according to the present invention is characterized by comprising: an acquisition unit that acquires video from a user; a video provision unit that provides the acquired video to a viewer; an imaging control unit that captures the viewer's reaction to the video and acquires a video of the reaction; a processing unit that generates reaction content including the acquired video and the acquired video of the reaction; and a reaction content provision unit that provides the reaction content.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In recent years, there is a technology for providing various videos via the Internet. In this type of technology, there is a technology in which a user who is viewing a video posts a reaction to the content of the video (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above conventional technology only posts reactions in text, and there is still room for further improvement in providing more appropriate reactions. In addition, there is also room for improving further charm that is not present in contents such as videos with conventional text-based reactions.

[0005] The present application has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program that can provide more appropriate reactions and impart new charm to contents such as videos by providing those reactions.

Means for Solving the Problems

[0006] The information processing device according to the present application is characterized by comprising: an acquisition unit that acquires moving images from a user; a moving image provision unit that provides the acquired moving images to a viewer; an imaging control unit that captures the viewer's reaction to the moving images and acquires moving images of the reaction; a processing unit that generates reaction content including the acquired moving images and the acquired moving images of the reaction; and a reaction content provision unit that provides the reaction content. [Effects of the Invention]

[0007] According to one embodiment, it is possible to provide reactions more appropriately and to generate and provide content with reactions attached. [Brief explanation of the drawing]

[0008] [Figure 1A] Figure 1A shows the first process performed by the information processing system according to the embodiment. [Figure 1B] Figure 1B shows the second process performed by the information processing system according to the embodiment. [Figure 2] Figure 2 shows an example of the configuration of a terminal device according to this embodiment. [Figure 3] Figure 3 shows an example of the configuration of a server device according to this embodiment. [Figure 4] Figure 4 is a flowchart showing the processing procedure for processing step 1 executed by the information processing system according to the embodiment. [Figure 5] Figure 5 is a flowchart showing the processing procedure for processing step 2 executed by the information processing system according to the embodiment. [Figure 6] Figure 6 shows an example of a hardware configuration. [Modes for carrying out the invention]

[0009] The following describes in detail, with reference to the drawings, embodiments for implementing the information processing device, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments"). Note that these embodiments do not limit the information processing device, information processing method, and information processing program according to the present application. Furthermore, the same parts are denoted by the same reference numerals in the following embodiments, and redundant descriptions are omitted.

[0010] [1. Regarding Information Processing Systems] [1-1. Processing Part 1] Referring to Figures 1A and 1B, an overview of the information processing system according to the embodiment will be described. Figure 1A is an explanatory diagram showing the first process executed by the information processing system according to the embodiment.

[0011] The first process shown in Figure 1A will now be described. As shown in Figure 1A, the information processing system 1 according to this embodiment includes a terminal device 10 and a server device 100. The terminal device 10 and the server device 100 are connected to each other via a network N, either by wire or wireless, enabling communication. This allows the terminal device 10 to cooperate with the server device 100. The network N is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

[0012] Terminal device 10 is an information processing device used by user U. For example, terminal device 10 may be a smart device such as a smartphone or tablet, a PC (Personal Computer) such as a desktop or notebook (laptop), a mobile phone such as a feature phone, a PDA (Personal Digital Assistant), a game console or AV equipment with communication functions, an information appliance or digital appliance, a car navigation system, a wearable device such as a smartwatch, head-mounted display, or smart glasses. Alternatively, terminal device 10 may be a house or building compatible with the Internet of Things (IoT), a car, a home appliance, an electronic device, etc.

[0013] In this embodiment, the terminal device 10 is a smart device such as a smartphone or tablet used by user U, and is a mobile terminal device that can communicate with any server device via wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), Bluetooth (registered trademark), or wireless LAN. The terminal device 10 also has a screen such as a liquid crystal display with touch panel functionality, and accepts various operations on displayed data such as content from user U using a finger or stylus, such as tapping, sliding, and scrolling. Operations performed on the area of ​​the screen where content is displayed may also be considered as operations on the content. Furthermore, the terminal device 10 may be an information processing device such as a desktop PC or notebook PC, not just a smart device.

[0014] The server device 100 is, for example, a computer such as a PC or blade server, or a mainframe or workstation. The server device 100 may also be implemented through cloud computing.

[0015] In this embodiment, the server device 100 is an information processing device that cooperates with the terminal device 10 of each user U and provides the terminal device 10 of each user U with API (Application Programming Interface) services for various applications (hereinafter referred to as apps) and the like, as well as various data, and is realized by a computer, a cloud system, or the like.

[0016] Further, the server device 100 may be an information processing device that provides some online service to the terminal device 10 of each user U. For example, as an online service, the server device 100 may provide services such as Internet connection, search service, chat service, dialogue service by voice, image, video, etc., SNS (Social Networking Service), e-commerce (EC: Electronic Commerce), electronic payment, online game, online banking, online trading, accommodation and ticket reservation, video and music distribution, news, map, route search, route guidance, route information, operation information, weather forecast, etc. Actually, the server device 100 may cooperate with various servers that provide the above online services and mediate the online services, or be responsible for the processing of the online services.

[0017] Note that the server device 100 can acquire user information regarding the user U. For example, as user information, the server device 100 acquires information (attribute information) regarding the attributes of the user U such as the gender, age, and residential area of the user U. Further, the server device 100 can acquire information regarding attributes such as the demographics (demographic attributes), psychographics (psychological attributes), geographics (geographical attributes), and behavioral (behavioral attributes) of the user U. Further, the server device 100 may acquire, as user information, the segment or persona (persona) to which the user U belongs in the field of marketing. Then, the server device 100 stores and manages the information (attribute information) regarding the attributes of the user U together with the identification information (such as user ID) indicating the user U.

[0018] In addition, the server device 100 acquires various types of history information (log data) indicating the actions of the user U from the terminal device 10 of the user U or from various servers or the like based on the user ID or the like. For example, the server device 100 acquires a location history, which is a history of the location and time of the user U, from the terminal device 10. In addition, the server device 100 acquires a search history, which is a history of search queries input by the user U, from a search server (search engine). In addition, the server device 100 acquires a browsing history, which is a history of the content browsed by the user U, from a content server. In addition, the server device 100 acquires a purchase history (settlement history), which is a history of the user U's product purchases and settlement processes, from an e-commerce server or a settlement processing server. In addition, the server device 100 may acquire a listing history or a sales history, which is a history of the user U's listings on the marketplace, from an e-commerce server or a settlement processing server. In addition, the server device 100 acquires a posting history, which is a history of the user U's posts, from a posting server or an SNS server that provides a word-of-mouth posting service. Note that each of the above-mentioned various servers or the like may be the server device 100 itself. That is, the server device 100 may function as each of the above-mentioned various servers or the like.

[0019] In addition, the number of each device included in the information processing system 1 shown in FIG. 1A is not limited to that shown. For example, in FIG. 1A, only one terminal device 10 is shown for simplicity of illustration, but this is merely an example and is not limited, and two or more may be used.

[0020] Note that the server device 100 may implement the mechanism of the reaction content SNS using an AI (Artificial Intelligence) such as GPT (Generative Pre-trained Transformer). GPT is a text generation AI and a language model capable of generating sentences using natural language processing.

[0021] In addition, when not distinguishing between a poster and a viewer, which will be described later, it is described as a user.

[0022] For example, as shown in Figure 1A, the server device 100 acquires video from the terminal device 10 (10A) of user U (U1), who is the poster, via the network N (step S1). The server device 100 receives video (posted content) from the user's (U1) terminal device 10 (10A) and acquires the received video. At this time, the server device 100 also obtains from the poster the destination of the video. The destination may be, for example, friends or the world. The server device 100 may also acquire correspondence information from the terminal device 10 (10A) that associates video with reactions. Correspondence information is, for example, information that associates video with reactions. For example, correspondence information is information that sets how to accept viewer reactions desired by the poster at a predetermined scene or time in the video. For example, the server device 100 receives requests from users for a certain type of reaction at a predetermined scene in the video. The server device 100 then associates the reaction desired by the user with the reaction of the viewer. Hereafter, this setting will be referred to as the "reaction reception setting," and the configured information will be referred to as the "reaction reception setting information." Figure 1 shows an example where the recipient of the video is selected as a friend.

[0023] Next, the server device 100 provides the video to the terminal device 10 (10B) of user U (U2), who will be a viewer, via the network N (step S2). The server device 100 provides the video to the viewer's terminal device 10 (10B) according to the destination specified by the poster's (user U1) terminal device 10. At this time, the server device 100 performs mosaic processing on the video, displaying it as a mosaic until playback begins, and provides the video with the mosaic processing reflected. The server device 100 may also perform hiding processing (e.g., masking) on ​​the video, or it may not perform any processing at all. Furthermore, it may accept the same processing from the poster in the reaction reception setting. In addition, along with displaying the video to the viewer's terminal device 10 (10B), the server device 100 displays a notice (such as a warning) indicating that a reaction will be captured when the video is played. Furthermore, a notice (such as a warning) indicating that reactions will be recorded may be displayed before the viewer receives the video (for example, "Do you want to receive a video that will record reactions?") or immediately before playing the video (for example, before pressing the play button). Subsequently, if the recipient is a friend, the server device 100 provides the video to the selected friend. Also, if the recipient of the video is worldwide, the server device 100 provides the video to users of reaction-enabled content SNS (described later) around the world, including domestically.

[0024] Next, the server device 100 captures the viewer's reaction via the network N and acquires the captured viewer's reaction (step S3). The server device 100 controls the viewer's terminal device 10 (10B) to capture the viewer's reaction (when viewing the video) during video playback and acquires the captured video of the reaction. At this time, the server device 100 may also control the viewer's terminal device 10 (10B) to record the viewer's voice during the reaction and acquire the recorded voice of the reaction.

[0025] Next, the server device 100 generates reaction content (step S4). The server device 100 generates reaction content that includes the acquired video and the acquired reactions. The server device 100 generates a wipe of the reaction. The server device 100 generates content with reactions (hereinafter referred to as reaction content) that includes the acquired video and the wipe. The server device 100 generates reaction content by placing the wipe around the video.

[0026] Next, the server device 100 provides reaction content via the network N (step S5). The server device 100 may also provide the reaction content to user U (U1), or to the world. The server device 100 may also provide the reaction content to viewers who have reacted (e.g., user U2). It may also provide it to the user's followers. Here, followers are users who follow the user who has received the video on the reaction content SNS (described later).

[0027] The length of the video is preferably around 30 seconds or 15 seconds. The length of the reaction is also preferably the same as the video, or shorter. That is, the server device 100 generates reaction content, including a reaction video synchronized with a short video, and provides it to the poster and the world. The video may be longer than 30 seconds. Furthermore, the server device 100 may acquire multiple reactions in the poster's reaction reception settings and generate reaction content with multiple reaction videos attached to a single video at different timings. In this case, the video may be a long video.

[0028] The server device 100 acquires video footage, provides the acquired video footage to the viewer along with reaction reception settings, and acquires the viewer's reactions. The server device 100 then generates reaction content and provides the generated reaction content to the poster. This allows the server device 100 to provide reactions to video footage more appropriately. Furthermore, the server device 100 allows the poster to select where the video footage is provided. Therefore, the server device 100 can provide reactions from friends or people all over the world. In other words, the poster can understand the reactions to video footage more appropriately.

[0029] [1-2. Processing Part 2] Next, we will describe the second process shown in Figure 1B. Figure 1B is an explanatory diagram showing the second process executed by the information processing system according to the embodiment. As shown in Figure 1B, the information processing system 1 according to the embodiment includes a server device 100, a terminal device 10, and an advertiser terminal 200. The terminal device 10, the server device 100, and the advertiser terminal 200 are connected to each other via a network N, either by wire or wireless means, enabling communication. Note that the information processing system 1 shown in Figure 1B may include multiple terminal devices 10, multiple server devices 100, and multiple advertiser terminals 200. The server device 100 and the terminal device 10 are the same as in the first process, so we will omit their explanation.

[0030] The advertiser terminal 200 is a terminal device used by an advertiser or an agent acting on their behalf (hereinafter referred to as "advertiser, etc." as appropriate). For example, the advertiser terminal 200 may be a smart device such as a smartphone or tablet, a PC (Personal Computer), a mobile phone, a PDA (Personal Digital Assistant), or office automation equipment. The advertiser terminal 200 also submits advertising content to the server device 100 according to the advertiser's instructions.

[0031] For example, the advertiser terminal 200 submits advertising content, such as still images, moving images, or text data, to the server device 100.

[0032] Furthermore, the advertiser terminal 200 instructs the server device 100 to set a target cost per conversion (CPC) for the corresponding advertising content, which is the cost to acquire one result that leads to a profit, such as a product purchase or membership registration, from the advertising content, according to the advertiser's instructions. Here, a conversion refers to a user who has viewed the advertising content taking an action that the advertiser expects (performing a predetermined action). For example, the conversions set for an advertisement can vary widely depending on the advertiser's settings, such as clicking the advertisement, staying on the landing page, requesting information, answering a survey, requesting a quote, requesting a sale, or purchasing a product (contract completion). In this embodiment, the advertising content is a video advertisement. Therefore, in this embodiment, the conversions set for an advertisement include the playback of the video advertisement when the advertisement is clicked.

[0033] For example, as shown in Figure 1B, the server device 100 acquires video advertisements from advertiser terminals 200, such as advertisers, via the network N (step S1). The server device 100 receives video advertisements from advertiser terminals 200 and acquires the received video advertisements. At this time, the server device 100 also acquires the recipients of the video advertisements from the advertisers. The recipients are, for example, viewers requested by the advertisers. For example, the recipients (viewers) requested by advertisers include not only general viewers but also influential people (influencers). The server device 100 acquires request information, which is information about the viewers requested by advertisers. Request information includes, for example, user attributes (e.g., teenagers), history information, and influencer information. Note that Figure 1B shows a case where the recipients of the video advertisements are not specified.

[0034] Furthermore, the server device 100 acquires reward information. The server device 100 acquires reward information set by advertisers, etc., for video advertisements from the advertiser terminal 200. The reward is preferably, for example, a cashback in response to viewer reactions, but it may also be the awarding of points or the provision of services corresponding to the video advertisement. The server device 100 also acquires correspondence information that associates video advertisements with reactions. Correspondence information is information that associates video advertisements with reactions. For example, correspondence information is information that the poster has set the desired reaction at a predetermined scene or time in the video.

[0035] Next, the server device 100 provides a video advertisement to, for example, the terminal device 10 of a user U who will be a viewer, via the network N (step S2). Based on the request information obtained from the advertiser terminal 200 of the advertiser, the server device 100 provides the video advertisement to the viewer's terminal device 10. For example, the server device 100 provides the video advertisement to viewers and influencers who match the user attributes specified by the advertiser. If the server device 100 has not obtained (specified) the recipient of the video advertisement request, it provides the video advertisement to users of the global reaction-enabled content SNS (described later). At this time, the server device 100 also applies a mosaic effect to the video advertisement, displaying it as a mosaic until it is played, and provides it to the viewer's terminal device 10. The server device 100 may also apply a hiding effect (for example, a masking effect) to the video image in addition to the mosaic effect, or it may choose not to apply any processing at all. The server device 100 may also accept this processing from the poster in the reaction reception settings. Furthermore, the server device 100 displays a video advertisement to the viewer's terminal device 10 along with a message indicating that a reaction will be captured when the video advertisement is played. Note that the message regarding the capture of a reaction (a warning, etc.) may be displayed before the viewer receives the video (for example, "Do you want to receive a video that will capture your reaction?") or immediately before the video is played (for example, before the play button is pressed).

[0036] Next, the server device 100 captures the viewer's reaction via the network N and acquires the captured viewer's reaction (step S3). The server device 100 controls the viewer's terminal device 10 to capture the viewer's reaction (after viewing the video ad) during video ad playback and acquires the captured video of the reaction. At this time, the server device 100 may also control the viewer's terminal device 10 to record the viewer's reaction audio and acquire the recorded audio of the reaction. The server device 100 obtains consent from the viewer to capture their reaction during or before the playback of the video ad.

[0037] The server device 100 generates reaction content (step S4). The server device 100 generates reaction content that includes the acquired video ad and the acquired reaction. The server device 100 generates a wipe of the reaction. The server device 100 generates reaction content that includes the acquired video ad and the wipe. The server device 100 generates reaction content by placing the wipe around the video ad.

[0038] The server device 100 provides reaction content (step S5). The server device 100 provides the reaction content to the advertiser's advertiser terminal 200.

[0039] Furthermore, the server device 100 may provide reaction content to parties other than advertisers. In this case, consent has been obtained from the viewer (their terminal device 10) to provide the reaction content to the world.

[0040] The server device 100 sets the reward (step S6). The server device 100 sets the reward according to the content of the reaction. The server device 100 sets at least a portion of the advertising cost of the video ad as the reward according to the content of the reaction.

[0041] For example, the server device 100 sets a reward for viewers based on reward information and correspondence information from the advertiser terminal 200. The server device 100 determines whether a reaction to a video ad matches the video ad and sets a reward for viewers who have made a matching reaction. For example, the server device 100 determines whether a predetermined reaction has been made based on the correspondence information. For example, the server device 100 determines what kind of reaction a viewer has made by inputting the reaction into a learned model. Then, the server device 100 sets a reward for viewers who have made a reaction that matches the correspondence information based on the reward information.

[0042] The video advertisement is preferably about 15 or 30 seconds long. The reaction is also preferably the same length as the video advertisement. That is, the server device 100 generates reaction content, including video reactions, that are timed to match a short video advertisement, and provides it to the advertiser or other users. The video may be longer than 30 seconds. Furthermore, the server device 100 may acquire multiple reactions in the poster's reaction reception settings and generate reaction content with multiple reaction videos attached to a single video at different timings.

[0043] [2. Social media with reaction-enabled content] In this embodiment, the server device 100 provides a reaction-enabled content SNS to the user U's terminal device 10 via the network N. Hereinafter, reaction-enabled content will be referred to as reaction content, and reaction-enabled content SNS will be referred to as reaction content SNS.

[0044] Reaction content SNS is a new type of social networking service that includes reactions in its content. In reaction content SNS, the content posted is, for example, video, video advertisements, etc. (hereinafter, these will be collectively referred to as "video"). Reaction content SNS captures the reactions of users who view the video and provides a new type of SNS platform that realizes reaction-enabled content that includes both the video and the captured reactions, or it provides a function that adds the same content to existing SNS (an additional function that makes existing SNS a content-in-stage disclosure type), thus offering a different value from existing SNS.

[0045] [2-1. Imaging of Reactions] The server device 100 may request consent from viewers to record their reactions. The server device 100 may also request consent from viewers to record their reactions. For example, the server device 100 may provide a display requesting consent to record reactions to viewers viewing the video for the first time. The server device 100 does not need to provide a display requesting consent to record reactions to viewers who have recorded their reactions multiple times in a row. For example, the server device 100 does not need to display a display requesting consent to record reactions to viewers who have recorded their reactions multiple times (e.g., five or more times) in a row.

[0046] The server device 100 captures the viewer's reaction once. For example, the server device 100 may display a message on the terminal device 10 informing the viewer that they are prohibited from reviewing the video, so that the viewer cannot capture their reaction again.

[0047] The server device 100 may re-capture the viewer's reaction. For example, the server device 100 may capture a second reaction depending on the content of the video or video advertisement. In this case, it is preferable that the content of the video or video advertisement includes things that cannot be noticed without careful attention. For example, the server device 100 may capture a reaction again after it has finished capturing the first reaction.

[0048] The server device 100 may process the viewer's reaction. Specifically, the server device 100 may provide processing options to the viewer's terminal device 10 (10B), process the reaction according to the processing option selected by the viewer, and obtain a reaction that reflects the processing option. The processing option may be, for example, a function to make the skin appear smoother.

[0049] [2-2. Variations in displaying reaction wipes] The server device 100 controls the display of reaction wipes. The server device 100 displays multiple reaction wipes in reaction content. It is preferable for the server device 100 to display multiple wipes simultaneously. For example, the server device 100 displays reaction wipes for up to six people. The server device 100 may also change the size of the wipes. The server device 100 may change the size of the wipes according to the reactions. For example, the server device 100 may extract the wipes of viewers with large reactions and display them larger. Alternatively, for example, the server device 100 may extract viewers who have the same reaction when playing video or advertising videos and display the wipes of those viewers. If there are six or more viewers who have captured reactions, the server device 100 may allow the poster or advertiser to switch the wipes of viewers who have captured reactions in response to their operation (e.g., swiping) of the reaction content.

[0050] Furthermore, the server device 100 may group and display viewers who have the same reaction. For example, the server device 100 may set the placement positions of wipes according to the reaction (e.g., joy, anger, sadness, surprise, etc.), and then re-display the wipes corresponding to the set placement positions. For example, the server device 100 may gather and display the wipes of viewers who have the same reaction at the designated placement positions. In addition, the server device 100 may surround the wipes of viewers who have the same reaction. Preferably, the server device 100 surrounds the wipes of viewers who have made the same (similar) reaction based on the response information set by the poster or advertiser.

[0051] Furthermore, the server device 100 may control the timing of providing the wipe. For example, the server device 100 may display the wipe in accordance with the scene of the video or video advertisement. The server device 100 may detect and display a reaction before a predetermined scene begins. Alternatively, the server device 100 may detect and display a reaction immediately after a predetermined scene ends. For example, the server device 100 may detect changes in the viewer's facial expression, etc., and control the timing of providing the wipe.

[0052] If the server device 100 captures a second reaction, it may display both the first and second reactions. The server device 100 may also display two side-by-side wipes of viewers who have viewed the video twice in areas where the video contains information that cannot be found without paying close attention to the video advertisement.

[0053] The server device 100 may display multiple captured reactions at different timings within a single video. For example, the server device 100 may display multiple reactions at different timings within a long video of 30 seconds or more.

[0054] [2-3. Acceptance of evaluations and follow-ups] Server device 100 may accept user ratings for videos and video advertisements. Server device 100 may also accept ratings for reactions. Server device 100 may accept ratings for reaction content. Server device 100 may also accept user follows not only for the poster but also for viewers who reacted. Examples of ratings include "like," "amazing," "funny," and "sad."

[0055] [2-4. Other variations] It is preferable for the server device 100 to provide audio of the viewer's reaction. In this case, the server device 100 may process the audio so that the reaction is audible. For example, the server device 100 may extract sounds such as laughter, "amazing," or "cute," and provide the extracted audio. If, for example, the reaction audio overlaps, the server device 100 may display speech bubbles or subtitles that transcribe the audio.

[0056] The server device 100 may capture images of reactions to reaction content. The server device 100 captures the reactions of users viewing the reaction content. The server device 100 associates the user's reactions with the reaction content, generates reaction reference content, and provides (displays) it. It is preferable for the server device 100 to display both the reaction content and the reaction reference content that associates the user's reactions. For example, the server device 100 may display the reaction content and the reaction reference content side by side. The server device 100 may also be able to switch between displaying the reaction content and displaying the reaction reference content.

[0057] Furthermore, the server device 100 recommends (provides) videos and reaction content of viewers' reactions as "recommendations" to users who are likely to be interested in such content. For example, the server device 100 may estimate the user's desired emotion from the user's browsing history and prioritize providing videos and reaction content of reactions that correspond to the estimated emotion. Alternatively, the server device 100 may estimate videos and reaction content of reactions of viewers who have reacted, based on the user's actions toward the reactions of viewers who have reacted, and provide such content. Actions may include the user's evaluation of the viewer who reacted, such as giving a "like," following the viewer, commenting, or viewing. The server device 100 may also provide videos and reaction content of reactions that have been frequently reacted to by other users with similar acquired user attributes as "recommendations." In this case, the server device 100 may acquire user attributes when the terminal device 10 captures the viewer's reaction and provide videos and reaction content of the reaction based on those user attributes.

[0058] The server device 100 may provide only the viewer's reaction when the viewer views the video advertisement. For example, the server device 100 may provide only the viewer's reaction based on the request information. For example, the server device 100 provides the viewer with a video advertisement based on the request information and obtains the viewer's reaction to the video advertisement. The server device 100 then generates a wipe of the viewer's reaction and provides the generated wipe.

[0059] The server device 100 provides the number of reactions. The server device 100 counts the number of reactions and provides the number of reactions. For example, the server device 100 counts the number of reactions made by the viewer (the viewer who made the reaction) and provides the total number of such reactions. For example, the server device 100 counts the number of reactions attached to reaction content and provides the total number of such reactions. For example, the server device 100 counts the number of reactions attached to a reaction reference video and provides the total number of such reactions. The server device 100 also counts and provides the number of views of the reaction content to which the viewer made the reaction. The server device 100 counts the number of views of the video of the reaction the viewer made and provides the number of views.

[0060] The server device 100 may notify the poster or advertiser each time a reaction is made to a video or video advertisement.

[0061] [2-5. Target Users and Value Proposition] The target users of reaction content on social media include young people, companies, and influential figures (influencers).

[0062] The value that reaction content SNS provides to users includes the discovery of content they are interested in and a sense of community participation. General users have a desire to find high-quality content that matches their interests. Reaction content SNS allows them to effectively discover content they are interested in. In addition, the mechanism that only people who have "liked" the content can see promotes the formation of communities with people who have similar interests. Furthermore, general users are tired of information overload and irrelevant posts on SNS and have a need to filter information. With reaction content SNS, only content that the user has actively participated in is displayed, making it easier to filter information.

[0063] Furthermore, one of the values ​​that reaction content on social media offers to companies is the ability to see more authentic user reactions. Companies want honest reactions to the services and products they offer. By reviewing reaction content, including reactions from viewers who have seen video ads, companies can easily grasp the real evaluation of their services and products. As a result, companies can improve their services and products based on the evaluations they receive, and provide high-quality services and products to users. In addition, companies can provide video ads to influencers and make it easier to deliver products and services to users by generating reaction content. Moreover, general users and influencers who view video ads may receive rewards (such as cashback or points), making it easier for companies to make their products and services known to a wider audience.

[0064] Other advantages include a new SNS experience and high-quality interaction. Reaction content SNS offers a new experience where users can enjoy live reactions to videos and images. Both also share a need for truly meaningful interaction. In reaction content SNS, content disclosure based on interests promotes deeper interaction among users with shared interests. There is also a pain point of dissatisfaction with the shallow relationships and superficial interactions on SNS. In reaction content SNS, active participation in the content is required, making it easier to create deeper relationships and meaningful interactions.

[0065] [2-6. Challenges of existing social networking services and advantages of social networking services with phased content disclosure] The following explains the challenges of existing social networking services (SNS) and the advantages of SNS with a phased content disclosure model.

[0066] (Problem 1) Information overload and low-quality content In existing social media platforms, high-quality content easily gets buried amid a large volume of posts. In a content-gradual disclosure social media platform, the mosaic function allows users to "like" and view only the content they are interested in. As a result, high-quality content naturally stands out.

[0067] (Challenge 2) Quality of engagement Existing social networking services (SNS) tend to have many superficial "likes," making it difficult for deeper interactions to occur. Reaction-based SNS, however, encourages active user participation through the gradual release of content. This makes it easier for deeper interactions to develop among users with shared interests.

[0068] (Challenge 3) Bias due to algorithms Existing social media platforms often have recommendation algorithms that tend to narrow users' perspectives. Reaction-based social media platforms, however, allow users to discover content based on their own interests, resulting in less algorithmic bias. Furthermore, they provide an environment where users can naturally discover new interests and perspectives.

[0069] (Challenge 4) Freshness of the user experience Existing social networking services (SNS) have become monotonous in terms of user experience. A reaction-based SNS will offer a new experience where users can see the reactions of viewers who have watched videos and images.

[0070] [3. Example of terminal device configuration] Next, the configuration of the terminal device 10 will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the terminal device 10 according to the embodiment. As shown in Figure 2, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40.

[0071] (Communications Section 11) The communication unit 11 is connected to the network N by wire or wireless connection and transmits and receives information to and from the server device 100 via the network N. For example, the communication unit 11 can be implemented using a NIC (Network Interface Card) or an antenna.

[0072] (Display section 12) The display unit 12 is a display device that displays various information such as location information. For example, the display unit 12 may be a liquid crystal display (LCD) or an organic electro-luminescent display (OLED). The display unit 12 may also be a touch panel display, but is not limited to this.

[0073] (Input section 13) The input unit 13 is an input device that receives various operations from the user U. For example, the input unit 13 has buttons for inputting characters, numbers, etc. The input unit 13 may also be an input / output port (I / O port) or a USB (Universal Serial Bus) port. If the display unit 12 is a touch panel display, a part of the display unit 12 functions as the input unit 13. The input unit 13 may also be a microphone that receives voice input from the user U. The microphone may be wireless.

[0074] (Positioning unit 14) The positioning unit 14 receives signals (radio waves) transmitted from GPS (Global Positioning System) satellites and, based on the received signals, acquires position information (e.g., latitude and longitude) indicating the current position of the terminal device 10. In other words, the positioning unit 14 determines the position of the terminal device 10. Note that GPS is just one example of a GNSS (Global Navigation Satellite System).

[0075] Furthermore, the positioning unit 14 can determine its position using various methods other than GPS. For example, the positioning unit 14 may use various communication functions of the terminal device 10 to determine its position as an auxiliary positioning means for position correction, etc., as described below.

[0076] (Wi-Fi positioning) For example, the positioning unit 14 determines the location of the terminal device 10 by utilizing the Wi-Fi® communication function of the terminal device 10 and the communication network provided by each telecommunications company. Specifically, the positioning unit 14 determines the location of the terminal device 10 by performing Wi-Fi communication, etc., and determining the distance to nearby base stations and access points.

[0077] (Beacon positioning) Furthermore, the positioning unit 14 may determine the location using the Bluetooth® function of the terminal device 10. For example, the positioning unit 14 determines the location of the terminal device 10 by connecting to a beacon transmitter connected via the Bluetooth® function.

[0078] (Geomagnetic positioning) Furthermore, the positioning unit 14 determines the position of the terminal device 10 based on the geomagnetic pattern of the structure, which has been measured in advance, and the geomagnetic sensor provided by the terminal device 10.

[0079] (RFID positioning) Furthermore, if, for example, the terminal device 10 is equipped with an RFID (Radio Frequency Identification) tag function equivalent to that of a contactless IC card used at a train station ticket gate or in a store, or if it is equipped with a function to read RFID tags, the location where it was used will be recorded along with the information on the payment or other transactions made by the terminal device 10. The positioning unit 14 may determine the location of the terminal device 10 by acquiring such information. Alternatively, the location may be determined by an optical sensor or infrared sensor equipped in the terminal device 10.

[0080] The positioning unit 14 may, if necessary, determine the position of the terminal device 10 using one or a combination of the positioning means described above.

[0081] (Sensor unit 20) The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. The connection can be wired or wireless. For example, the sensors may be detection devices other than the terminal device 10, such as wearable devices or wireless devices. In the example shown in Figure 2, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, a barometric pressure sensor 23, a temperature sensor 24, a sound sensor 25, a light sensor 26, a magnetic sensor 27, and an image sensor (camera) 28.

[0082] The sensors 21-28 described above are merely examples and not limiting. In other words, the sensor unit 20 may be configured to include some of the sensors 21-28, or it may include other sensors such as humidity sensors in addition to or instead of the sensors 21-28.

[0083] The acceleration sensor 21 is, for example, a 3-axis acceleration sensor and detects the physical movement of the terminal device 10, such as its direction of movement, velocity, and acceleration. The gyro sensor 22 detects the physical movement of the terminal device 10, such as its tilt in the three axes, based on its angular velocity. The barometric pressure sensor 23 detects the atmospheric pressure around the terminal device 10, for example.

[0084] Since the terminal device 10 is equipped with the acceleration sensor 21, gyroscope 22, barometric pressure sensor 23, etc., it becomes possible to determine the position of the terminal device 10 using technologies such as pedestrian dead-reckoning (PDR) that utilize these sensors 21 to 23. This makes it possible to obtain indoor location information that is difficult to obtain with positioning systems such as GPS.

[0085] For example, a pedometer using an accelerometer 21 can calculate the number of steps, walking speed, and distance walked. Additionally, a gyroscope 22 can be used to determine the user U's direction of movement, gaze direction, and body tilt. Furthermore, the barometric pressure detected by the barometric pressure sensor 23 can be used to determine the altitude and floor number of the user U's terminal device 10.

[0086] The temperature sensor 24 detects, for example, the ambient temperature around the terminal device 10. The sound sensor 25 detects, for example, ambient sound around the terminal device 10. The light sensor 26 detects the ambient illumination around the terminal device 10. The magnetic sensor 27 detects, for example, the Earth's magnetic field around the terminal device 10. The image sensor 28 is, for example, a camera and captures moving images of the surroundings of the terminal device 10. These moving images include images of a user of the terminal device 10 viewing the display unit 12 of the terminal device 10. Preferably, the image sensor 28 is, for example, an in-camera that captures images of the user of the terminal device 10.

[0087] The aforementioned pressure sensor 23, temperature sensor 24, sound sensor 25, light sensor 26, and image sensor 28 can detect the surrounding environment and conditions of the terminal device 10 by detecting atmospheric pressure, temperature, sound, and illuminance, respectively, and by capturing images of the surroundings. Furthermore, it becomes possible to improve the accuracy of the location information of the terminal device 10 based on the surrounding environment and conditions.

[0088] (Control Unit 30) The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit) or MPU (Micro Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), input / output ports, and various circuits. Alternatively, the control unit 30 may be composed of hardware such as an integrated circuit (ASIC) or FPGA (Field Programmable Gate Array). The control unit 30 includes a transmission unit 31, a reception unit 32, and a processing unit 33.

[0089] (Transmitter 31) The transmission unit 31 can transmit various information, such as information input by the user U using the input unit 13, various information detected by sensors 21-28 mounted on or connected to the terminal device 10, and location information of the terminal device 10 determined by the positioning unit 14, to the server device 100 via the communication unit 11.

[0090] (Receiving unit 32) The receiving unit 32 can receive various information provided by the server device 100, as well as requests for various information from the server device 100, via the communication unit 11.

[0091] (Processing 33) The processing unit 33 controls the entire terminal device 10, including the display unit 12. For example, the processing unit 33 can output and display various information transmitted by the transmission unit 31 and various information received from the server device 100 by the reception unit 32 to the display unit 12.

[0092] (Storage unit 40) The storage unit 40 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), and optical discs. Various programs and various data are stored in this storage unit 40.

[0093] [4. Example of Server Device Configuration] Next, the configuration of the server device 100 according to the embodiment will be described using Figure 3. Figure 3 is a diagram showing an example of the configuration of the server device 100 according to the embodiment. As shown in Figure 3, the server device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0094] (Communications Department 110) The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to the network N by wire or wireless connection.

[0095] (Storage unit 120) The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as HDDs, SSDs, and optical discs. The storage unit 120 may store identification information (such as a user ID) indicating user U, as well as attribute information and history information (log data) of user U.

[0096] (Control unit 130) The control unit 130 is a controller, and is realized by various programs (corresponding to an example of an information processing program) stored in the internal storage device of the server device 100, such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array), executing them using a storage area such as RAM as the working area. In the example shown in Figure 3, the control unit 130 includes an acquisition unit 131, a video provision unit 132, an imaging control unit 133, a processing unit 134, a reaction content provision unit 135, and a setting unit 136.

[0097] (Acquisition part 131) The acquisition unit 131 acquires the search query entered by the user U. For example, when the user U enters a search query into a search engine or the like and performs a keyword search, the acquisition unit 131 acquires the search query via the communication unit 110. In other words, the acquisition unit 131 acquires the keyword entered by the user U into the search box of a search engine, website, or application via the communication unit 110.

[0098] Furthermore, the acquisition unit 131 acquires user information about user U via the communication unit 110. For example, the acquisition unit 131 acquires identification information (such as user ID), location information, and attribute information of user U from user U's terminal device 10. The acquisition unit 131 may also acquire identification information and attribute information of user U when user U is registered. The acquisition unit 131 then stores the user information in the storage unit 120.

[0099] Furthermore, the acquisition unit 131 acquires various historical information (log data) indicating the user U's actions via the communication unit 110. For example, the acquisition unit 131 acquires various historical information indicating the user U's actions from the user U's terminal device 10, or from various servers based on the user ID, etc. The acquisition unit 131 then stores the various historical information in the storage unit 120.

[0100] Furthermore, the acquisition unit 131 acquires video from the user via the communication unit 110. The acquisition unit 131 receives video (posted content) from the user U's terminal device 10 and acquires the received video. At this time, the acquisition unit 131 also acquires the destination of the video from the poster. The acquisition unit 131 then stores the video and the video destination in the storage unit 120. The acquisition unit 131 also acquires reaction reception setting information.

[0101] Furthermore, the acquisition unit 131 acquires video advertisements from advertisers, etc., via the communication unit 110. The acquisition unit 131 acquires video advertisements from advertiser terminals 200 of advertisers, etc. The acquisition unit 131 receives video advertisements from advertiser terminals 200 and acquires the received video advertisements. The acquisition unit 131 also acquires viewer request information requested by advertisers, etc. Then, the acquisition unit 131 stores the video advertisements and request information in the storage unit 120.

[0102] Furthermore, the acquisition unit 131 acquires compensation information related to compensation. The acquisition unit 131 acquires compensation information set by advertisers, etc., for video advertisements from the advertiser terminal 200. The acquisition unit 131 then stores the compensation information in the storage unit 120. The acquisition unit 131 also acquires correspondence information that associates moving images or video advertisements with reactions. The acquisition unit 131 then stores the correspondence information in the storage unit 120.

[0103] Furthermore, the acquisition unit 131 may acquire user attributes when the terminal device 10 captures the viewer's reaction.

[0104] Furthermore, the acquisition unit 131 may acquire reactions multiple times in the poster's reaction acceptance settings.

[0105] (Video / Image Provider Section 132) The video provision unit 132 provides the acquired video to viewers. For example, the video provision unit 132 provides the video to the viewer's terminal device 10 via the communication unit 110, according to the recipient information obtained from the uploader's terminal device 10.

[0106] Furthermore, the video provision unit 132 provides the acquired video advertisements to viewers. For example, the video provision unit 132 provides the video advertisement to the terminal device 10 of user U, who is a viewer, via the communication unit 110.

[0107] The video / image provider 132 may request the viewer's consent to record their reactions. For example, the video / image provider 132 may provide a message requesting consent to record reactions. The video / image provider 132 does not need to provide a message requesting consent to record reactions to viewers who have recorded reactions multiple times in a row.

[0108] Furthermore, the video / image provision unit 132 provides video advertisements to viewers based on the requested information.

[0109] Furthermore, the video provisioning unit 132 performs mosaic processing on the video, displaying it as a mosaic until playback begins, and provides the video with the mosaic processing applied. The video provisioning unit 132 may also perform hiding processing (e.g., masking) on ​​the video in addition to mosaic processing, or it may choose not to perform any processing at all. It may also accept requests for this processing from the poster through the reaction reception settings.

[0110] (Imaging control unit 133) The imaging control unit 133 captures the viewer's reaction to the video and acquires a video of the reaction. For example, the imaging control unit 133 controls the viewer's terminal device 10 via the communication unit 110 to capture the viewer's reaction and acquires a video of the captured viewer's reaction. The imaging control unit 133 controls the image sensor 28 to cause the viewer's terminal device 10 to capture the viewer's reaction during video playback and acquires a video of the captured reaction. The imaging control unit 133 then stores the captured video of the reaction in the storage unit 120.

[0111] Furthermore, the imaging control unit 133 captures and acquires the reactions of viewers who have viewed the video advertisement. For example, the imaging control unit 133 controls the viewer's terminal device 10 via the communication unit 110 to capture the viewer's reactions and acquires video footage of the captured reactions. The imaging control unit 133 controls the image sensor 28 of the terminal device 10 to cause the viewer's terminal device 10 to capture the viewer's reactions when the video advertisement is played and acquires video footage of the captured reactions. The imaging control unit 133 then stores the captured video footage of the reactions in the storage unit 120.

[0112] Furthermore, the imaging control unit 133 controls the viewer's terminal device 10 to record the viewer's reactions and acquires the recorded reactions. The imaging control unit 133 controls the sound sensor 25 of the viewer's terminal device 10 to record the viewer's reactions and acquires the recorded reactions. The imaging control unit 133 then stores the reactions in the storage unit 120.

[0113] The imaging control unit 133 captures the viewer's reaction once. For example, the imaging control unit 133 may display a message on the terminal device 10 prohibiting the viewer from re-capturing the reaction, so that the viewer cannot capture the reaction again.

[0114] The image capture control unit 133 may re-capture the viewer's reaction. For example, the image capture control unit 133 may capture a second reaction depending on the content of the video or video advertisement. For example, the image capture control unit 133 may capture another reaction after it has finished capturing the first reaction.

[0115] The imaging control unit 133 may process the viewer's reaction. Specifically, the imaging control unit 133 may provide processing options to the viewer's terminal device 10, process the reaction according to the processing option selected by the viewer, and obtain a reaction that reflects the processing option.

[0116] The imaging control unit 133 may capture reactions to reaction content. The imaging control unit 133 captures the reactions of users viewing the reaction content and acquires the captured reactions.

[0117] The imaging control unit 133 captures and acquires the reactions of viewers who have given their consent to have their reactions to moving images or video advertisements captured.

[0118] (Processing Unit 134) Processing unit 134 generates reaction content. Processing unit 134 generates reaction content that includes the acquired video and the acquired reaction video. Processing unit 134 generates reaction content that includes the acquired video ad and the acquired reaction. Processing unit 134 generates a wipe of the reaction video. Processing unit 134 generates reaction content that includes a wipe of the reaction video that matches the length of the video and video ad. Processing unit 134 generates reaction content that includes a video of the viewer's reaction based on the request information.

[0119] The processing unit 134 may set the placement position of the wipe according to the reaction.

[0120] Furthermore, the processing unit 134 receives evaluations from users. The processing unit 134 also receives follow-up requests from users for viewers who have reacted to the content.

[0121] Processing unit 134 may process the audio so that the reaction sounds are audible. Processing unit 134 may also generate speech bubbles or subtitles that transcribe the audio.

[0122] Furthermore, the processing unit 134 associates the reactions of users viewing the reaction content with the reaction content itself, and generates reaction reference content.

[0123] The processing unit 134 estimates the user's desired emotions from the user's history information (browsing history).

[0124] Furthermore, the processing unit 134 counts the number of reactions. For example, the processing unit 134 counts the number of reactions made by the viewer who made the reaction. The processing unit 134 also counts the number of views of the video of the reaction made by the viewer.

[0125] The processing unit 134 may, for example, generate reaction content in which multiple reaction videos are added at different timings within a single video in a long video.

[0126] The processing unit 134 may generate reaction content based on the reaction reception setting information acquired by the acquisition unit 131.

[0127] (Reaction Content Provider 135) The reaction content provider unit 135 provides reaction content generated by the processing unit 134. The reaction content provider unit 135 provides reaction content including video footage of reactions matched to the length of the video footage or video advertisement. The reaction content provider unit 135 provides the generated reaction content to the poster's terminal device 10. The reaction content provider unit 135 provides the generated reaction content to the advertiser's advertiser terminal 200. The reaction content provider unit 135 provides reaction content including video footage of viewers' reactions based on the request information. The reaction content provider unit 135 provides the reaction content to the world.

[0128] Furthermore, the reaction content provider 135 controls the display of reaction wipes. The reaction content provider 135 displays multiple reaction wipes in the reaction content simultaneously. The reaction content provider 135 may also change the size of the wipes. The reaction content provider 135 may change the size of the wipes according to the reaction. For example, the reaction content provider 135 displays a larger wipe for viewers with large reactions.

[0129] The reaction content provider 135 may extract and display viewers who have the same reaction based on the playback time of the video or video advertisement. The reaction content provider 135 may also group and display viewers who have the same reaction. For example, the reaction content provider 135 may reposition and redisplay reaction wipes according to the set placement positions. Furthermore, the reaction content provider 135 may surround the wipes of viewers who have the same reaction. The reaction content provider 135 may accept operations (e.g., swiping) from users or advertisers viewing reaction content and switch the wipes of viewers whose reactions have been captured according to the operations.

[0130] The reaction content provider 135 may control the timing of providing wipes. For example, the reaction content provider 135 may display wipes in accordance with scenes in moving images or video advertisements. The reaction content provider 135 may detect and display reactions before a predetermined scene begins. Alternatively, the reaction content provider 135 may detect and display reactions immediately after a predetermined scene ends.

[0131] The reaction content provider 135 may display both the first and second reactions if it captures a second reaction. The reaction content provider 135 may also display two side-by-side wipes of a viewer who has viewed a section twice, starting from a point that contains information that would not be easily noticed without close attention.

[0132] The reaction content provider unit 135 provides video and digital images of reactions made by viewers who have been rated or followed by the user. The reaction content provider unit 135 also provides reaction content that includes the video and digital images of such reactions.

[0133] The reaction content provider 135 preferably provides audio of the viewer's reaction. In this case, the reaction content provider 135 may provide audio extracted by the processing unit 134. If audio cannot be provided, the reaction content provider 135 may display speech bubbles or subtitles that transcribe the audio, as determined by the processing unit 134.

[0134] The reaction content provider 135 displays reaction reference content associated with user U's reaction. Preferably, the reaction content provider 135 displays both the reaction content and the reaction reference content associated with the user's reaction. The reaction content provider 135 may, for example, display the reaction content and the reaction reference content side by side. Alternatively, the reaction content provider 135 may allow the user (terminal device 10) to select which of the reaction content or the reaction reference content to display.

[0135] Furthermore, the reaction content provider unit 135 recommends reaction videos and reaction content as "recommendations" to users who are likely to be interested in such content. The reaction content provider unit 135 may also prioritize providing reaction videos and reaction content based on the emotions estimated by the processing unit 134. Alternatively, for example, the reaction content provider unit 135 may estimate reaction videos and reaction content in response to user actions and provide such content. For example, the reaction content provider unit 135 may provide reaction videos and reaction content that include reactions made by viewers rated or followed by the user. The reaction content provider unit 135 may also recommend reaction videos and reaction content that have been frequently reacted to by other users with similar user attributes. In this case, the reaction content provider unit 135 may acquire user attributes when the acquisition unit 131 captures the viewer's reaction on the terminal device 10, and provide reaction videos and reaction content based on those user attributes.

[0136] Furthermore, the reaction content provider 135 may provide only reactions. For example, the reaction content provider 135 may provide only viewer reactions based on the request information.

[0137] The reaction content provider unit 135 provides the number of reactions. The reaction content provider unit 135 provides the total number of reactions counted by the processing unit 134. The reaction content provider unit 135 also provides the number of views counted by the processing unit 134.

[0138] The reaction content provider unit 135 may notify the poster or advertiser each time a reaction is made to a video or video advertisement.

[0139] The reaction content provider 135 may, for example, provide multiple reactions at different timings within a single long video.

[0140] (Settings section 136) The settings unit 136 sets the reward. When a video ad is provided, the settings unit 136 sets the reward for the viewer based on the content of their reaction, based on the reward information. The settings unit 136 sets the advertising cost based on the content of the reaction.

[0141] [5. Processing Procedure] Next, the processing procedure by the server device 100 according to the embodiment will be described using Figures 4 and 5. Figure 4 is a flowchart of the processing procedure of process 1 executed by the server device 100 according to the embodiment. Figure 5 is a flowchart of the processing procedure of process 2 executed by the server device 100 according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 130 of the server device 100.

[0142] First, the processing procedure for process 1 will be explained using Figure 4. For example, as shown in Figure 4, the acquisition unit 131 of the server device 100 acquires video images from the poster along with reaction reception setting information (step S101).

[0143] Next, the video provision unit 132 of the server device 100 provides the acquired video to the viewer (step S102). At this time, the video provision unit 132 may perform processing such as mosaic processing that is displayed when the video is provided.

[0144] Next, the imaging control unit 133 of the server device 100 captures the viewer's reaction to the video after obtaining the viewer's consent, and acquires the captured reaction (step S103).

[0145] Next, the processing unit 134 of the server device 100 generates reaction content (step S104). At this time, the processing unit 134 may also extract audio or generate subtitles.

[0146] Next, the reaction content provision unit 135 of the server device 100 provides reaction content (step S105).

[0147] Next, the processing procedure for process 2 will be explained using Figure 5. For example, as shown in Figure 5, the acquisition unit 131 of the server device 100 acquires video advertisements along with corresponding information from advertisers, etc. (step S201).

[0148] Next, the video provision unit 132 of the server device 100 provides the acquired video advertisement to the viewer (step S202). At this time, the video provision unit 132 may perform processing such as mosaic processing that is displayed when the video is provided.

[0149] Next, the imaging control unit 133 of the server device 100 captures the reactions of viewers who have viewed the video advertisement with their consent, and acquires the captured reactions (step S203).

[0150] Next, the processing unit 134 of the server device 100 generates reaction content and sets rewards according to the reaction (step S204). When generating reaction content, the processing unit 134 may also extract audio or generate subtitles.

[0151] Next, the reaction content provision unit 135 of the server device 100 provides reaction content (step S205).

[0152] Next, the setting unit 136 of the server device 100 sets a reward according to the content of the reaction (step S206).

[0153] [6. Variant Example] The terminal device 10 and server device 100 described above may be implemented in various other forms besides those of the embodiment described above. Therefore, the following describes modifications of the embodiment.

[0154] In the above embodiment, some or all of the processing performed by the server device 100 may actually be performed by the terminal device 10 (or an application running on the terminal device 10). For example, the processing may be completed in a standalone manner (by the terminal device 10 alone). In this case, the terminal device 10 is assumed to have the same functions as the server device 100 in the above embodiment. Furthermore, in the above embodiment, since the terminal device 10 is in cooperation with the server device 100, from the perspective of the user U, it appears as if the processing of the server device 100 is also being performed by the terminal device 10. In other words, from another perspective, it can be said that the terminal device 10 is equipped with the server device 100.

[0155] Furthermore, in the above embodiment, the reaction content may be delivered via streaming. For example, the server device 100 may display viewer reactions during a live broadcast.

[0156] Furthermore, in the above embodiment, the reaction content SNS is not limited to SNS, but may also be a news site / news app, a video sharing service (video distribution site), an illustration / photo posting site, a novel posting site, etc. Also, the content may be advertising content. For example, it may be advertising content that promotes and announces new dramas, movies, anime, etc. It may also be advertising content that introduces products, services, events, etc. that attract the interest of users.

[0157] Furthermore, in the above embodiment, the reaction content SNS may post the generated reaction content not only to its own SNS, but also to existing SNS or other media.

[0158] Furthermore, in the above embodiment, when displaying only reactions, only reactions may also be displayed for videos and images provided by the poster.

[0159] Furthermore, in the above embodiment, the server device 100 may generate and provide rankings. The server device 100 may generate rankings of viewers who have reacted. The server device 100 may generate a ranking of popular reactions. The server device 100 may also generate and provide a ranking of reaction content.

[0160] Furthermore, in the above embodiment, the server device 100 builds a matching platform between influencers and advertisers. Advertisers can select specific influencers and develop advertising campaigns based on their reactions. This platform allows advertisers to effectively reach their target audience, and influencers can be matched with appropriate advertising products.

[0161] [7. Effects] As described above, the information processing device (terminal device 10 and server device 100) according to the present application is characterized by comprising: an acquisition unit 131 that acquires video from a user; a video provision unit 132 that provides the acquired video to a viewer; an imaging control unit 133 that captures the viewer's reaction to the video and acquires a video of the reaction; a processing unit 134 that generates reaction content including the acquired video and the acquired video of the reaction; and a reaction content provision unit 135 that provides the reaction content.

[0162] This allows for more appropriate reactions to be provided, and for content with reactions to be generated and delivered.

[0163] The processing unit 134 generates a wipe of the reaction video and generates reaction content that includes the wipe of the reaction video, which is matched to the length of the video.

[0164] This allows for more appropriate reactions to be provided, and for content with reactions to be generated and delivered.

[0165] The processing unit 134 generates a wipe of the reaction video and controls the timing of displaying the captured reaction in the wipe, according to the content of the video and the captured viewer.

[0166] This allows for the more appropriate presentation of video and the aforementioned reactions as a single reaction content.

[0167] The processing unit 134 receives feedback from the user regarding the reaction.

[0168] This allows for providing reactions tailored to the user's preferences.

[0169] The processing unit 134 receives evaluations of the reaction content from the user.

[0170] This allows for the provision of reaction content tailored to the user's preferences.

[0171] The processing unit 134 generates reaction content based on reception setting information that associates the reaction desired by user U with the reaction of the viewer.

[0172] This allows for the provision of reaction content tailored to the user's preferences.

[0173] Processing unit 134 processes the audio so that the reaction sounds are audible. Processing unit 134 also generates speech bubbles and subtitles that transcribe the audio.

[0174] This makes it easier to provide users with clear audio of reactions to reaction content.

[0175] The reaction content provider unit 135 displays multiple reaction video images in a wipe.

[0176] This allows for more appropriate responses.

[0177] The reaction content provider unit 135 changes the size of the reaction video wipe according to the reaction.

[0178] This allows for more appropriate responses.

[0179] The image capture control unit 133 captures a second reaction according to the content of the video. The reaction content provision unit 135 displays the first and second reactions.

[0180] The video output unit 132 performs a mosaic process on the video, displaying it as a mosaic until playback begins, and then provides the video with the mosaic process applied.

[0181] This increases viewers' expectations for the video content.

[0182] This allows users viewing reaction content to enjoy the reactions more.

[0183] The processing unit 134 extracts wipes from viewers with the same reaction. The reaction content provision unit 135 surrounds the wipes of people with the same reaction.

[0184] This makes it easier for users viewing reaction content to see other viewers who are reacting similarly, allowing them to enjoy the reaction content more.

[0185] The reaction content provider unit 135 surrounds the wipes of viewers with the same reaction based on correspondence information that associates the video and the reaction.

[0186] This makes it easier for users viewing reaction content to see other viewers who are reacting similarly, allowing them to enjoy the reaction content more.

[0187] The reaction content provider unit 135 controls the timing of providing the reaction video wipe.

[0188] This allows for more appropriate responses.

[0189] The reaction content provider unit 135 displays multiple reactions at different times within a single long video.

[0190] This allows for more appropriate responses.

[0191] The reaction content provider unit 135 receives user input and switches the reaction video wipe according to the input.

[0192] This allows you to switch the display of wipes if there are many wipes in the reaction video.

[0193] The processing unit 134 sets the placement position of the reaction video wipe according to the reaction. The reaction content providing unit 135 places the reaction video wipe at the designated position and redisplays it.

[0194] This allows for more appropriate responses.

[0195] The processing unit 134 receives requests from users to follow up on viewers who have reacted to them. The reaction content provision unit 135 provides reaction content, including the reactions made by the followed viewers.

[0196] This makes it easier for users to provide reactions from viewers they follow.

[0197] The imaging control unit 133 captures the reactions of users viewing reaction content and acquires video images of the captured user reactions. The processing unit 134 associates the reactions of users viewing reaction content with reaction content and generates reaction content. The reaction content provision unit 135 provides reaction reference content.

[0198] This will allow users to view the reactions of viewers who have viewed reaction content.

[0199] The acquisition unit 131 acquires the user's history information. The processing unit 134 estimates the user's emotions based on the history information. The reaction content provision unit 135 provides reaction content based on the estimated emotions.

[0200] This allows for more appropriate responses.

[0201] The reaction content provision unit 135 provides reaction content to users who become posters.

[0202] This allows for more appropriate responses.

[0203] Reaction Content Provision Department 135 provides reaction content to the world.

[0204] This allows us to obtain reactions from people around the world and provide more appropriate responses.

[0205] By any or a combination of the above-described processes, the information processing device according to the present invention can provide more appropriate reactions and add new appeal to content such as videos.

[0206] [8. Hardware Configuration] Furthermore, the terminal device 10 and server device 100 according to the above-described embodiment are realized by a computer 1000 having a configuration such as that shown in Figure 6. The following explanation will use the server device 100 as an example. Figure 6 is a diagram showing an example of the hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a primary storage device 1040, a secondary storage device 1050, an output interface 1060, an input interface 1070, and a network interface 1080 are connected by a bus 1090.

[0207] The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, as well as programs read from the input device 1020, and executes various processes. The arithmetic unit 1030 can be implemented using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array).

[0208] The primary storage device 1040 is a memory device, such as RAM (Random Access Memory), that temporarily stores data used by the arithmetic unit 1030 for various calculations. The secondary storage device 1050 is a storage device where data used by the arithmetic unit 1030 for various calculations and various databases are registered, and can be implemented using ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. The secondary storage device 1050 may be internal storage or external storage. The secondary storage device 1050 may also be a removable storage medium such as USB (Universal Serial Bus) memory or SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), NAS (Network Attached Storage), file server, etc.

[0209] The output I / F 1060 is an interface for transmitting information to be output to output devices 1010, such as displays, projectors, and printers, and is implemented using connectors of standards such as USB (Universal Serial Bus), DVI (Digital Visual Interface), and HDMI (High Definition Multimedia Interface). The input I / F 1070 is an interface for receiving information from various input devices 1020, such as mice, keyboards, keypads, buttons, and scanners, and is implemented using, for example, USB.

[0210] Furthermore, the output interface 1060 and input interface 1070 may be wirelessly connected to the output device 1010 and input device 1020, respectively. In other words, the output device 1010 and input device 1020 may be wireless devices.

[0211] Furthermore, the output device 1010 and the input device 1020 may be integrated as a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated as an input / output I / F.

[0212] The input device 1020 may also be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0213] The network interface 1080 receives data from other devices via network N and sends it to the computing unit 1030, and also transmits data generated by the computing unit 1030 to other devices via network N.

[0214] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output interface 1060 and the input interface 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0215] For example, when computer 1000 functions as a server device 100, the arithmetic unit 1030 of computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the primary storage device 1040. Alternatively, the arithmetic unit 1030 of computer 1000 may load a program obtained from another device via the network interface 1080 onto the primary storage device 1040 and execute the loaded program. Furthermore, the arithmetic unit 1030 of computer 1000 may cooperate with other devices via the network interface 1080 and call and use program functions, data, etc., from other programs on other devices.

[0216] [9. Other] Although embodiments of the present invention have been described above, the present invention is not limited by the content of these embodiments. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above.

[0217] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0218] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0219] For example, the server device 100 described above may be implemented using multiple server computers, and the configuration can be flexibly changed, such as by calling external platforms via APIs (Application Programming Interfaces) or network computing depending on the function.

[0220] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0221] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit," etc. For example, the acquisition unit can be replaced with acquisition means or acquisition circuit. [Explanation of Symbols]

[0222] 1. Information Processing System 10 Terminal devices 100 Server Devices 110 Communications Department 120 Storage section 130 Control Unit 131 Acquisition Department 132 Video and Image Provision Department 133 Imaging Control Unit 134 Processing Unit 135 Reaction Content Provider 136 Settings Section

Claims

1. An acquisition unit that acquires video images from the user, A video provision unit that provides the acquired video to the viewer, An imaging control unit captures the reaction of the viewer who viewed the aforementioned video, and acquires a video of the reaction. A processing unit that generates reaction content including the acquired video and the acquired video of the reaction, A reaction content provider unit that provides the aforementioned reaction content and An information processing device characterized by comprising:

2. The processing unit generates a wipe of the reaction video and generates reaction content that includes a wipe of the reaction video that matches the length of the video. The information processing apparatus according to feature 1.

3. The processing unit receives an evaluation of the reaction from the user. The information processing apparatus according to claim 1 or 2.

4. The processing unit receives evaluations of the reaction content from the user. The information processing apparatus according to claim 1 or 2.

5. The processing unit generates reaction content based on reception setting information that associates the reaction desired by the user with the reaction of the viewer. The information processing apparatus according to claim 1 or 2.

6. The reaction content provider displays multiple wipes of the reaction video. The information processing apparatus according to claim 1 or 2.

7. The reaction content provider unit changes the size of the reaction's video wipe in accordance with the reaction. The information processing apparatus according to claim 1 or 2.

8. The imaging control unit captures a second reaction according to the content of the video, The reaction content provider displays the first reaction and the second reaction. The information processing apparatus according to claim 1 or 2.

9. The processing unit extracts wipes from viewers with the same reaction, The reaction content provider unit surrounds the wipe of viewers who have the same reaction. The information processing apparatus according to claim 1 or 2.

10. The reaction content provider unit, based on correspondence information that associates video images with reactions, surrounds the wipes of viewers with the same reaction. The information processing apparatus according to feature 8.

11. The reaction content provider controls the timing of providing the reaction video wipe. The information processing apparatus according to claim 1 or 2.

12. The reaction content provider displays multiple reactions at different timings within a single long video. The information processing apparatus according to feature 11.

13. The reaction content provider unit receives the user's input and switches the wipe of the reaction video in accordance with the input. The information processing apparatus according to claim 1 or 2.

14. The processing unit sets the placement position of the wipe of the reaction video in accordance with the reaction, The reaction content provider unit then places a wipe of the reaction video at the designated location and redisplays it. The information processing apparatus according to claim 1 or 2.

15. The processing unit receives a request for follow-up from the viewer who made the reaction. The reaction content provider unit provides reaction content that includes reactions made by the followed viewers. The information processing apparatus according to claim 1 or 2.

16. The imaging control unit captures the reactions of users viewing the reaction content, and acquires video footage of the captured reactions of the users. The processing unit associates the video footage of the user's reaction to the reaction content with the reaction content to generate reaction reference content. The reaction content providing unit provides the reaction reference content. The information processing apparatus according to claim 1 or 2.

17. The acquisition unit acquires the user's history information, The processing unit estimates the user's emotions based on the historical information. The reaction content provider unit provides the reaction content based on the estimated emotion. The information processing apparatus according to claim 1 or 2.

18. The reaction content providing unit provides the reaction content to the user. The information processing apparatus according to claim 1 or 2.

19. The reaction content provision unit provides the reaction content to the world. The information processing apparatus according to claim 1 or 2.

20. An information processing method performed by an information processing device, The acquisition process involves obtaining video images from the user, A video provision step, which involves providing the acquired video to the viewer, A motion capture control step involves capturing the reaction of the viewer who viewed the aforementioned video, and acquiring a video of the reaction. A processing step for generating reaction content that includes the acquired video and the acquired video of the reaction, The reaction content provision process provides the aforementioned reaction content, An information processing method characterized by including

21. Procedure for obtaining video images from users, A procedure for providing the acquired video to the viewer, A motion capture control procedure for capturing the reaction of the viewer who viewed the aforementioned video, and acquiring a video of the reaction, A processing procedure for generating reaction content that includes the acquired video and the acquired video of the reaction, A procedure for providing the aforementioned reaction content, An information processing program characterized by causing a computer to execute it.

Images