Information processing system, control program and information processing device

Abstract

To improve an entertainment property while suppressing complexity in a mechanism for displaying a reaction of a viewer on a display screen of a viewing object.SOLUTION: According to one aspect of the present invention, an information processing system (for example, a group 5) includes: first control means (for example, an image display control part) for displaying a first element based on first operation by a first user being viewing a viewing object in a first information processing device (for example, a mobile terminal 3) in a first range at least partially overlapping a basic area for displaying the viewing object in the first information processing device; and second control means (for example, an image display control part) for displaying a second element based on second operation by a second user being viewing the viewing object in a second information processing device (for example, a mobile terminal 3) in a second range not overlapping the basic area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing system, a control program, and an information processing apparatus.

Background Art

[0002] There is known a mechanism for allowing viewers of video content being distributed to have a group chat about the content of the video. For example, Patent Document 1 discloses a client device that displays a fixed message regarding the content of video content on a chat screen in which avatars corresponding to users participating in the group chat are arranged around the video content according to buttons selected by the users.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The client device described in Patent Document 1 displays a fixed message selected by the user in a speech bubble beside the avatar representing the user. As a result, although the user can confirm the reactions of other users watching the video content, simply displaying the fixed message selected by the user in a speech bubble lacks entertainment value.

[0005] Also, Patent Document 1 discloses displaying speech bubbles in sizes corresponding to the selection frequencies of fixed messages by a plurality of users. However, if the fixed messages of other users are displayed large, users watching the video content may find this annoying.

[0006] The problem to be solved by the present invention is to improve the entertainment value while suppressing annoyance in a mechanism for displaying the viewer's reaction on the display screen of the viewing target.

Means for Solving the Problem

[0007] To solve the above problems, the present invention makes it possible to superimpose a first element based on the operation of the person himself on the screen on which the viewing target is displayed, while making it impossible to superimpose a second element based on the operation of others on the viewing target.

[0008] An information processing system according to an aspect of the present invention includes first control means for displaying a first element based on a first operation by a first user who is viewing the viewing target in a first information processing apparatus in a first range that at least partially overlaps a basic area in which the viewing target is displayed in the first information processing apparatus, and second control means for displaying a second element based on a second operation by a second user who is viewing the viewing target in a second information processing apparatus in a second range that does not overlap the basic area.

Effects of the Invention

[0009] According to the present invention, it is possible to improve the entertainment value while suppressing annoyance in a mechanism for displaying the viewer's reaction on the display screen of the viewing target.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0011] [1. Embodiment] [1-1. Main Features] The present invention relates to a mechanism for displaying a viewer's reaction on a display screen of an object to be viewed. In order to improve entertainment while suppressing annoyance, each embodiment according to the present invention displays a first element based on a first operation by a first user who is viewing the object to be viewed in a first range that at least partially overlaps with a basic area where the object to be viewed is displayed, and a second element based on a second operation by a second user who is viewing the object to be viewed in a second range that does not overlap with the basic area.

[0012] The object to be viewed refers to video content that can be at least visually grasped by a user. The video content may include audio content that can be aurally grasped by a user. The video content is, for example, distributed from a distribution device to an information processing device. For the distribution of the video content, for example, streaming distribution or progressive download distribution is suitable. The video content is, for example, a video. The object to be viewed is displayed in a basic area that is a part of the screen of the information processing device.

[0013] A user (viewer) who views the object to be viewed can add a predetermined element (additional element) to the object to be viewed by a predetermined operation (additional operation). The predetermined operation is, for example, an input element It is a pointing operation on an (operation element). The input element is, for example, a button displayed on the screen. The pointing operation is, for example, a combination of a touch operation on a touch panel and a positioning operation and a click operation of a pointer displayed within the screen. The predetermined element is, for example, an image representing the user's reaction to the viewing target. The user's reaction is, for example, the user's emotion. An image representing the user's emotion towards the viewing target may be called an emotion image. The emotion image is, for example, an image in which onomatopoeia (also referred to as onomatopoeic words, onomatopoeic sounds, mimetic words, etc.) indicating the user's emotion is designed. An image in which onomatopoeia is designed may be called an onomatopoeia image. Adding an element to the viewing target means associating the display timing of the element with the progress of the viewing target. When the display timing is associated with the progress, the display timing is specified at a specific position on the timeline. The element added to the viewing target can be displayed on the screen at the associated display timing during the progress of the viewing target (i.e., during playback display).

[0014] The first user is the user (the person himself / herself) who uses the information processing device. A predetermined operation (the first operation) by the first user is, for example, received by the information processing device. The second user is another user (another person) who uses another information processing device. A predetermined operation (the second operation) by the second user is, for example, received by the other information processing device. A predetermined element (the first element) based on the first operation is displayed in a first range that at least partially overlaps the basic area. A predetermined element (the second element) based on the second operation is displayed in a second range that does not overlap the basic area. The first range may have a portion (the first portion) that does not overlap the basic area in addition to the portion (the second portion) that overlaps the basic area. The first range may further have a portion (the third portion) that overlaps the second range.

[0015] [1-2. Addendum] The present invention at least includes the following forms from [A] to [D] below.

[0016] 〔A〕An information processing system comprising: first control means for displaying a first element based on a first operation by a first user who is viewing the viewing target in a first range that at least partially overlaps a basic area where the viewing target is displayed in the first information processing apparatus; and second control means for displaying a second element based on a second operation by a second user who is viewing the viewing target in a second range that does not overlap the basic area in a second information processing apparatus.

[0017] 〔B〕A control program that causes a computer of an information processing apparatus included in an information processing system including a plurality of information processing apparatuses used by a plurality of users who are viewing a viewing target to realize: a first control function for displaying a first element based on a first operation by a user who is viewing the viewing target in a first range that at least partially overlaps a basic area where the viewing target is displayed; and a second control function for displaying a second element based on a second operation on another information processing apparatus by another user who is viewing the viewing target in a second range that does not overlap the basic area. Note that the control program may be non-temporarily recorded on a recording medium.

[0018] 〔C〕An information processing apparatus comprising: reception means for receiving a first operation by a first user who is viewing a viewing target; first control means for displaying a first element based on the first operation received by the reception means in a first range that at least partially overlaps a basic area where the viewing target is displayed; acquisition means for acquiring information on a second operation by a second user who is viewing the viewing target from management means that manages the information on the second operation; and second control means for displaying a second element based on the second operation acquired by the acquisition means in a second range that does not overlap the basic area.

[0019] 〔D〕A reception step of receiving a first operation by a first user while viewing an object to be viewed, and a first control step of displaying a first element based on the first operation received in the reception step in a first range that at least partially overlaps a basic area where the object to be viewed is displayed; an acquisition step of acquiring information on a second operation by a second user while viewing the object to be viewed from management means for managing information on the second operation; and a second control step of displaying a second element based on the second operation acquired in the acquisition step in a second range that does not overlap the basic area. A control method for an information processing apparatus, comprising:

[0020] The "information processing system" in the above [A] may be further limited technically as follows. Similarly, the same technical limitations may be added to the "control program" in the above [B], the "information processing apparatus" in the above [C], and the "control method for an information processing apparatus" in the above [D], respectively.

[0021] (1) The first information processing apparatus further includes reception means for receiving the first operation via an operation element arranged at a position that does not overlap the basic area. Thereby, the element based on the operation received by the information processing apparatus becomes the first element.

[0022] (2) The first control means displays the first element in the first range having a first part that does not overlap either the basic area or the second range and a second part that overlaps the basic area while changing the display mode from the first part toward the second part. The display mode is, for example, size, shape, position, and the number of components. The first element displayed while changing the display mode from the first part toward the second part changes from a state of not overlapping the object to be viewed to a state of overlapping the object to be viewed. Thereby, the entertainment property is further improved.

[0023] (3) The first control means causes the first element to be displayed in the first range, which further has a third part overlapping the second range, while changing the display mode from the first part, through the second part, towards the third part. The display mode is, for example, size, shape, position, and the number of components. The first element displayed while changing the display mode from the first part, through the second part, towards the third part changes from a state where it does not overlap the viewing object to a state where it overlaps the viewing object, and also changes from a state where it cannot overlap the second element to a state where it can overlap the second element. Thereby, the entertainment property is further improved.

[0024] (4) The first control means causes the first element to be displayed in the first range while changing the display mode so as to extend starting from the vicinity of the operation element arranged in the first part. Thereby, the entertainment property is further improved.

Embodiment

[0025] [2. Embodiment] Hereinafter, embodiments of the present invention will be described with reference to the drawings. This embodiment relates to a mechanism for displaying an image (image representing a reaction) representing the emotion of a user (viewer) on the display screen of a moving image (viewing object). In this embodiment, the information processing apparatus will be described as a mobile terminal, a so-called smartphone. Note that the mobile terminal is not limited to a smartphone, and may be a tablet terminal, a notebook personal computer, or the like. Further, the information processing apparatus may be a desktop personal computer. Each program according to the embodiment is stored in the information processing apparatus, but is not limited thereto, and may be stored in a storage medium such as an optical disk such as a CD-ROM (Compact Disk Read Only Memory) or a portable semiconductor memory such as a USB (Universal Serial Bus) flash memory.

[0026] [2-1. Features] In this embodiment, in order to improve the entertainment while suppressing annoyance, in a range (first range) that partially overlaps with the distribution video display area (basic area), a first image (additional element, first element) based on an additional operation (first operation) by a first user (the user himself / herself) who is watching the video is extended and displayed (displayed while changing the display mode) starting from the vicinity of an input element (operation element) arranged in the first image display area (first part) toward the second image display area (third part) via the distribution video display area (second part) from the first image display area, and in a range (second range) that does not overlap with the distribution video display area, a second image (additional element, second element) based on an additional operation (second operation) by a second user (another person) who is watching the video is displayed.

[0027] Also, in this embodiment, in order to efficiently specify an element to be added to the video by a viewer of the video, while the video and the input element (operation element) are being displayed on the screen, a touch operation (additional operation) for displaying a first image (additional element) representing the emotion (reaction) of the user (the user himself / herself) with respect to the video on the screen is received, and when the touch operation is received, the image is extended and displayed (displayed while changing the display mode) on the screen so that the longer the long-press time (duration of the additional operation), the larger the display.

[0028] [2-2. Video distribution system] The mobile terminal according to the embodiment is included in the video distribution system, and a video is distributed from the server.

[0029] [2-2-1. Overall configuration of the video distribution system] FIG. 1 is a schematic configuration diagram of a video distribution system 1 according to the embodiment. The video distribution system 1 includes a communication line 2, a plurality of mobile terminals 3, and a server 4.

[0030] The communication line 2 forms a computer network and is, for example, a wide-area communication line provided by an electric utility company.

[0031] The mobile terminal 3 is an information processing terminal such as a smartphone, a tablet terminal, or a notebook computer. The mobile terminal 3 includes a touch panel display 3a for displaying images, a speaker 3b for outputting sound, a microphone 3c for inputting sound, a camera 3d for imaging a subject, and the like. For example, in order to display a video distributed from the server 4, a dedicated application software (hereinafter referred to as "video viewing application") is installed in the mobile terminal 3 as an example.

[0032] The server 4 is an information processing device that distributes content to a plurality of mobile terminals 3 via the communication line 2. The content according to the embodiment is, for example, a video. The video has a story such as an animation or a drama, and the story progresses along the story, or a music video, a live video of music, various sports videos, etc., and is not particularly limited. Note that the video according to the embodiment is, for example, an animation. In addition, the content distributed from the server is not limited to a video, and other services such as a set of continuous still images may be used as long as the user can view them.

[0033] Furthermore, the viewing here means that the user recognizes the content by at least one of vision and hearing. In addition to the above, the content may be, for example, a video without sound or a sound without an image.

[0034] The plurality of mobile terminals 3 to which the video is distributed are registered with the server 4 as a group 5 as an example (hereinafter referred to as "group registration"). The server 4 distributes the video to the group-registered mobile terminals 3 at the same time. That is, the plurality of group-registered mobile terminals 3 display the video distributed from the server 4 on the touch panel display 3a at the same timing, in other words, synchronously. As a result, each user of the group-registered mobile terminals 3 will view the video at the same time.

[0035] Note that the video may be distributed at a predetermined date and time, or at any timing for each Group 5. When the video is distributed at any timing, for example, the host mobile terminal 3 makes a video distribution request to the server 4, and the server 4 that has received the distribution request distributes the video to the mobile terminals 3 that constitute Group 5.

[0036] In FIG. 1, four mobile terminals 3 that constitute Group 5 are shown, but this is just an example, and the number of mobile terminals 3 that constitute Group 5 is not limited as long as Group 5 is composed of a plurality of mobile terminals 3. Also, the types of the mobile terminals 3 that constitute Group 5 are not limited, and smartphones, tablet terminals, notebook personal computers, etc. may be mixed.

[0037] [2-2-2. Configuration of Server] FIG. 2 is a block diagram showing the electrical configuration of the server 4 according to the embodiment.

[0038] The server 4 according to the embodiment includes a CPU (Central Processing Unit) 20 which is a main control unit that controls the operation of the entire server 4, a ROM (Read Only Memory) 22 in which various programs and various data are stored in advance, a RAM (Random Access Memory) 24 used as a work area when the CPU 20 executes various programs, and an HDD (Hard Disk Drive) 26 as a storage means for storing various programs and various data.

[0039] The HDD 26 stores information regarding the registered Group 5, videos (video data) to be distributed to the mobile terminals 3, and other various information. Note that the storage means is not limited to the HDD 26, and may be other storage media such as semiconductor memories such as flash memories.

[0040] Furthermore, the server 4 includes an operation input unit 28 composed of a keyboard, a mouse, etc. for receiving inputs of various operations, a monitor 30 such as a liquid crystal display device for displaying various images, and is connected to other information processing devices such as the mobile terminal 3 via a communication line 2, and has an external interface 32 for transmitting and receiving various data to and from other information processing devices etc.

[0041] These CPU 20, ROM 22, RAM 24, HDD 26, operation input unit 28, monitor 30, and external interface 32 are electrically connected to each other via a system bus 34. Therefore, the CPU 20 can respectively perform access to the ROM 22, RAM 24, and HDD 26, grasp the operation state of the operation input unit 28, display images on the monitor 30, and transmit and receive various data to and from other information processing devices etc. via the external interface 32.

[0042] [2-2-3. Electrical Configuration of Mobile Terminal] FIG. 3 is a functional block diagram showing the electrical configuration of the mobile terminal 3. In addition to the configuration shown in FIG. 1, the mobile terminal 3 includes a main control unit 40, a main memory unit 42, an auxiliary storage unit 44, a communication unit 46, and operation buttons 48.

[0043] The main control unit 40 is, for example, a CPU, a microprocessor, a DSP (Digital Signal Processor), etc., and controls the overall operation of the mobile terminal 3.

[0044] The main memory unit 42 is composed of, for example, a RAM or a DRAM (Dynamic Random Access Memory), etc., and is used as a work area etc. when executing processing based on various programs by the main control unit 40.

[0045] The auxiliary storage unit 44 is a non-volatile memory such as a flash memory, and stores various data such as images and programs used for the processing of the main control unit 40. The programs stored in the auxiliary storage unit 44 are, for example, an OS (Operating System) for realizing the basic functions of the mobile terminal 3, drivers for controlling various hardware, programs for realizing functions such as e-mail, web browsing, and other various functions. Further, the auxiliary storage unit 44 stores in advance application software such as a video viewing application for playing back a video distributed from the server 4 and displaying it on the touch panel display 3a.

[0046] The communication unit 46 is, for example, a NIC (Network Interface Controller), and has a function of connecting to the communication line 2. Note that the communication unit 46 may have a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), a function enabling short-range wireless communication such as Bluetooth (registered trademark), and infrared communication, instead of or together with the NIC.

[0047] The operation buttons 48 are provided on the side surface of the mobile terminal 3, and are a power button for starting or stopping the mobile terminal 3, a volume adjustment button for the sound output from the speaker 3b, and the like.

[0048] These main control unit 40, main storage unit 42, auxiliary storage unit 44, communication unit 46, touch panel display 3a, speaker 3b, microphone 3c, camera 3d, and operation buttons 48 are electrically connected to each other via the system bus 49. Therefore, the main control unit 40 can access the main storage unit 42 and the auxiliary storage unit 44, display an image on the touch panel display 3a, grasp the operation state of the touch panel display 3a and the operation buttons 48 by the user, input sound to the microphone 3c, output sound from the speaker 3b, control the camera 3d, and access various communication networks and other information processing devices via the communication unit 46.

[0049] [2-3. Onomatopoeia display] While the mobile terminal 3 according to the embodiment displays the video distributed from the server 4 on the touch panel display 3a, it is also possible to display an image (onomatopoeia image, details of which will be described later) according to the user's operation. In the following description, the function of displaying the onomatopoeia image on the touch panel display 3a is referred to as the onomatopoeia display function, and the touch panel display 3a is also referred to as the screen 3a.

[0050] As an example, the onomatopoeia display function is one of the functions of the video viewing application according to the embodiment. Note that, as an example, onomatopoeia display is not possible for all the videos displayed on the screen 3a using the video viewing application, and the videos for which onomatopoeia display is possible are predetermined.

[0051] [2-3-1. Screen configuration] FIGS. 4 to 6 show the display modes of the screen 3a of the mobile terminal 3 on which the video viewing application according to the embodiment is launched. On the screen 3a, a distributed video display area 50A, a first image display area 50B, and a second image display area 50C are set.

[0052] The distributed video display area 50A is an area for displaying the video distributed from the server 4. The first image display area 50B is an area for displaying the first image 51 (see FIG. 5) according to the user's operation. The second image display area 50C is an area for displaying the second image 52 (see FIG. 6) based on the first image 51 displayed on another mobile terminal 3 by the operation of another user. Note that, as will be described in detail later, the second image 52 is a reduced image of the first image 51 displayed in the first image display area 50B of another mobile terminal 3. Also, as an example, the first image display area 50B and the second image display area 50C according to the embodiment are provided at positions facing each other with the distributed video display area 50A in between.

[0053] Here, the first image 51 is an image including an emotion image indicating the emotion of a user who is watching a video. The emotion image according to the embodiment is, as an example, an image in which onomatopoeia (also referred to as mimetic words or mimetic expressions) indicating the emotion of the user is designed, in other words, an onomatopoeia image. Examples of onomatopoeia images include "Suggee", "Yabai", "Bravoh", etc. Note that the design means, for example, that the characters indicating onomatopoeia are drawn decoratively.

[0054] In addition, the first image 51 may include, in addition to the onomatopoeia image, character images indicating characters appearing in the video, dialogue images indicating the lines of the characters, texts input by the user himself / herself, and various other images.

[0055] Also, the first image 51 may vary according to the scene of the video. For example, according to a happy scene, a sad scene, a confrontation scene between characters, etc., onomatopoeia images or character images that the user is likely to select while empathizing with the scene may be displayable.

[0056] Also, the displayable first image 51 may vary depending on the user. For example, the types of displayable first image 51 may increase by obtaining specific items through services such as games related to the video being distributed. Also, for example, the types of displayable first image 51 may increase on the condition of bearing a predetermined consideration (for example, monetary value such as currency, virtual currency, points, etc.).

[0057] In the first image display area 50B according to the embodiment, a plurality of display buttons 53A to 53D, which are operation units operated by the user and correspond to different first images 51, are displayed. Thereby, the user can display the first image 51 in the first image display area 50B by operating the display buttons 53A to 53D while watching the video. Note that the operation of the display buttons 53A to 53D is, for example, touching (pressing) the display buttons 53A to 53D with an indicator. The indicator according to the embodiment is, for example, the user's finger. In the following description, when the display buttons 53A to 53D are not distinguished for explanation, the notation of the last alphabet is omitted.

[0058] In the examples of FIGS. 4 to 6, four display buttons 53A to 53D are displayed in a horizontal row at the lower part of the first image display area 50B. The display button 53A corresponds to an onomatopoeic image in which "suggee" is designed, the display button 53B corresponds to an onomatopoeic image in which "yabai" is designed, and the display button 53C corresponds to an onomatopoeic image in which "bravo" is designed. Further, the display button 53D corresponds to a character image. Note that the display button 53 has a design in which the corresponding onomatopoeic image, character image, etc. can be intuitively recognized by the user.

[0059] In this way, by displaying a plurality of display buttons 53 in the first image display area 50B, the user can easily display the first image 51 that best suits his or her reaction (emotion) to the video on the screen 3a. Further, since the first image 51 includes an onomatopoeic image indicating the user's emotion, the user can display his or her emotion after watching the video on the screen 3a.

[0060] Note that in the examples of FIGS. 4 to 6, four types of display buttons 53 are displayed in the first image display area 50B, but this is only an example, and the number and types of the display buttons 53 are not limited to this. Further, the display buttons 53 may be appropriately switched and displayed so as to be able to display different first images 51 according to the scene of the video or the like.

[0061] Also, the arrangement of the display buttons 53 is just an example. For example, they may be collectively displayed on the right or left side of the first image display area 50B, or may be displayed in two horizontal rows at the bottom of the first image display area 50B. Further, for example, by the user swiping the lower part of the screen 3a to the left or right, display buttons 53 for displaying different types of first images 51 may be newly displayed.

[0062] [2-3-2. Display of the First Image] Referring to FIG. 5, the display mode of the first image 51 will be described in detail. In the example of FIG. 5, the display button 53C is operated by the user, and an onomatopoeia image in which "BRAVO" is schematized is displayed on the screen 3a.

[0063] The onomatopoeia display function according to the embodiment accepts a user operation for displaying an onomatopoeia image (first image 51) on the screen 3a while a video is being displayed on the screen 3a. That is, while watching the video, the user operates the display button 53 to display an onomatopoeia image corresponding to the emotion felt while watching the video on the screen 3a.

[0064] And the onomatopoeia image according to the embodiment is displayed on the screen 3a on which the video is being displayed while changing according to the operation mode of the display button 53 by the user. As an example, the onomatopoeia image according to the embodiment is displayed so as to extend from the first image display area 50B to the distribution video display area 50A according to the operation mode of the display button 53 by the user.

[0065] In this way, the user can change the onomatopoeia image displayed on the screen 3a by operating the mobile terminal 3 while watching the video. Therefore, the user can easily display the emotion felt while watching the video on the screen while watching the video.

[0066] In the example of FIG. 5, the onomatopoeia image is displayed so as to overlap the distribution video display area 50A beyond the first image display area 50B. As described above, the onomatopoeia display function according to the embodiment overlaps the onomatopoeia image with the distribution video display area 50A beyond the first image display area 50B according to the user's operation mode. Therefore, the user can more easily empathize with the video being viewed, and the entertainment value of viewing the video is further enhanced.

[0067] In FIG. 5, the onomatopoeia image also overlaps the second image display area 50C beyond the distribution video display area 50A. However, this is just an example, and the onomatopoeia image may be made non-overlappable with the second image display area 50C.

[0068] Next, the details of the onomatopoeia image that change according to the operation mode of the display button 53 will be described.

[0069] As an example, the display mode of the first image 51 according to the embodiment changes according to the duration of the user's operation of the display button 53. The duration of the operation of the display button 53 is, for example, the time during which the display button 53 is continuously pressed (hereinafter referred to as the "long press time").

[0070] When the user presses the display button 53, the onomatopoeia display function according to the embodiment starts displaying the onomatopoeia image starting from the vicinity of the position of the display button 53. The display of the onomatopoeia image starting from the vicinity of the position of the display button 53 means that, for example, the onomatopoeia image is displayed such that the lower end of the onomatopoeia image substantially coincides with the position of the display button 53.

[0071] Then, the onomatopoeia image extends (hereinafter referred to as "extended display") from the first image display area 50B toward the distribution video display area 50A according to the long press time with respect to the display button 53.

[0072] The stretching display is, for example, one in which the onomatopoeia image is displayed in a long and thin or enlarged manner. Note that the stretching display may be one in which the onomatopoeia image stretches while changing its form, such as the number of characters or the types of characters. For example, when the onomatopoeia image is "BRAVO", it may stretch so that "A" increases between "BRA" and "BO" according to the long-press time, or it may stretch so that "O" increases between "BRAVO" and "-" according to the long-press time.

[0073] That is, when the user has stronger feelings towards the video being viewed, the user continues to press the display button 53 for displaying the onomatopoeia image corresponding to this feeling. As a result, the onomatopoeia image is stretched and displayed from the first image display area 50B towards the distribution video display area 50A, and is displayed on the screen 3a so that the onomatopoeia image is more prominent.

[0074] In this way, by changing the onomatopoeia image included in the first image 51 according to the operation mode by the user, the user can display their own reaction (feeling) to the content on the screen 3a in a more entertaining manner.

[0075] Also, since the first image 51 is displayed so as to stretch from the first image display area 50B towards the distribution video display area 50A according to the operation mode by the user, the user can display their own reaction (feeling) to the content on the screen 3a in a more entertaining manner.

[0076] Note that when the operation (long-press operation) of the display button 53 by the user ends, the display of the onomatopoeia image on the screen 3a stops. The stop of the display of the onomatopoeia image may be, for example, that the displayed onomatopoeia image disappears from the screen 3a instantaneously, or the stretched onomatopoeia image shrinks towards the display button 53 and disappears from the screen 3a.

[0077] Also, in the example of FIG. 5, in order to display the onomatopoeia image, the area around the display button 53C being operated by the user is highlighted. This allows the user to easily recognize the display button 53C they are operating and enhances the entertainment value related to the onomatopoeia display.

[0078] Also, character images and the like, which are the first images 51 other than the onomatopoeia images according to the embodiment, are displayed on the screen 3a without changing. However, this is not the only case. The first images 51 other than the onomatopoeia images may also be displayed on the screen 3a where a video is being displayed while changing according to the operation mode of the display button 53 by the user.

[0079] [2-3-3. Other Display Examples of the First Image] The operation for displaying the first image 51 and the display mode of the first image 51 are not limited to the above-described mode, and may be other modes exemplified below.

[0080] The onomatopoeia image is not limited to a long-press operation. For example, the display mode may change according to a gesture operation on the screen 3a by the user. This allows the user to easily display on the screen 3a while watching the video the emotions felt while watching the video.

[0081] The gesture operation for performing the onomatopoeia display is, for example, an operation in which the user presses the display button 53 and flicks it in a desired direction by themselves, so-called flick operation. When the flick operation is performed, as an example, the onomatopoeia image is displayed starting from the vicinity of the position of the display button 53 in the direction of the flick operation. That is, when the user presses the display button 53 and performs a flick operation in the upper right direction, the onomatopoeia image is displayed in the upper right direction starting from the position of the display button 53. Also, the onomatopoeia image may have different elongation display modes according to the speed of the flick operation. For example, the faster the flick speed, the larger the onomatopoeia image may be elongated, or the faster the flick speed, the faster the elongation speed of the onomatopoeia image may be.

[0082] Also, when the user's finger pressing the display button 53 slides without leaving the screen 3a, the onomatopoeia image may be extended and displayed starting from the position of the display button 53 in accordance with the movement of the finger of the user performing the slide operation. Also, when the display button 53 is continuously pressed, such as by double-tapping, the onomatopoeia image may be extended and displayed according to the number of consecutive presses.

[0083] Also, the change in the display mode of the onomatopoeia image is not limited to extended display. For example, when a flick operation is performed by the user, the onomatopoeia image may be displayed so as to move in the direction of the flick operation without changing its form, starting from the vicinity of the position of the display button 53. In this case, the moving speed of the onomatopoeia image may change according to the speed of the flick operation. For example, the faster the speed of the flick operation, the faster the moving speed of the onomatopoeia image.

[0084] Also, the onomatopoeia image may be displayed such that the start of the display is the largest and it gradually shrinks according to the long-press time.

[0085] Also, the onomatopoeia image is not limited to being displayed starting from the position of the display button 53. For example, when the display button 53 is pressed, the onomatopoeia image may be displayed at the center of the screen 3a (distribution video display area 50A) and extended and displayed in the vertical direction according to the long-press time.

[0086] Also, the onomatopoeia image may be displayed not at the timing when the display button 53 is pressed with a finger, but at the timing when the finger is separated from the display button 53. Also in this case, for example, the display mode of the onomatopoeia image changes according to the long-press time of the display button 53.

[0087] Furthermore, the onomatopoeia image does not have to change in size or shape regardless of the operation mode of the display button 53. For example, at the timing when the user operates the display button 53, an onomatopoeia image of a size that overlaps with the first image display area 50B, the distribution video display area 50A, and the second image display area 50C may be displayed for a predetermined time.

[0088] [2-3-4. Display of the Second Image] Referring to FIG. 6, the display mode of the second image 52 will be described in detail. Note that, as described above, the second image 52 is a reduced image of the first image 51 displayed on another mobile terminal 3 by the operation of another user.

[0089] The onomatopoeia display function according to the embodiment displays the second image 52 in the second image display area 50C with a slight time delay from the timing when another user displays the first image 51 on another mobile terminal 3. Thereby, the user can know the reaction of another user who is watching the same video while watching the video.

[0090] Also, if the second image 52 is displayed superimposed on the distribution video display area 50A like the first image 51, the user may feel that the second image 52 interferes with the viewing of the video and is troublesome. Therefore, the second image 52 is not displayed superimposed on the distribution video display area 50A beyond the second image display area 50C regardless of the operation mode of another user. Thereby, the user can confirm the reaction of another user to the video being viewed without feeling bothered.

[0091] Note that, as described above, the first image 51 can also be superimposed on the distribution video display area 50A, but since this is done according to the user's own intention, the user does not feel bothered by the superimposition of the first image 51 on the distribution video display area 50A.

[0092] Also, since the screen 3a is set such that the first image display area 50B and the second image display area 50C are different areas, the first image 51 displayed by the user and the second image 52 displayed by another user are not displayed mixedly. Thereby, the user can easily distinguish and recognize the first image 51 displayed by himself / herself and the second image 52 displayed by another user.

[0093] Also, other users according to the embodiment are, for example, other users registered in the same group as the user. That is, when the user is viewing a video without group registration, the second image 52 is not displayed in the second image display area 50C. Note that this is not the only case, and regardless of whether there is group registration, the second image 52 by the operations of all other users viewing the video may be displayed in the second image display area 50C.

[0094] Also, in the second image display area 50C according to the embodiment, a plurality of second images 52 by the operations of a plurality of other users are displayed. And the second image display area 50C according to the embodiment is, for example, not defined with the position for displaying the second image 52 for each other user. That is, in the second image display area 50C, a plurality of second images 52 by the operations of a plurality of other users are displayed in a random position in a mixed manner. Further, the second image 52 may be displayed while moving in the vertical, horizontal, left, and right directions within the second image display area 50C. Note that this is not the only case, and within the second image display area 50C, the position for displaying the second image 52 for each other user may be defined.

[0095] Also, the second image 52 is displayed with an identification image (hereinafter referred to as "user icon") 54 indicating the other user who has the first image 51 as the second image 52 displayed on his or her own mobile terminal 3 adjacent thereto. Thereby, the user can recognize which user the second image 52 displayed in the second image display area 50C is from. In the example of FIG. 6, the user icon 54 has a single alphabet letter displayed at the center of a circle, but this is just an example, and the user icon 54 may be an image set for each user or the initials of the user, and the design of the user icon 54 is not limited.

[0096] Also, the second image 52 shown in FIG. 6 has an onomatopoeia image displayed horizontally, but this is set corresponding to the size (aspect ratio) of the second image display area 50C, and the second image 52 may have the onomatopoeia image displayed vertically.

[0097] Also, although the second image 52 is not superimposed on the distribution video display area 50A, it may be displayed while changing (for example, expanding) within the second image display area 50C in response to the operation mode by other users. Note that if the second image 52 is not superimposed on the distribution video display area 50A, it may be superimposed on the first image display area 50B from the second image display area 50C.

[0098] [2-4. Functional Blocks] Next, the onomatopoeia display will be described using a functional block diagram. Note that the function related to the onomatopoeia display in the server 4 is included in the video distribution function.

[0099] [2-4-1. Functional Blocks Related to the Video Distribution Function of the Server] FIG. 7 is a functional block diagram related to the video distribution function of the server 4 according to the embodiment.

[0100] The CPU 20 included in the server 4 includes a group registration unit 60, a video distribution unit 61, a first image display information reception control unit 62, a second image display information generation unit 63, and a second image display information transmission control unit 64. Each function included in the CPU 20 is realized by a program stored in the HDD 26.

[0101] The group registration unit 60 receives the registration of a group 5 composed of a plurality of mobile terminals 3. Note that the registration of the group 5 is performed by registering a user ID as an example.

[0102] The video distribution unit 61 distributes a video to the mobile terminal 3 in response to a request from the mobile terminal 3.

[0103] The first image display information reception control unit 62 receives the first image display information received from the mobile terminal 3 via the external interface 32, and stores the first image display information in the HDD 26 in association with the corresponding video.

[0104] The first image display information is information regarding the first image 51 displayed on the screen 3a of the mobile terminal 3, and is transmitted from the mobile terminal 3 to the server 4 at the timing when the first image 51 is displayed on the mobile terminal 3. The first image display information is, for example, the type of the displayed first image 51, the user ID indicating the user who displayed the first image 51, the operation mode of the user performed to display the first image 51, the elapsed time from the start of the video when the first image 51 is displayed, and the like.

[0105] The second image display information generation unit 63 generates second image display information based on the received first image display information, and associates the second image display information with the corresponding video and stores it in the HDD 26.

[0106] The second image display information is information regarding the second image 52 to be displayed in the second image display area 50C, and includes the type of the second image 52, the user ID of the user who displayed the first image 51 that is the source of the second image 52 on his / her own mobile terminal 3, and the like.

[0107] The second image display information transmission control unit 64 controls the external interface 32 to transmit the second image display information to the mobile terminal 3. The transmission destination of the second image display information is another mobile terminal 3 different from the mobile terminal 3 that transmitted the first image display information.

[0108] [2-4-2. Functional Blocks Related to Onomatopoeia Display of Mobile Terminal] FIG. 8 is a functional block diagram related to onomatopoeia display of the mobile terminal 3 according to the embodiment.

[0109] The main control unit 40 provided in the mobile terminal 3 includes an image display control unit 70, a display area setting unit 71, a first image display operation reception unit 72, an operation state determination unit 73, a first image display information generation unit 74, a first image display information transmission control unit 75, and a second image display information reception control unit 76. Each function provided in the main control unit 40 is realized by a program stored in the auxiliary storage unit 44.

[0110] The image display control unit 70 controls the touch panel display 3a to display an image based on the image data. The image display control unit 70 according to the embodiment causes the video distributed from the server 4 to be displayed in the distributed video display area 50A, causes the first image 51 to be displayed in the first image display area 50B according to the user's operation, and causes the second image 52 based on the first image 51 displayed on another mobile terminal 3 by another user to be displayed in the second image display area 50C. Further, the image display control unit 70 according to the embodiment causes the first image 51 to be superimposed and displayed on the distributed video display area 50A beyond the first image display area 50B according to the user's operation mode.

[0111] The display area setting unit 71 sets the positions and sizes (vertical and horizontal lengths) of the distributed video display area 50A, the first image display area 50B, and the second image display area 50C as the display mode of the screen 3a when a video is distributed from the server 4. Although these positions and sizes are predetermined, the user may set them arbitrarily.

[0112] The first image display operation reception unit 72 receives the operation of the display button 53 by the user in a state where the screen 3a is displaying a video. That is, the first image display operation reception unit 72 recognizes the display button 53 operated by the user among the plurality of display buttons 53, and causes the first image 51 corresponding to the display button 53 to be displayed on the screen 3a via the image display control unit 70. Note that the first image display operation reception unit 72 causes the first image 51 to be displayed on the screen 3a so that the display mode of the first image 51 changes according to the determination result by the operation state determination unit 73.

[0113] The operation state determination unit 73 determines the operation state of the display button 53 by the user. The operation state here is, for example, an operation mode including the long-press time of the display button 53.

[0114] The first image display information generation unit 74 generates first image display information corresponding to the first image 51 displayed on the screen 3a by the user's operation.

[0115] The first image display information transmission control unit 75 controls the communication unit 46 to transmit the first image display information to the server 4.

[0116] The second image display information reception control unit 76 outputs the second image display information received from the server 4 via the communication unit 46 to the image display control unit 70.

[0117] [2-5. Processing flow] Next, the flow of onomatopoeia display processing in the mobile terminal 3 and the server 4 will be described using a flowchart.

[0118] [2-5-1. Flow related to onomatopoeia display in the mobile terminal] FIG. 9 is a flowchart showing the flow of onomatopoeia display processing according to an embodiment executed by the main control unit 40 provided in the mobile terminal 3. A program for executing the onomatopoeia display processing is stored in advance in a predetermined area of the auxiliary storage unit 44. The onomatopoeia display processing is started, for example, when the mobile terminal 3 starts playing a video distributed from the server 4, and ends when the video playback ends.

[0119] In step S100, the occurrence of a trigger is detected. The trigger in this embodiment is either a first trigger (receiving an operation of the display button 53 by the user) or a second trigger (receiving the second image display information). The mobile terminal 3 according to this embodiment updates the display content of the screen each time the occurrence of a trigger is detected. The first trigger and the second trigger can occur in parallel. Therefore, during the execution of the process in response to the occurrence of one trigger, the process in response to the occurrence of the other trigger may be executed. Also, a plurality of second triggers can occur in parallel. Therefore, during the execution of the process in response to the occurrence of one second trigger, the process in response to the occurrence of the other second trigger may be executed.

[0120] In step S102, if the detected trigger is the first trigger, the process proceeds to step S106, and if the detected trigger is the second trigger, the process proceeds to step S104.

[0121] In step S104, the image display control unit 70 causes the second image 52 indicated by the second image display information to be displayed in the second image display area 50C, and ends the process.

[0122] In step S106, the image display control unit 70 causes the first image 51 corresponding to the display button 53 operated by the user to be displayed in the first image display area 50B. Note that the first image 51 may be stored in the mobile terminal 3 in advance, or may be downloaded from the server 4 each time the display button 53 is operated by the user.

[0123] In step S108, the first image display information generation unit 74 generates the first image display information corresponding to the first image 51 displayed in the first image display area 50B.

[0124] In step S110, the first image display information transmission control unit 75 controls the communication unit 46 to transmit the first image display information to the server 4.

[0125] In step S112, the operation state determination unit 73 determines whether or not the operation of the display button 53 by the user has ended. If the determination is affirmative, the process proceeds to step S116, and if the determination is negative, the process proceeds to step S114.

[0126] In step S114, the image display control unit 70 causes the first image 51 (onomatopoeia image) to be stretched and displayed until the operation of the display button 53 by the user ends, and returns to step S112.

[0127] In step S116, the image display control unit 70 ends the display of the first image 51 and ends the process.

[0128] In the flowchart shown in FIG. 9, first image display information is generated along with the display of the first image 51, and the first image display information is transmitted to the server 4. However, this is not the only case, and the first image display information may be transmitted to the server 4 after the display of the first image 51 ends. Further, the first image display information transmission control unit 75 may transmit the first image display information indicating the start of the display of the first image 51 to the server 4 at the timing when the display of the first image 51 starts, and transmit the first image display information indicating the end of the display of the second image 52 to the server 4 at the timing when the display of the first image 51 ends. Thereby, the server 4 can recognize that the first image 51 (onomatopoeia image) is being extendedly displayed during the reception interval between the first image display information indicating the start of the display of the first image 51 and the first image display information indicating the end of the display.

[0129] [2-5-2. Flowchart regarding onomatopoeia display on server] FIG. 10 is a flowchart showing the flow of onomatopoeia display processing according to an embodiment executed by the CPU 20 provided in the server 4. A program for executing the onomatopoeia display processing is stored in advance in a predetermined area of the HDD 26. The onomatopoeia display processing is started, for example, when the server 4 is delivering a video to the mobile terminal 3, and ends when the delivery of the video ends.

[0130] In step S200, the first image display information reception control unit 62 determines whether or not the first image display information has been received from the mobile terminal 3. If the determination is affirmative, the process proceeds to step S202; if the determination is negative, the process is in a waiting state.

[0131] In step S202, the received first image display information reception control unit 62 stores it in the HDD 26.

[0132] In step S204, based on the received first image display information, the second image display information generation unit 63 generates the second image display information and stores it in the HDD 26.

[0133] In step S206, the second image display information transmission control unit 64 controls the external interface 32 to transmit the second image display information to a mobile terminal 3 different from the mobile terminal 3 that has transmitted the first image display information, and returns to step S200.

[0134] In addition, when the second image display information is already associated with the video distributed to the mobile terminal 3, the server 4 may also transmit the second image display information to the mobile terminal 3. Thereby, the user can recognize the first image 51 displayed by other users in the past instead of in real time. For example, even when a user registered in a group watches a video as a group 5 and then watches the video alone (asynchronous with other mobile terminals 3) instead of as the group 5, the second image 52 based on the first image 51 displayed in the past will be displayed in the second image display area 50C.

[0135] As described above, in the mobile terminal 3 according to the embodiment, a distribution video display area 50A for displaying a video distributed from the server 4, a first image display area 50B for displaying a first image 51 according to a user's operation, and a second image 52 based on the first image 51 displayed on another mobile terminal 3 by the operation of another user are set on the screen 3a. And although the first image 51 is displayed superimposed on the distribution video display area 50A beyond the first image display area 50B according to the user's operation mode, the second image 52 is not displayed superimposed on the distribution video display area 50A beyond the second image display area 50C regardless of the operation mode of another user.

[0136] Thereby, the mobile terminal 3 can display the user's own reaction to the video on the screen 3a with higher entertainment, and can confirm the reaction of other users without annoying the user watching the video.

[0137] In addition, while the screen 3a of the mobile terminal 3 according to the embodiment is displaying a video, the mobile terminal 3 receives a user operation for displaying an onomatopoeia image indicating the user's emotion on the screen 3a. Then, the onomatopoeia image is displayed on the screen 3a that is displaying the video while its display mode changes according to the operation mode by the user. Therefore, the user can change the display mode of the onomatopoeia image displayed on the screen according to the operation mode while viewing the content.

[0138] Thereby, the mobile terminal 3 can easily display on the screen 3a the emotion felt while viewing the video by the user while the user is viewing the video.

[0139] [3. Modification Example] The technical scope of the present invention is not limited to the scope described in the above embodiment. Various changes or improvements can be made to the above embodiment without departing from the gist of the invention, and the forms with such changes or improvements are also included in the technical scope of the present invention.

[0140] In the above embodiment, the form in which the user operation for displaying the first image 51 on the screen 3a is the operation of the display button 53 has been described, but the present invention is not limited to this. For example, the first image may be displayed on the screen 3a when the user operates an operation button 48 provided on the side surface or the like of the mobile terminal 3. Also, when the mobile terminal 3 is provided with a keyboard, the first image 51 may be displayed on the screen 3a when a predetermined key input is performed.

[0141] Also, in the above embodiment, the form in which the onomatopoeia display process starts together with the distribution of the video and ends together with the re - distribution of the video has been described, but the present invention is not limited to this. For example, the onomatopoeia display process may start a predetermined time before the distribution of the video.

[0142] For example, when the distribution start time of the video is predetermined, the first image 51 and the second image 52 may be displayable in the first image display area 50B and the second image display area 50C by a user's operation starting from a predetermined time (e.g., 30 minutes) before the distribution start time. Thereby, even before the start of the video distribution, the user can display on the screen 3a his / her reactions such as the sense of expectation for the video, and can also confirm the reactions of other users. Also, the first image 51 and the second image 52 may be displayable in the first image display area 50B and the second image display area 50C by a user's operation until a predetermined time (e.g., 30 minutes) after the end of the video distribution. Thereby, even after the start of the video distribution, the user can display on the screen 3a his / her reactions such as the overall impression of the video, and can also confirm the reactions of other users.

[0143] Also, the positions and sizes of the distribution video display area 50A, the first image display area 50B, and the second image display area 50C, and the display button 53 on the screen 3a shown in FIG. 4 and the like are merely examples, and the present invention is not limited thereto. For example, the mode shown in FIG. 11 may be used. The example of FIG. 11(A) is an example of the display mode of the screen 3a when the screen 3a is horizontal (the long axis direction is horizontal with respect to the user, and the short axis direction is vertical with respect to the user). The example of FIG. 11(B) is an example of the display mode of the screen 3a when the screen 3a is vertical (the long axis direction is vertical with respect to the user, and the short axis direction is horizontal with respect to the user).

[0144] In the example of FIG. 11(A), with the distribution video display area 50A at the center, the first image display area 50B is displayed on the right side thereof, and the second image display area 50C is displayed on the left side thereof. Then, a plurality of display buttons 53 are displayed vertically on the right side of the first image display area 50B. In the example of FIG. 11(B), the distribution video display area 50A is displayed in the upper half of the screen 3a, the first image display area 50B is displayed on the right side of the lower half of the screen 3a, and the second image display area 50C is displayed on the left side thereof. Note that in both FIGS. 11(A) and (B), the first image display area 50B may be displayed on the left side of the screen 3a and the second image display area 50C may be displayed on the right side thereof according to the dominant hand of the user.

[0145] In the above-described embodiment, the server 4 receives the first image display information from the mobile terminal 3, transmits the second image display information generated based on the first image display information to another mobile terminal 3, and the other mobile terminal 3 displays the second image 52 indicated by the second image display information in the second image display area 50C. However, the present invention is not limited to this.

[0146] For example, the mobile terminal 3 may display the second image 52 in the second image display area 50C by transmitting and receiving the first image display information among a plurality of mobile terminals 3 without passing through the server 4. That is, when the mobile terminal 3 receives the first image display information from another mobile terminal 3, the mobile terminal 3 reads information regarding the first image 51 displayed on the other mobile terminal 3 from the first image display information, and displays the second image 52 based on the information in the second image display area 50C.

[0147] Also, the flow of various processes described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist of the present invention.

Explanation of Reference Numerals

[0148] 1 Video distribution system 2 Communication line 3 Mobile terminal (an example of "information processing device") 3a Screen (an example of "display screen") 4 Server (an example of "management means") 5 Group (an example of "information processing system") 50A Distribution video display area (an example of "basic area", an example of "second part" of the first range) 50B First image display area (an example of "first part" of the first range) 50C Second image display area (an example of "third part" of the first range, an example of "second range") 51 First image (an example of "first element", "additional element") 52 Second image (an example of "second element") 53 Display button (an example of "operation element") 70 Image display control unit (an example of "first control means", "second control means", and "control means") 71 Display area setting unit 72 First image display operation reception unit (an example of "reception means") 73 Operation state determination unit 74 First image display information generation unit 75 First image display information transmission control unit 76 Second image display information reception control unit (an example of "acquisition means")

Claims

[Claim 1] a first control means for displaying a first element based on a first operation by a first user viewing the viewing target on a first information processing device in a first range at least partially overlapping a basic region in which the viewing target is displayed on the first information processing device; a second control means for displaying a second element based on a second operation by a second user viewing the viewing target on a second information processing device in a second range not overlapping with the basic region; An information processing system comprising:

Images