Shared information processing apparatus and control program

Abstract

To prevent unexpected workload from increasing in a terminal for displaying elements, concerning a system where the elements are shared among a plurality of users viewing each of common subjects to be viewed.SOLUTION: A shared information processing device (e.g., a server 4) representing one embodiment of the present invention comprises: identifying means (e.g., a replacement processing unit 67) for, among shared elements in which timings to be displayed in a user terminal (e.g., a mobile terminal 3) viewing subjects to be viewed are associated with the subjects to be viewed, identifying the shared elements in a prescribed count as an element group based on the timings; and replacement means (e.g., a replacement processing unit 67) for replacing the element group identified by the identifying means with substitute elements fewer than the prescribed count.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to a mechanism for sharing elements among a plurality of users who respectively view a common viewing target.

Background Art

[0002] There is known a communication system that shares comments input according to reproduced content among a plurality of users. For example, in the communication system described in Patent Document 1, a user terminal that has acquired from a comment server comments to be displayed together with content before the start of reproduction of the content excludes old comments from being displayed so that the number of displayed comments in each of a plurality of time segments into which a timeline is divided becomes equal to or less than a predetermined number.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the communication system described in Patent Document 1, processing for leveling the temporal display number of comments displayed together with content is executed before the start of reproduction of the content. Therefore, it is impossible to cope with a sudden increase in load caused by comments added during the reproduction of content.

[0005] The problem to be solved by the present invention is to suppress a sudden increase in load on a terminal that displays an element in a mechanism for sharing elements among a plurality of users who respectively view a common viewing target.

Means for Solving the Problems

[0006] To solve the above problems, the present invention specifies a predetermined number of the shared elements as an element group based on the timing from among the shared elements associated with the viewing target at the timing of display on the terminal of the user viewing the viewing target, and replaces the element group with a smaller number of alternative elements than the predetermined number.

[0007] A shared information processing apparatus according to an aspect of the present invention includes specifying means for specifying, as an element group, a predetermined number of the shared elements based on the timing from among the shared elements associated with the viewing target at the timing of display on the terminal of the user viewing the viewing target, and replacing means for replacing the element group specified by the specifying means with a smaller number of alternative elements than the predetermined number.

Advantages of the Invention

[0008] According to the present invention, a sudden increase in load on the terminal that displays the shared elements can be suppressed.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

[0010] [1. Embodiment] [1-1. Main Features] This embodiment relates to a mechanism for sharing elements among a plurality of users who respectively view a common viewing target. In this embodiment, in order to suppress a sudden increase in load on the terminal that displays the elements to be shared, the timing of display on the terminal of the user who is viewing the viewing target is based on the timing from among the shared elements associated with the viewing target. A predetermined number of the shared elements are specified as an element group, and the element group is replaced with a smaller number of alternative elements than the predetermined number.

[0011] "Viewing target" refers to a visual and auditory object that can be grasped by a user through at least one of vision and hearing. A visual and auditory object is a concept that includes video, audio, and combinations thereof. A visual and auditory object includes an object that is not signalized and an object that is reproduced based on a signal. The signal of the object to be reproduced can be recorded on a recording medium, for example, and can be carried by a carrier wave. The object to be reproduced is reproduced as at least one of video and audio by a device having a reproduction function, for example.

[0012] The "progression" of the viewing object refers to the change of the viewing object over time. The change is a concept that encompasses immutability and change. Immutability means that visual information and auditory information continue to remain constant. Change means that visual information and auditory information are not constant. Changes include, for example, irregular changes, regular changes, periodic repetitions, temporary disappearances, and combinations thereof. When the viewing object is an object that is not signalized, the timing (time point) during progression can be specified by time information. When the viewing object is an object that is reproduced based on a signal, the time point during progression can be specified by time information or the elapsed time from the start time of progression (i.e., the position on the timeline).

[0013] The "viewing" of the viewing object means grasping the progression of the viewing object by at least either vision or hearing. When the viewing object is an object that is not signalized, the said viewing object is viewed almost simultaneously with the progression. The viewing of a non-signalized object includes, for example, viewing a sports game at a stadium, viewing natural phenomena (e.g., solar eclipse, lunar eclipse, aurora, sunrise, sunset, sunset glow, rare meteorological phenomena, etc.) at an appropriate location, viewing an event (e.g., festival, street speech, etc.) at an appropriate location. When the viewing object is an object that is reproduced based on a signal, the said viewing object is viewed when it is reproduced. The viewing of a reproduced object includes, for example, viewing a television broadcast, viewing a radio broadcast, viewing an online-distributed video, viewing an online-distributed music.

[0014] A user (viewer) who views the viewing target can associate a predetermined element (shared element) with the viewing target by a predetermined operation (additional operation). The predetermined operation is, for example, a pointing operation on an input element (operation element). The input element is, for example, a button displayed within the screen. The pointing operation is, for example, a combination of a touch operation on a touch panel, a positioning operation, and a click operation of a pointer displayed within the screen. When a predetermined element is associated with the viewing target, the timing for displaying the shared element during the progress of the viewing target is associated with the viewing target. The element associated with the viewing target can be displayed on the screen at the associated display timing during the progress of the viewing target.

[0015] The predetermined element is a visual element representing the user's reaction to the viewing target. The user's reaction is, for example, the user's emotion. The visual element is, for example, a still image, an animated image, a moving image, text, or a combination thereof. An image representing the user's emotion towards the viewing target may be called an emotion image. The emotion image is, for example, an image in which onomatopoeia (also referred to as onomatopoeic words, sound words, or mimetic words) indicating the user's emotion is designed. An image in which onomatopoeia is designed may be called an onomatopoeia image. Also, the predetermined element can be shared among a plurality of users who view a common viewing target. The element that can be shared is transmitted and received online and displayed on the screen of the terminal. The terminal having the screen on which the element that can be shared is displayed may be the same device as the device that plays the object that is the viewing target, or a different device.

[0016] When specified "based on timing", a predetermined number of shared elements with adjacent display timings can be specified as an element group. For two shared elements, the display timings being in an adjacent relationship means that the time difference between the timing when one is displayed on the screen and the timing when the other is displayed on the screen is equal to or less than a predetermined threshold. For three or more shared elements, the display timings being in an adjacent relationship means that the display timings of any two shared elements selected from among the three or more shared elements are in an adjacent relationship. When determining the presence or absence of an adjacent relationship regarding the display timing, a certain point in time between the start and end of the display of each shared element is adopted as the display timing. The point in time adopted as the display timing is, for example, the start point, the end point, or a predetermined reference point.

[0017] "Alternative element" refers to visual information associated with the viewing target in place of an element group (replacement target) that includes two or more shared elements. The alternative element may be visual information common to the replacement target or visual information with a different display mode from the replacement target. The display mode is, for example, display time, display size, display color, presence or absence of additional visual information, and presence or absence of movement. By changing the display mode, the multiple occurrences of an element may be shown to the user. Specifically, the display mode may be changed so that the greater the multiple occurrences, the more elaborate (rich) it becomes. Being elaborate means, for example, having a long display time, a large display size, a dark display color, a large number of additional visual information, a large movement, etc. Also, by changing the display mode, the multiple occurrences of an element may be shown to the user. Specifically, visual information indicating the multiple occurrences may be added to the alternative element. Visual information indicating the multiple occurrences is, for example, an icon that displays a number corresponding to the multiple occurrences, or icons in a number corresponding to the multiple occurrences.

[0018] [Appendix 1-2] The present invention at least includes the forms from the following [A] to the following [C].

[0019] 〔A〕A shared information processing apparatus comprising: a specifying means for specifying, as an element group, a predetermined number of the shared elements based on the timing at which the shared elements associated with a viewing target are to be displayed on the terminal of the user viewing the viewing target; and a replacing means for replacing the element group specified by the specifying means with a smaller number of alternative elements than the predetermined number.

[0020] 〔B〕A control program for causing a computer of an information processing apparatus to realize: a specifying function for specifying, as an element group, a predetermined number of the shared elements based on the timing at which the shared elements associated with a viewing target are to be displayed on the terminal of the user viewing the viewing target; and a replacing function for replacing the element group specified by the specifying function with a smaller number of alternative elements than the predetermined number. Note that the control program may be non-temporarily recorded on a recording medium.

[0021] 〔C〕A control method of an information processing apparatus including: a specifying stage for specifying, as an element group, a predetermined number of the shared elements based on the timing at which the shared elements associated with a viewing target are to be displayed on the terminal of the user viewing the viewing target; and a replacing stage for replacing the element group specified in the specifying stage with a smaller number of alternative elements than the predetermined number.

[0022] The "information processing apparatus" in the above [A] may be subject to the following technical limitations. Similarly, the same technical limitations may be applied to the "control program" in the above [B] and the "control method of the information processing apparatus" in the above [C], respectively.

[0023] (1) The specific means identifies the group of elements based on the additional attributes added to the shared elements, and the replacement means adds, as the shared elements, the alternative elements that inherit the additional attributes added to at least any one of the multiple shared elements included in the group of elements, to the viewing target. When identified "based on the additional attributes", a predetermined number of shared elements having commonality in the additional attributes can be identified as the group of elements. Having commonality means being identical or highly relevant. This enables the alternative elements after replacement to be further replaced by other alternative elements.

[0024] (2) The specific means identifies the group of elements based on the additional attributes that can identify at least either the user group to which the user who associates the shared elements with the viewing target belongs or the element group to which the shared elements associated with the viewing target belong. This replaces multiple shared elements that are likely to overlap in time with alternative elements.

[0025] Note that a user who can associate shared elements with a viewing target can belong to at least one user group. Each user can voluntarily join any user group. Also, each user may automatically join any user group according to their attributes and circumstances. The shared elements associated with the viewing target can belong to at least one element group. Each shared element is pre-assigned to at least one element group.

[0026] (3) The specific means identifies the group of elements based on the additional attributes that can identify both the user group and the element group. This replaces multiple shared elements that are likely to overlap in time with alternative elements.

[0027] (4) When the types of the predetermined number of the shared elements included in the group of elements identified by the specific means are the same, the replacement means replaces the group of elements with the alternative elements corresponding to the multiple occurrences of the type. This enables a visual effect to be applied so that the load does not increase excessively.

[0028] (5) The specific means identifies the set only when a predetermined activation condition is satisfied. Thereby, replacement with alternative elements is activated only in situations where a sudden load increase is likely to occur.

[0029] The "activation condition" refers to a condition for determining whether to activate a replacement process (a process of replacing a predetermined number of shared elements with a number of alternative elements less than the predetermined number). The activation condition is preferably set as a condition related to the number of elements in the group to which the shared elements in the proximity relationship belong. The condition related to the number of elements is, for example, the truth or falsehood of whether the number of elements has reached or exceeded a predetermined upper limit value. The predetermined upper limit value may be a fixed value or a variable value. The variable value is preferably set according to the processing capacity of the terminal. The processing capacity of the terminal may be the processing capacity related to information display or the processing capacity related to information reception. The processing capacity related to information display is, for example, the clock frequency of the control processing device of the terminal. The processing capacity related to information reception is, for example, the speed of the communication line of the terminal.

[0030] (6) The specific means identifies the predetermined number of the shared elements included in the proximity element group as the element group, except when the number of elements in the proximity element group including a plurality of the shared elements whose timings are close does not satisfy the activation condition. Thereby, replacement with alternative elements is not activated in a situation where a sudden load increase is unlikely to occur.

[0031] (7) The specific means identifies the predetermined number of the shared elements included in the proximity element group as the element group, except when the activation condition that varies according to the processing capacity of the terminal is not satisfied. Thereby, it is possible to flexibly determine whether a situation where a sudden load increase is unlikely to occur.

Example

[0032] [2. Example] Hereinafter, embodiments of the present invention will be described with reference to the drawings. This embodiment relates to a mechanism for displaying an image (image representing a reaction) representing the emotion of a user (viewer) on the display screen of a video (object to be viewed). In this embodiment, the shared information processing device is described as a server, and the information processing devices (first device, second device) are described as mobile terminals, so-called smartphones. Note that the mobile terminal is not limited to a smartphone, and may be a tablet terminal, a notebook computer, or the like. Further, the information processing device may be a desktop computer. Each program according to the embodiment is stored in the shared information processing device or the information processing device, but is not limited thereto, and may be stored in a storage medium such as an optical disk such as a CD-ROM (Compact Disk Read Only Memory) or a portable semiconductor memory such as a USB (Universal Serial Bus) flash memory.

[0033] [2-1. Features] In this embodiment, the operation information of an addition operation (first operation) performed on the first mobile terminal (first device) by the first user (the user himself / herself) who is viewing the video (object to be viewed) displayed on the screen (first screen) of the first mobile terminal to add a first image (first element) to the video, and the operation information of an addition operation (second operation) performed on the second mobile terminal (second device) by the second user (another person) who is viewing the video displayed on the screen (second screen) of the second mobile terminal to add a first image (second element) to the video are provided from each mobile terminal to the server (first instruction means, shared information processing device). The server displays a combined image (third element) based on the combination of the first operation and the second operation corresponding to the received operation information on the first screen and the second screen, respectively, in order to improve the entertainment value.

[0034] In addition, in this embodiment, in order to suppress a sudden load increase in the mobile terminal, during the playback of a video, the timing for display on the mobile terminal of the user who is viewing the video is associated with the video, and two or more (predetermined number) second images are specified as an image group (element group) based on the proximity relationship (timing) of the display timing and the commonality of the subject and the object (additional attribute) among the second images (shared elements). A replacement process is executed in which one new second image (replacement element) that is less than the predetermined number and that inherits the additional attribute is added to the video while invalidating the second images included in the image group. The subject refers to the user who added the second image to the video, and having commonality in the subject means belonging to the same user group. Each user belongs to at least one user group. Also, the object refers to the type of the second image (element) added to the video, and having commonality in the object means belonging to the same element group. Each element belongs to at least one element group. (The element) type, and having commonality in the object means belonging to the same element group. Each element belongs to at least one element group.

[0035] [2-2. Video distribution system] The server according to the embodiment is included in the video distribution system and distributes videos to the mobile terminals. The mobile terminal according to the embodiment is included in the video distribution system and receives videos distributed from the server.

[0036] [2-2-1. Overall configuration of the video distribution system] FIG. 1 is a schematic configuration diagram of a video distribution system 1 according to the embodiment. The video distribution system 1 includes a communication line 2, a plurality of mobile terminals 3, and a server 4.

[0037] The communication line 2 forms a computer network and is, for example, a wide-area communication line provided by an electric utility company.

[0038] The mobile terminal 3 is an information processing terminal such as a smartphone, a tablet terminal, or a notebook computer. The mobile terminal 3 includes a touch panel display 3a for displaying images, a speaker 3b for outputting sound, a microphone 3c for inputting sound, a camera 3d for imaging a subject, and the like. In order to display the video distributed from the server 4, as an example, dedicated application software (hereinafter referred to as "video viewing application") is installed in the mobile terminal 3.

[0039] The server 4 is a shared information processing device that distributes content to a plurality of mobile terminals 3 via the communication line 2. The content according to the embodiment is, for example, a video. The video has a story such as an animation or a drama, and the story progresses along the story, or a music video, a live video of music, various sports videos, etc., and is not particularly limited. Note that the video according to the embodiment is, for example, an animation. In addition, the content distributed from the server is not limited to videos, and other services such as a set of continuous still images may be used as long as they are viewable by the user.

[0040] Furthermore, the viewing here means that the user recognizes the content by at least one of vision and hearing. The content may be, in addition to the above, for example, a video without sound or a sound without an image.

[0041] A plurality of mobile terminals 3 to which the video is distributed are registered with the server 4 as a group 5 (hereinafter referred to as "group registration"). The server 4 distributes the video to the group-registered mobile terminals 3 at the same time. That is, the plurality of group-registered mobile terminals 3 display the video distributed from the server 4 on the touch panel display 3a at the same timing, in other words, synchronously. As a result, each user of the group-registered mobile terminals 3 will view the video at the same time.

[0042] Note that the video distribution may be at a predetermined date and time, or may be at an arbitrary timing for each Group 5. When the video is distributed at an arbitrary timing, for example, the host mobile terminal 3 makes a video distribution request to the server 4, and the server 4 that has received the distribution request distributes the video to the mobile terminals 3 that constitute Group 5.

[0043] In FIG. 1, four mobile terminals 3 that constitute Group 5 are shown, but this is just an example, and the number of mobile terminals 3 that constitute Group 5 is not limited as long as Group 5 is composed of a plurality of mobile terminals 3. Also, the types of mobile terminals 3 that constitute Group 5 are not limited, and smartphones, tablet terminals, notebook computers, etc. may be mixed.

[0044] [2-2-2. Configuration of Server] FIG. 2 is a block diagram showing the electrical configuration of the server 4 according to the embodiment.

[0045] The server 4 according to the embodiment includes a CPU (Central Processing Unit) 20 which is a main control unit that controls the operation of the entire server 4, a ROM (Read Only Memory) 22 in which various programs and various data are stored in advance, a RAM (Random Access Memory) 24 used as a work area when the CPU 20 executes various programs, and an HDD (Hard Disk Drive) 26 as a storage means for storing various programs and various data.

[0046] The HDD 26 stores information regarding the registered Group 5, videos (video data) to be distributed to the mobile terminals 3, and other various information. Note that the storage means is not limited to the HDD 26, and may be other storage media such as semiconductor memories such as flash memories.

[0047] Furthermore, the server 4 includes an operation input unit 28 composed of a keyboard, a mouse, etc. for receiving inputs of various operations, a monitor 30 such as a liquid crystal display device for displaying various images, and is connected via a communication line 2 to other information processing devices such as a mobile terminal 3, etc., and is provided with an external interface 32 for transmitting and receiving various data to and from other information processing devices, etc.

[0048] These CPU 20, ROM 22, RAM 24, HDD 26, operation input unit 28, monitor 30, and external interface 32 are electrically connected to each other via a system bus 34. Therefore, the CPU 20 can perform access to the ROM 22, RAM 24, and HDD 26, grasp the operation state of the operation input unit 28, display images on the monitor 30, and perform various data transmissions and receptions with other information processing devices, etc. via the external interface 32, respectively.

[0049] [2-2-3. Electrical Configuration of Mobile Terminal] FIG. 3 is a functional block diagram showing the electrical configuration of the mobile terminal 3. In addition to the configuration shown in FIG. 1, the mobile terminal 3 includes a main control unit 40, a main memory unit 42, an auxiliary storage unit 44, a communication unit 46, and operation buttons 48.

[0050] The main control unit 40 is, for example, a CPU, a microprocessor, a DSP (Digital Signal Processor), etc., and controls the overall operation of the mobile terminal 3.

[0051] The main memory unit 42 is composed of, for example, a RAM or a DRAM (Dynamic Random Access Memory), etc., and is used as a work area, etc. when executing processes based on various programs by the main control unit 40.

[0052] The auxiliary storage unit 44 is a non-volatile memory such as a flash memory, and stores various data such as images and programs used for the processing of the main control unit 40. The programs stored in the auxiliary storage unit 44 are, for example, an OS (Operating System) for realizing the basic functions of the mobile terminal 3, drivers for controlling various hardware, programs for realizing functions such as e-mail, web browsing, and other various functions. In addition, application software such as a video viewing application for playing back the video distributed from the server 4 and displaying it on the touch panel display 3a is stored in the auxiliary storage unit 44 in advance.

[0053] The communication unit 46 is, for example, a NIC (Network Interface Controller), and has a function of connecting to the communication line 2. Note that the communication unit 46 may have a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), a function of enabling short-range wireless communication such as Bluetooth (registered trademark), and infrared communication, instead of or together with the NIC.

[0054] The operation buttons 48 are provided on the side surface of the mobile terminal 3, and include a power button for starting or stopping the mobile terminal 3 and a volume adjustment button for the sound output from the speaker 3b.

[0055] The main control unit 40, the main storage unit 42, the auxiliary storage unit 44, the communication unit 46, the touch panel display 3a, the speaker 3b, the microphone 3c, the camera 3d, and the operation buttons 48 are electrically connected to each other via the system bus 49. Therefore, the main control unit 40 can access the main storage unit 42 and the auxiliary storage unit 44, display an image on the touch panel display 3a, grasp the operation state of the touch panel display 3a and the operation buttons 48 by the user, input sound to the microphone 3c, output sound from the speaker 3b, control the camera 3d, and access various communication networks and other information processing devices via the communication unit 46.

[0056] [2-3. Onomatopoeia display] While the mobile terminal 3 according to the embodiment displays the video distributed from the server 4 on the touch panel display 3a, it is also possible to display an image (onomatopoeia image whose details will be described later) according to the user's operation. In the following description, the function of displaying the onomatopoeia image on the touch panel display 3a is referred to as the onomatopoeia display function, and the touch panel display 3a is also referred to as the screen 3a.

[0057] As an example, the onomatopoeia display function is one of the functions of the video viewing application according to the embodiment. As an example, it is not possible to perform onomatopoeia display for all the videos displayed on the screen 3a using the video viewing application, and the videos for which onomatopoeia display is possible are predetermined.

[0058] [2-3-1. Screen configuration] Figures 4 to 6 show the display modes of the screen 3a of the mobile terminal 3 in which the video viewing application according to the embodiment is started. On the screen 3a, a distributed video display area 50A, a first image display area 50B, and a second image display area 50C are set.

[0059] The distributed video display area 50A is an area for displaying the video distributed from the server 4. The first image display area 50B is an area for displaying a first image 51 (see FIG. 5) according to the user's operation. The second image display area 50C is an area for displaying a second image 52 (see FIG. 6) based on the first image 51 displayed on another mobile terminal 3 by the operation of another user. As will be described in detail later, the second image 52 is a reduced image of the first image 51 displayed in the first image display area 50B of another mobile terminal 3. Also, as an example, the first image display area 50B and the second image display area 50C according to the embodiment are provided at positions facing each other with the distributed video display area 50A in between.

[0060] Here, the first image 51 is an image including an emotion image indicating the emotion of a user who is watching a video. The emotion image according to the embodiment is, as an example, an image in which onomatopoeia (also referred to as mimetic words or mimetic expressions) indicating the emotion of the user is designed, in other words, an onomatopoeia image. Examples of onomatopoeia images include "Suggee", "Yabai", "Braboh", etc. Note that the design means, for example, that the characters indicating onomatopoeia are drawn decoratively.

[0061] In addition to the onomatopoeia image, the first image 51 may also include various other images such as a character image showing a character appearing in the video, a dialogue image showing the dialogue of the character, text input by the user himself / herself, etc.

[0062] Also, the first image 51 may vary according to the scene of the video. For example, according to a happy scene, a sad scene, a confrontation scene between characters, etc., onomatopoeia images or character images that the user is likely to select while empathizing with the scene may be displayed.

[0063] Also, the displayable first image 51 may vary depending on the user. For example, the types of the displayable first image 51 may increase by obtaining a specific item through a service such as a game related to the distributed video. Also, for example, the types of the displayable first image 51 may increase on the condition of bearing a predetermined consideration (for example, a monetary value such as currency, virtual currency, points, etc.).

[0064] In the first image display area 50B according to the embodiment, a plurality of display buttons 53A to 53D, which are operation units operated by the user and correspond to different first images 51, are displayed. Thereby, the user can display the first image 51 in the first image display area 50B by operating the display buttons 53A to 53D while viewing the video. Note that the operation of the display buttons 53A to 53D is, for example, touching (pushing) the display buttons 53A to 53D with an indicator. The indicator according to the embodiment is, for example, the user's finger. In addition, in the following description, when the display buttons 53A to 53D are not distinguished and described, the notation of the last alphabet is omitted.

[0065] In the examples of FIGS. 4 to 6, four display buttons 53A to 53D are displayed in a horizontal row at the lower part of the first image display area 50B. The display button 53A corresponds to an onomatopoeia image depicting "suggee", the display button 53B corresponds to an onomatopoeia image depicting "yabai", and the display button 53C corresponds to an onomatopoeia image depicting "bravo". Further, the display button 53D corresponds to a character image. Note that the display button 53 has a design that allows the user to intuitively recognize the corresponding onomatopoeia image, character image, or the like.

[0066] As described above, by displaying a plurality of display buttons 53 in the first image display area 50B, the user can easily display the first image 51 that best suits his / her reaction (emotion) to the video on the screen 3a. In addition, since the first image 51 includes an onomatopoeia image indicating the user's emotion, the user can display his / her emotion after viewing the video on the screen 3a.

[0067] Note that in the examples of FIGS. 4 to 6, four types of display buttons 53 are displayed in the first image display area 50B, but this is only an example, and the number and types of the display buttons 53 are not limited thereto. Further, the display buttons 53 may be appropriately switched and displayed so as to be able to display different first images 51 according to the scene of the video or the like.

[0068] Also, the arrangement of the display buttons 53 is just an example. For example, they may be collectively displayed on the right or left side of the first image display area 50B, or may be displayed in two horizontal rows at the lower part of the first image display area 50B. Further, for example, when the user swipes the lower part of the screen 3a to the left or right, display buttons 53 for displaying different types of first images 51 may be newly displayed.

[0069] [2-3-2. Display of the First Image] Referring to FIG. 5, the display mode of the first image 51 will be described in detail. In the example of FIG. 5, the display button 53C is operated by the user, and an onomatopoeia image in which "Bravo" is designed is displayed on the screen 3a.

[0070] The onomatopoeia display function according to the embodiment accepts a user operation for displaying an onomatopoeia image (first image 51) on the screen 3a while a video is being displayed on the screen 3a. That is, while watching the video, the user operates the display button 53 to display an onomatopoeia image corresponding to the emotion felt by watching the video on the screen 3a.

[0071] And the onomatopoeia image according to the embodiment is displayed on the screen 3a where the video is being displayed while changing according to the operation mode of the display button 53 by the user. As an example, the onomatopoeia image according to the embodiment is displayed so as to extend from the first image display area 50B to the distribution video display area 50A according to the operation mode of the display button 53 by the user.

[0072] In this way, the user can change the onomatopoeia image displayed on the screen 3a by operating the mobile terminal 3 while watching the video. Therefore, the user can easily display the emotion felt by watching the video on the screen while watching the video.

[0073] Also, in the example of FIG. 5, the onomatopoeia image is displayed superimposed on the distribution video display area 50A beyond the first image display area 50B. In this way, the onomatopoeia display function according to the embodiment superimposes the onomatopoeia image on the distribution video display area 50A beyond the first image display area 50B according to the user's operation mode. Therefore, the user can more easily empathize with the video being viewed, and the entertainment value of viewing the video is further enhanced.

[0074] Note that in FIG. 5, the onomatopoeia image is also superimposed on the second image display area 50C beyond the distribution video display area 50A. However, this is just an example, and the onomatopoeia image may be made non - superimposable on the second image display area 50C.

[0075] Next, the details of the onomatopoeia image that change according to the operation mode of the display button 53 will be described.

[0076] As an example, the display mode of the first image 51 according to the embodiment changes according to the duration of the user's operation of the display button 53. The duration of the operation of the display button 53 is, for example, the time during which the display button 53 is continuously pressed (hereinafter referred to as the "long - press time").

[0077] When the user presses the display button 53, the onomatopoeia display function according to the embodiment starts displaying the onomatopoeia image starting from the vicinity of the position of the display button 53. Displaying the onomatopoeia image starting from the vicinity of the position of the display button 53 means, for example, that the onomatopoeia image is displayed such that the lower end of the onomatopoeia image substantially coincides with the position of the display button 53.

[0078] Then, the onomatopoeia image extends (hereinafter referred to as "extended display") from the first image display area 50B toward the distribution video display area 50A according to the long - press time with respect to the display button 53.

[0079] The elongation display is, for example, one in which the onomatopoeia image is displayed in a long and thin or enlarged manner. Note that the elongation display may be one in which the onomatopoeia image elongates while changing its form, such as the number of characters or the type of characters. For example, when the onomatopoeia image is "BRAVO", it may elongate so that "A" increases between "BRA" and "BO" according to the long-press time, or it may elongate so that "O" increases between "BRAVO" and "-" according to the long-press time.

[0080] That is, when the user has stronger feelings towards the video being viewed, the user continues to press the display button 53 for displaying the onomatopoeia image corresponding to this feeling. As a result, the onomatopoeia image is elongately displayed from the first image display area 50B towards the distribution video display area 50A, and is displayed on the screen 3a so that the onomatopoeia image is more prominent.

[0081] In this way, by changing the onomatopoeia image included in the first image 51 according to the operation mode by the user, the user can display their own reaction (feeling) towards the content on the screen 3a with higher entertainment value.

[0082] Also, since the first image 51 is displayed so as to elongate from the first image display area 50B towards the distribution video display area 50A according to the operation mode by the user, the user can display their own reaction (feeling) towards the content on the screen 3a with higher entertainment value.

[0083] Note that when the operation (long-press operation) of the display button 53 by the user ends, the display of the onomatopoeia image on the screen 3a stops. The stop of the display of the onomatopoeia image may be, for example, that the displayed onomatopoeia image instantaneously disappears from the screen 3a, or the elongated onomatopoeia image may shrink towards the display button 53 and disappear from the screen 3a.

[0084] In the example of FIG. 5, in order to display the onomatopoeia image, the area around the display button 53C being operated by the user is highlighted. As a result, the user can easily recognize the display button 53C that they are operating, and the entertainment value regarding the onomatopoeia display becomes higher.

[0085] Also, the character image or the like, which is the first image 51 other than the onomatopoeia image according to the embodiment, is displayed on the screen 3a without changing. However, not limited to this, the first image 51 other than the onomatopoeia image may also be displayed on the screen 3a where the video is being displayed while changing according to the operation mode of the display button 53 by the user.

[0086] [2-3-3. Display of the Second Image] Referring to FIG. 6, the display mode of the second image 52 will be described in detail. As described above, the second image 52 is a reduced image of the first image 51 displayed on another mobile terminal 3 by the operation of another user.

[0087] The onomatopoeia display function according to the embodiment displays the second image 52 in the second image display area 50C with a slight time delay from the timing when another user displays the first image 51 on another mobile terminal 3. As a result, the user can know the reaction of another user who is watching the same video while watching the video.

[0088] Also, if the second image 52 is displayed overlapping the distribution video display area 50A like the first image 51, the user may feel that the second image 52 obstructs the viewing of the video and is troublesome. Therefore, the second image 52 is not displayed overlapping the distribution video display area 50A beyond the second image display area 50C regardless of the operation mode of another user. As a result, the user can confirm the reaction of another user to the video being watched without feeling troubled.

[0089] Note that, as described above, the first image 51 can also be superimposed on the distribution video display area 50A. However, since this is done at the user's own discretion, the user does not feel bothered by the superimposition of the first image 51 on the distribution video display area 50A.

[0090] Also, since the screen 3a is set such that the first image display area 50B and the second image display area 50C are different areas, the first image 51 displayed by the user and the second image 52 displayed by other users do not appear mixed. As a result, the user can easily distinguish and recognize the first image 51 displayed by himself / herself and the second image 52 displayed by other users.

[0091] Also, other users according to the embodiment are, as an example, other users registered in the same group as the user. That is, when the user is watching a video without group registration, the second image 52 is not displayed in the second image display area 50C. Note that this is not the only case, and regardless of whether there is group registration, the second image 52 resulting from the operations of all other users watching the video may be displayed in the second image display area 50C.

[0092] Also, in the second image display area 50C according to the embodiment, a plurality of second images 52 resulting from the operations of a plurality of other users are displayed. And, as an example, in the second image display area 50C according to the embodiment, the position for displaying the second image 52 is not determined for each other user. That is, in the second image display area 50C, a plurality of second images 52 resulting from the operations of a plurality of other users are displayed mixed at random positions. Further, the second image 52 may be displayed while moving in the vertical and horizontal directions within the second image display area 50C. Note that this is not the only case, and within the second image display area 50C, the position for displaying the second image 52 may be determined for each other user.

[0093] In addition, in the second image 52, an identification image (hereinafter referred to as "user icon") 54 indicating another user who has the first image 51 displayed on his or her own mobile terminal 3 is displayed adjacent thereto. As a result, the user can recognize which user the second image 52 displayed in the second image display area 50C is from. In the example of FIG. 6, the user icon 54 has a single-letter alphabet displayed at the center of a circle, but this is just an example. The user icon 54 may be an image set for each user or the initials of the user, and the design of the user icon 54 is not limited.

[0094] Also, in the second image 52 shown in FIG. 6, the onomatopoeia image is displayed horizontally, but this is set corresponding to the size (aspect ratio) of the second image display area 50C, and the second image 52 may have the onomatopoeia image displayed vertically.

[0095] Also, although the second image 52 is not superimposed on the distribution video display area 50A, it may be displayed while changing (e.g., stretching) within the second image display area 50C corresponding to the operation mode by another user. Note that if the second image 52 is not superimposed on the distribution video display area 50A, it may be superimposed on the first image display area 50B from the second image display area 50C.

[0096] [2-3-4. Combined Image Display] The mobile terminal 3 according to the embodiment has a combined image display function of displaying a combined image on the screen 3a according to the combination of an operation for displaying the first image 51 by a user and an operation for displaying the first image 51 by another user.

[0097] Thereby, the mobile terminal 3 can display the reactions of the user and other users to the video on the screen with higher entertainment value. In addition, an image that is not displayed by the operation of one user is displayed by the combination with the operation of another user, so the mobile terminal 3 can encourage more users to watch the content.

[0098] In addition, the combined image is displayed superimposed on the video being displayed on screen 3a. That is, the combined image is displayed superimposed not only on the first image display area 50B, but also on the distribution video display area 50A and the second image display area 50C. As a result, the user can easily recognize that the combined image has been displayed, and the entertainment value of watching the video is further enhanced.

[0099] When the combined image is displayed on screen 3a, it is displayed so that the user can recognize that it is a combined image. For example, the user ID of another user may be displayed on screen 3a together with the combined image so that it can be recognized that another user performed a display operation at the same timing.

[0100] FIG. 7 is a timing chart showing the case where the combined image is displayed on screen 3a.

[0101] The combined image display function according to the embodiment causes the combined image to be displayed on screen 3a when the timing of the user's operation and the operation of another user is within a predetermined time (hereinafter referred to as the "combination period"). That is, when a plurality of users perform a display operation of the first image 51 within the combination period, it is understood as the reaction of the users who watched the same scene. Therefore, since the combined image display function displays the combined image based on the reactions of a plurality of users according to the scene of the video, the entertainment value of the user watching the video is further enhanced.

[0102] The start period of the combination period is the timing when a certain user starts a display operation of the first image 51. That is, as shown in FIG. 7(A), when user B performs a display operation within the combination period starting from the start timing of the display operation by user A, the combined image is displayed on the screens 3a of the mobile terminals 3 of users A and B.

[0103] As an example, the display period of the combined image is set until the display operation by the user ends. In the example of FIG. 7(A), the combined image is displayed on the screens 3a of the mobile terminals 3 of users A and B from the start to the end of the display operation by user B.

[0104] In the example of FIG. 7(B), although it is within the combination period, a case is shown where, after the display operation of user A, which has reached the start timing, has ended, a display operation by user C is performed. In the example of FIG. 7(B), the combined image is displayed on the screen 3a of the mobile terminals 3 of users A, B, and C from the start of the display operation of user B until the end of the display operation of user C.

[0105] Note that the display mode of the combined image shown in FIG. 7 is an example and is not limited thereto. For example, there may be no concept of a combination period, and the combined image may be displayed on the screen 3a at the timing when the display operation of a user overlaps with the display operation of another user. Also, a display time may be predetermined for each combined image, and the display of the combined image may end at the timing when the said display time has elapsed.

[0106] Also, the combined image according to the embodiment is displayed on the screen 3a instead of the first image 51. In the example of FIG. 7(A), first, the first image 51 corresponding to the display operation of user A is displayed on the screen 3a of the mobile terminal 3 of user A. However, when the display operation of user B starts and the display of the said first image 51 ends, the combined image is displayed on the screen 3a of the mobile terminal 3 of user A. On the other hand, on the mobile terminal 3 of user B, the first image 51 that user B himself / herself tried to display is not displayed on the screen 3a, and the combined image is displayed. In this way, since the combined image is displayed on the screen 3a instead of the first image 51, an image different from the first image 51 that the user himself / herself tried to display is displayed on the screen 3a, so the entertainment of the user watching the video is further enhanced.

[0107] Also, the combined image according to the embodiment is displayed on the screen 3a when the first image 51 selected by the user and the first image 51 selected by another user are preset images. Thereby, since there is a possibility that a combined image unexpected by the user himself / herself is displayed according to the image selection by the user himself / herself, the entertainment by watching the video is further enhanced.

[0108] For example, the combined image is displayed on the screen 3a when the first image 51 selected by the user is the same as the first image 51 selected by another user. That is, when users watching a video select the same first image 51, it is highly likely that the users are experiencing similar emotions. Therefore, by displaying the combined image, the user can recognize that they have empathized with other users while watching the video. More specifically, for example, when the first image 51 selected by the user is the same first image 51 as the first image 51 selected by another user, an enlarged image of the first image 51 is displayed on the screen 3a as the combined image. Note that this is not the only case, and a special image that is not displayed by operating the display button 53 may be displayed as the combined image.

[0109] Also, even if the first image 51 selected by the user is a different first image 51 from the first image 51 selected by another user, if the combination is a preset combination, the combined image may be displayed. In this case, for example, a special image that is not displayed by operating the display button 53 is displayed as the combined image.

[0110] Note that in the combined image display function according to the embodiment, in the second image display area 50C of the mobile terminal 3 of a user for whom the combined image is not displayed, a second image based on the combined image displayed on the screen 3a of another user is not displayed. However, this is not the only case, and a second image based on the combined image displayed on the screen 3a of another user may be displayed in the second image display area 50C of the mobile terminal 3 of a user for whom the combined image is not displayed.

[0111] [2-4. Replacement Process] The server 4 according to the embodiment has a replacement processing function that repeatedly performs a unit process of identifying two second images 52 whose display timings are close from among the second images as an image group and replacing the image group with one new second image 52 until no image group can be identified. Thereby, the number of second images 52 displayed during video playback on the mobile terminal 3 can be reduced, and a sudden increase in load on the mobile terminal 3 can be suppressed.

[0112] FIG. 8 is a timing chart showing the case where the second image is replaced.

[0113] On the premise that the time line of the video is divided into a plurality of sections. The length of each section is, for example, the same as the length of the combination period. The replacement processing function according to the embodiment, for example, specifies two second images 52 having commonality in both the subject and the object from among a plurality of second images 52 whose start periods of the display periods are respectively included in the same section, and invalidates the two second images 52 included in the image group while associating one new second image 52 with the time line of the video. This unit process is repeated until no image group can be specified.

[0114] Attributes (second image display information) that can specify the subject (user ID of the user who added the second image to the video and the user group to which the user belongs), the object (type of the second image and the element group to which the type belongs), the display period, the number of syntheses, the status, etc. are added to the second image 52. The type is determined by the object and the number of syntheses. The design of the second image 52 is determined by the type. It is preferable that the number of syntheses is reflected in the appearance of the design, such as a number indicating the number of syntheses being displayed. As shown in FIG. 8(A), there may be a case where the start period of the display period of the second image A and the start period of the display period of the second image B are both included in the nth section (n is an appropriate natural number) on the time line. And it is assumed that the type of the second image A and the object of the second image B are the same. At this time, the replacement processing function specifies the second image A and the second image B as an image group, invalidates the statuses of the second image A and the second image B respectively, and associates one new second image C with the time line of the video.

[0115] The second image 52 (second image C) after replacement inherits the additional attributes of the second image 52 (second images A and B) before replacement. As an example, the second image 52 after replacement inherits the subject and object as they are. As an example, the number of composite times of the second image 52 after replacement is the total value of the number of composite times of each second image 52 before replacement. As an example, the display period of the second image 52 after replacement is the period from the earlier start period to the later end period. As shown in FIG. 8(A), the display period of the second image C corresponds to the period from the start period of the second image A to the end period of the second image B.

[0116] The second image 52 before replacement is invalidated and the association with the video is released. On the other hand, the second image 52 after replacement can form an image group together with other second images 52 and be replaced with a new second image 52. As shown in FIG. 8(B), both the start period of the display period of the second image C and the start period of the display period of the second image D are included in the n-th section (n is an appropriate natural number) on the timeline. And the user who added the second image C and the user who added the second image D belong to the same user group, and it is assumed that the types of the second image C and the second image D are the same. At this time, the replacement processing function identifies the second image C and the second image D as an image group, invalidates the statuses of the second image C and the second image D respectively, and associates one new second image E with the timeline of the video. The display period of the second image E is the period from the start period of the second image C to the end period of the second image D.

[0117] [2-5. Functional Blocks] Next, the combined image display and replacement processing according to the embodiment will be described using a functional block diagram. Note that the function related to the combined image display in the mobile terminal 3 is included in the onomatopoeia display function, and the functions related to the combined image display and replacement processing in the server 4 are included in the video distribution function respectively.

[0118] [2-5-1. Functional Blocks Related to the Video Distribution Function of the Server] FIG. 9 is a functional block diagram related to the video distribution function of the server 4 according to the embodiment.

[0119] The CPU 20 included in the server 4 includes a group registration unit 60, a video distribution unit 61, a first image display information reception control unit 62, a second image display information generation unit 63, a second image display information transmission control unit 64, a combined image information generation unit 65, a combined image information transmission control unit 66, and a replacement processing unit 67. Each function included in the CPU 20 is realized by a program stored in the HDD 26.

[0120] The group registration unit 60 receives registration of a group 5 composed of a plurality of mobile terminals 3. Note that, as an example, the registration of the group 5 is performed by registering a user ID.

[0121] The video distribution unit 61 distributes a video to the mobile terminal 3 in response to a request from the mobile terminal 3.

[0122] The first image display information reception control unit 62 receives the first image display information received from the mobile terminal 3 via the external interface 32, and stores the first image display information in the HDD 26 in association with the corresponding video.

[0123] The first image display information is information regarding a first image 51 displayed on the screen 3a of the mobile terminal 3, and is transmitted from the mobile terminal 3 to the server 4 at the timing when the first image 51 is displayed on the mobile terminal 3. The first image display information is, for example, the type of the displayed first image 51, the user ID indicating the user who displayed the first image 51, the operation mode of the user performed to display the first image 51, the elapsed time from the start of the video when the first image 51 is displayed, and the like.

[0124] The second image display information generation unit 63 generates second image display information based on the received first image display information, and stores the second image display information in the HDD 26 in association with the corresponding video.

[0125] The second image display information is information regarding a second image 52 to be displayed in the second image display area 50C, and includes the type of the second image 52, the user ID of the user who displayed the first image 51 that is the source of the second image 52 on his or her own mobile terminal 3, the display period, the number of syntheses, the status, and the like.

[0126] The second image display information transmission control unit 64 controls the external interface 32 to transmit the second image display information to the mobile terminal 3. The transmission destination of the second image display information is another mobile terminal 3 different from the mobile terminal 3 that transmitted the first image display information.

[0127] The combined image information generation unit 65 performs a combination determination based on the received first image display information. Then, the combined image information generation unit 65 generates combined image information indicating a combined image according to the result of the combination determination, and stores the combined image information in the HDD 26.

[0128] The combined image information is information regarding the combined image to be displayed on the screen 3a, and includes the type of the combined image, the user IDs of a plurality of users who performed display operations at the same timing, and the like.

[0129] In addition, the combination determination is performed for each first image display information, and it is determined whether or not the display operation of the first image 51 indicated by the first image display information is within a combination period based on the start timing of the display operation of the first image 51 by another user.

[0130] The combined image information transmission control unit 66 controls the external interface 32 to transmit the combined image information to the mobile terminal 3. Note that the mobile terminal 3 to which the combined image information is transmitted is a plurality of mobile terminals 3 corresponding to the user ID included in the combined image information.

[0131] The replacement processing unit 67 executes replacement processing.

[0132] [2-5-2. Functional Blocks Related to Onomatopoeia Display of Mobile Terminal] FIG. 10 is a functional block diagram related to onomatopoeia display of the mobile terminal 3 according to the embodiment.

[0133] The main control unit 40 provided in the mobile terminal 3 includes an image display control unit 70, a display area setting unit 71, a first image display operation reception unit 72, an operation state determination unit 73, a first image display information generation unit 74, a first image display information transmission control unit 75, a second image display information reception control unit 76, and a combined image information reception control unit 77. Each function provided in the main control unit 40 is realized by a program stored in the auxiliary storage unit 44.

[0134] The image display control unit 70 controls the touch panel display 3a to display an image based on the image data. The image display control unit 70 according to the embodiment causes the video distributed from the server 4 to be displayed in the distribution video display area 50A, causes the first image 51 to be displayed in the first image display area 50B according to the user's operation, and causes the second image 52 based on the first image 51 displayed by another user on another mobile terminal 3 to be displayed in the second image display area 50C. Further, the image display control unit 70 according to the embodiment causes the first image 51 to be superimposed and displayed on the distribution video display area 50A beyond the first image display area 50B according to the user's operation mode.

[0135] The display area setting unit 71 sets the positions and sizes (vertical and horizontal lengths) of the distribution video display area 50A, the first image display area 50B, and the second image display area 50C as the display mode of the screen 3a when a video is distributed from the server 4. Although these positions and sizes are predetermined, the user may set them arbitrarily.

[0136] The first image display operation reception unit 72 receives the operation of the display button 53 by the user in a state where the screen 3a is displaying a video. That is, the first image display operation reception unit 72 recognizes the display button 53 operated by the user among the plurality of display buttons 53, and causes the first image 51 corresponding to the display button 53 to be displayed on the screen 3a via the image display control unit 70. Note that the first image display operation reception unit 72 causes the first image 51 to be displayed on the screen 3a so that the display mode of the first image 51 changes according to the determination result by the operation state determination unit 73.

[0137] The operation state determination unit 73 determines the operation state of the display button 53 by the user. The operation state here is, for example, an operation mode including the long-press time of the display button 53.

[0138] The first image display information generation unit 74 generates first image display information corresponding to the first image 51 displayed on the screen 3a by the operation of the user.

[0139] The first image display information transmission control unit 75 controls the communication unit 46 to transmit the first image display information to the server 4.

[0140] The second image display information reception control unit 76 outputs the second image display information received from the server 4 via the communication unit 46 to the image display control unit 70.

[0141] The combined image information reception control unit 77 outputs the combined image information received from the server 4 via the communication unit 46 to the image display control unit 70. Thereby, the image display control unit 70 displays the combined image on the screen 3a where the video is being displayed. Note that the image display control unit 70 according to the embodiment displays the combined image on the screen 3a instead of the first image 51 according to the operation of the display button 53 by the user.

[0142] [2-6. Processing flow] Next, the flow of the onomatopoeia display process according to the embodiment in the mobile terminal 3 and the server 4 will be described using a flowchart.

[0143] [2-6-1. Flow related to onomatopoeia display in the mobile terminal] FIG. 11 is a flowchart showing the flow of the onomatopoeia display process executed by the main control unit 40 provided in the mobile terminal 3. A program for executing the onomatopoeia display process is stored in advance in a predetermined area of the auxiliary storage unit 44. The onomatopoeia display process is started, for example, when the mobile terminal 3 starts playing a video distributed from the server 4, and ends when the video playback ends.

[0144] In step S100, the occurrence of a trigger is detected. The trigger in this embodiment is either a first trigger (receiving an operation of the display button 53 by the user) or a second trigger (receiving second image display information). Each time the occurrence of a trigger is detected in the mobile terminal 3 according to this embodiment, the display content of the screen is updated. The first trigger and the second trigger may occur in parallel. Therefore, during the execution of the process in response to the occurrence of one trigger, the process in response to the occurrence of the other trigger may be executed. Also, a plurality of second triggers may occur in parallel. Therefore, during the execution of the process in response to the occurrence of one second trigger, the process in response to the occurrence of the other second trigger may be executed.

[0145] In step S102, if the detected trigger is the first trigger, the process proceeds to step S106, and if the detected trigger is the second trigger, the process proceeds to step S104.

[0146] In step S104, the image display control unit 70 causes the second image 52 indicated by the second image display information to be displayed in the second image display area 50C, and the process ends.

[0147] In step S106, the image display control unit 70 causes the first image 51 corresponding to the display button 53 operated by the user to be displayed in the first image display area 50B. Note that the first image 51 may be stored in the mobile terminal 3 in advance, or may be downloaded from the server 4 each time the display button 53 is operated by the user.

[0148] In step S108, the first image display information generation unit 74 generates the first image display information corresponding to the first image 51 displayed in the first image display area 50B.

[0149] In step S110, the first image display information transmission control unit 75 controls the communication unit 46 to transmit the first image display information to the server 4.

[0150] In step S112, the operation state determination unit 73 determines whether or not the operation of the display button 53 by the user has ended. If the determination is affirmative, the process proceeds to step S120; if the determination is negative, the process proceeds to step S114.

[0151] In step S114, the combined image information reception control unit 77 determines whether or not the combined image information has been received via the communication unit 46. If the determination is affirmative, the process proceeds to step S118; if the determination is negative, the process proceeds to step S116.

[0152] In step S116, the image display control unit 70 extends and displays the first image 51 (onomatopoeia image) until the operation of the display button 53 by the user ends, and then returns to step S112.

[0153] In step S118, the image display control unit 70 displays the combined image indicated by the combined image information on the screen 3a, and then returns to step S112.

[0154] In step S120, the image display control unit 70 ends the display of the first image 51 and ends the process.

[0155] Note that in the flowchart shown in FIG. 11, the first image display information is generated along with the display of the first image 51 and is transmitted to the server 4. However, this is not the only case, and the first image display information may be transmitted to the server 4 after the display of the first image 51 ends. Further, the first image display information transmission control unit 75 may transmit the first image display information indicating the start of the display of the first image 51 to the server 4 at the timing when the display of the first image 51 starts, and transmit the first image display information indicating the end of the display of the second image 52 to the server 4 at the timing when the display of the first image 51 ends. Thereby, the server 4 can recognize that the first image 51 (onomatopoeia image) is being extended and displayed during the reception interval between the first image display information indicating the start of the display of the first image 51 and the first image display information indicating the end of the display.

[0156] [2-6-2. Flowchart Regarding Onomatopoeia Display on Server] FIG. 12 is a flowchart showing the flow of onomatopoeia display processing executed by the CPU 20 included in the server 4. A program for executing the onomatopoeia display processing is stored in advance in a predetermined area of the HDD 26. The onomatopoeia display processing is started as an example when the server 4 is delivering a video to the mobile terminal 3, and ends when the delivery of the video ends.

[0157] In step S200, the first image display information reception control unit 62 determines whether the first image display information has been received from the mobile terminal 3. If the determination is affirmative, the process proceeds to step S202, and if the determination is negative, the process enters a waiting state.

[0158] In step S202, the first image display information reception control unit 62 stores the received first image display information in the HDD 26.

[0159] In step S204, the combined image information generation unit 65 performs a combination determination. If the determination is affirmative, the process proceeds to step S208, and if the determination is negative, the process proceeds to step S206. That is, when the display operation of the first image 51 to be combined is performed by a plurality of users within the combination period, the determination is affirmative and the process proceeds to step S208. On the other hand, when the display operation of the first image 51 to be combined is not performed by a plurality of users within the combination period, the determination is negative and the process proceeds to step S206.

[0160] In step S206, based on the received first image display information, the second image display information generation unit 63 generates the second image display information and stores it in the HDD 26.

[0161] In step S208, the combined image information generation unit 65 generates combined image information indicating a combined image corresponding to the display operation of the first image 51 to be combined.

[0162] In step S210, the combined image information transmission control unit 66 controls the external interface 32 to transmit the combined image information and returns to step S200. Note that the transmission destination of the combined image information is the plurality of mobile terminals 3 that have transmitted the first image display information to be combined, that is, the mobile terminals 3 corresponding to the plurality of user IDs included in the combined image information.

[0163] FIG. 13 is a flowchart showing the flow of the onomatopoeia display process executed by the CPU 20 provided in the server 4. The flow shown in FIG. 13 targets the second image display information whose start period of the display period is included in the section each time the end period of the section in the time line of the video arrives.

[0164] In step S300, the replacement processing unit 67 makes a determination. In the case of an affirmative determination, it proceeds to step S302, and in the case of a negative determination, it proceeds to step S304. That is, when there is second image display information having commonality in both the subject and the object, an affirmative determination is made and it proceeds to step S302. On the other hand, when there is no second image display information having commonality in both the subject and the object, a negative determination is made and it proceeds to step S304.

[0165] In step S302, the replacement processing unit 67 performs replacement processing. In the unit process of the replacement processing, the replacement processing unit 67 identifies two pieces of second image display information having commonality in both the subject and the object as an image group, invalidates the statuses of the two pieces of second image display information included in the image group, and generates one new piece of second image display information. In step S302, this unit process is repeated until there are no two pieces of second image display information having commonality in both the subject and the object.

[0166] In step S304, the second image display information transmission control unit 64 controls the external interface 32 to transmit all the second image display information included in the target section to a mobile terminal 3 different from the mobile terminal 3 that has transmitted the first image display information. Therefore, the display period of the second image 52 based on the second image display information on the mobile terminal 3 will be delayed compared to the display period associated with the video.

[0167] As described above, for the mobile terminal 3 according to the embodiment, operation information of an addition operation performed on the first mobile terminal 3 by a first user who is viewing a video displayed on the screen 3a of the first mobile terminal 3 to add a first image 51 to the video, and operation information of an addition operation performed on the second mobile terminal 3 by a second user who is viewing the video displayed on the screen 3a of the second mobile terminal 3 to add the first image 51 to the video are provided from each mobile terminal 3 to the server 4. The server 4 causes a combined image based on a combination of the addition operation by the first user and the addition operation by the second user to be displayed on the screen 3a of the first mobile terminal 3 and the screen 3a of the second mobile terminal 3, respectively, in order to improve entertainment. As a result, an image that is not displayed by the operation of one user is displayed by a combination with the operation of another user, so that the mobile terminal 3 can encourage more users to view the video.

[0168] Also, for the server 4 according to the embodiment, during the playback of a video, two or more second images 52 whose display timings are close and which have commonality in the subject and object among the second images 52 associated with the video with respect to the timing of being displayed on the mobile terminal 3 of the user viewing the video are replaced with one new second image 52. As a result, the number of second images 52 to be displayed on the mobile terminal 3 is reduced, so that the server 4 can suppress a sudden increase in load on the mobile terminal 3.

[0169] [3. Modification Example] The technical scope of the present invention is not limited to the scope described in the above embodiment. Various changes or improvements can be made to the above embodiment without departing from the gist of the invention, and the forms to which such changes or improvements are made are also included in the technical scope of the present invention.

[0170] Also, the flow of various processes described in the above embodiment is also an example, and unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed within the scope not departing from the gist of the present invention.

[0171] [3-1. Modification Example 1 (Sharing Images Without Going Through a Server)] In the above embodiment, the server 4 receives the first image display information from the mobile terminal 3, transmits the second image display information generated based on the first image display information to another mobile terminal 3, and the other mobile terminal 3 displays the second image 52 indicated by the second image display information in the second image display area 50C. However, the present invention is not limited to this.

[0172] For example, by transmitting and receiving the first image display information between a plurality of mobile terminals 3 without going through the server 4, the mobile terminal 3 may display the second image 52 in the second image display area 50C. That is, when the mobile terminal 3 receives the first image display information from another mobile terminal 3, it reads out the information about the first image 51 displayed on the other mobile terminal 3 from the first image display information, and displays the second image 52 based on the information in the second image display area 50C.

[0173] [3-2. Modification Example 2 (Watching a Video Alone)] The above embodiment has been described by taking as an example the case of sharing an image representing the emotion towards a video among a plurality of users who are watching the video simultaneously. In contrast, an image representing the emotion of a user who has watched a certain video first can be shared with a user who will watch the video later. Thereby, a combined image can be displayed according to the combination of the additional operation by the first user who watched first and the additional operation by the second user who watched later.

[0174] Hereinafter, consider the case where the timing at which a first user using the first mobile terminal 3 (first device) watches a certain video (video to be watched) is later than the timing at which a second user using the second mobile terminal 3 (second device) watches the video (Fig. 13(A)). In this case, the additional operation by the first user and the additional operation by the second user may not be in a proximity relationship in real time. However, they are in a proximity relationship on the progress axis (timeline) of the video (Fig. 13(B)). The first user and the second user may be the user himself / herself and another person, or the past user himself / herself and the current user himself / herself.

[0175] In the second modification example, in order for the second user who is viewing the video displayed on the screen 3a (second screen) of the second mobile terminal 3 to add the first image (second element) to the video, the operation information of the addition operation (second operation) performed on the second mobile terminal 3a is sent from the second mobile terminal 3a to the server 4 (second instruction means, shared information processing device). The server 4 pre-transmits the second image display information (second instruction) for causing the second image (second element) to be displayed at a timing corresponding to the addition operation to the first mobile terminal 3 used by the first user who intends to view the same video.

[0176] In order to improve the entertainment, when the first mobile terminal 3 receives an addition operation (first operation) performed on the first mobile terminal 3a by the first user who is viewing the video displayed on the screen 3a (first screen) of the first mobile terminal 3 to add the first image (first element) to the video, the first mobile terminal 3 displays a combined image (third element) based on the combination of the received addition operation (first operation) and the addition operation (second operation) related to the second image display information on the screen 3a of the first mobile terminal 3. In the second modification example, since the timing of the second operation performed on the second mobile terminal 3 is known in advance, after the first mobile terminal 3 receives the first operation, it is also possible to display, without delay, the combined image (third element) based on the combination of the received addition operation (first operation) and the addition operation (second operation) related to the second image display information on the screen 3a of the first mobile terminal 3.

[0177] [3-3. Second Modification Example (When the combined image is displayed, the first image is not displayed)] In the above embodiment, the first image is also displayed when the combined image is displayed. In contrast, the first image may not be displayed when the combined image is displayed.

[0178] Specifically, the image display control unit 70 (second control means) of the mobile terminal 3 waits for the reception of the combined image information until a predetermined period (combination period) elapses after the additional operation is received by the first image display operation reception unit 72 (first reception means). Then, when the combined image information is received within the predetermined time, the image display control unit 70 displays a combined image (third element) based on the received combined image information, and when the combined image information is not received within the predetermined period, the first image (first element) is displayed based on the received additional operation.

[0179] [3-4. Modification Example 4 (Displaying the Second Image Even When the Combined Image is Displayed)] In the above embodiment, the second image is not displayed when the combined image is displayed. In contrast, the second image may also be displayed when the combined image is displayed. Specifically, when the server 4 receives the first image information, it may generate the second image display information regardless of the presence or absence of the display operation to be combined and transmit it to the mobile terminal 3.

[0180] [3-5. Modification Example 5 (Limiting the Activation of the Replacement Process)] In the above embodiment, the unit process is repeated every predetermined period (combination period), and a plurality of second image display information to be replaced is replaced with one second image display information. In contrast, in a situation where a sudden increase in load is unlikely to occur in the mobile terminal 3, the replacement process may be omitted.

[0181] Specifically, when the number of elements in a proximity element group including a plurality of second image display information (shared element) whose display timings are close does not satisfy the activation condition, the replacement processing unit 67 (specification means) of the server 4 omits the unit processing (replacement processing) of specifying two pieces (predetermined number) of second image display information included in the proximity element group as an image group (element group). The activation condition is, for example, that the number of elements is equal to or greater than a predetermined threshold. The activation condition may be flexibly varied according to the processing capacity of the mobile terminal 3. For example, a predetermined threshold may be set according to the processing capacity of the mobile terminal with a lower capacity among the plurality of mobile terminals 3 belonging to the group 5. Note that the processing capacity is, for example, the display processing capacity (clock frequency of the DSP) or the reception processing capacity (speed of the communication line) of the mobile terminal 3.

Explanation of Signs

[0182] 1 Video distribution system 2 Communication line 3 Mobile terminal (an example of the "first device", "second device", "information processing device") 3a Screen (an example of the "first screen", "second screen") 4 Server (an example of the "first instruction means", "second instruction means", "shared information processing device") 5 Group (an example of the "information processing system") 50A Distribution video display area 50B First image display area 50C Second image display area 51 First image 52 Second image (an example of the "shared element", "alternative element") 53 Display button 60 Group registration unit 61 Video distribution unit 62 First image reception information reception control unit 63 Second image display information generation unit 64 Second image display information transmission control unit 65 Composite image information generation unit 66 Composite image information transmission control unit 67 Replacement processing unit (an example of the "specification means", "replacement means") 70 Image display control unit (an example of the "first control means", "second control means", "third control means", "display control means") 71 Display area setting unit 72 First image display operation reception unit (an example of the "first reception means", "second reception means") 73 Operation state determination unit 74 First image display information generation unit 75 First image display information transmission control unit (an example of the "first provision means", "second provision means") 76 Second image display information reception control unit (an example of the "third reception means") 77 Composite image information reception control unit

Claims

1. a processor, the processor comprising: Displaying a common viewing target on terminals of a plurality of users belonging to the same user group; Displaying multiple operation areas on a user's device; receiving a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users using a terminal displaying the common viewing target; associate, based on an operation timing at which each of the selection operations is accepted, a display timing at which each of the elements determined based on each of the selection operations is displayed with the viewing target; When the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, instead of the elements of the same type, displaying alternative elements in a number less than the number of the elements of the same type on the terminals of the plurality of users currently displaying the viewing target; Accepting registrations for the user group from a plurality of users; The alternative element is displayed on terminals of a plurality of users who are registered in the same user group, and the alternative element is not displayed on terminals of users who are not registered in the user group and who view the viewing target. Information processing device.

2. a processor, the processor comprising: Displaying a common viewing target on terminals of a plurality of users belonging to the same user group; Displaying multiple operation areas on a user's device; receiving a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users using a terminal displaying the common viewing target; associate, based on an operation timing at which each of the selection operations is accepted, a display timing at which each of the elements determined based on each of the selection operations is displayed with the viewing target; When the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, instead of the elements of the same type, displaying alternative elements in a number less than the number of the elements of the same type on the terminals of the plurality of users currently displaying the viewing target; the predetermined threshold is set according to the processing capabilities of the terminals of the users belonging to the user group; Information processing device.

3. A processor displays a common viewing target on terminals of a plurality of users belonging to the same user group; A processor displays a plurality of operation areas on a user's terminal; A processor receives a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users using terminals displaying the common viewing target; a processor associates, with the viewing target, a display timing for displaying each of the elements determined based on each of the selection operations based on an operation timing at which each of the selection operations is accepted; When the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, a processor displays, instead of the elements of the same type, alternative elements whose number is less than the number of the elements of the same type on the terminals of the users currently displaying the viewing target; A processor accepts registrations from a plurality of users to the user group; a processor causes the alternative element to be displayed on terminals of a plurality of users who are registered in the same user group, and does not cause the alternative element to be displayed on terminals of users who are not registered in the user group and who view the viewing target; Information processing methods.

4. A processor displays a common viewing target on terminals of a plurality of users belonging to the same user group; A processor displays a plurality of operation areas on a user's terminal; A processor receives a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users using terminals displaying the common viewing target; a processor associates, with the viewing target, a display timing for displaying each of the elements determined based on each of the selection operations based on an operation timing at which each of the selection operations is accepted; When the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, a processor displays, instead of the elements of the same type, alternative elements whose number is less than the number of the elements of the same type on the terminals of the users currently displaying the viewing target; the predetermined threshold is set according to the processing capabilities of the terminals of the users belonging to the user group; Information processing methods.

5. The processor: A function of displaying a common viewing target on terminals of multiple users belonging to the same user group; A function for displaying multiple operation areas on a user's device; A function of accepting a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users who use a terminal displaying the common viewing target; a function of associating, with the viewing target, a display timing for displaying each of the elements determined based on each of the selection operations, based on an operation timing at which each of the selection operations is accepted; a function of displaying, in a case where the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, alternative elements, the number of which is less than the number of the elements of the same type, instead of the elements of the same type, on the terminals of the users currently displaying the viewing target; A function of accepting registration of a plurality of users to the user group; A function of displaying the alternative element on terminals of a plurality of users who are registered in the same user group, and not displaying the alternative element on terminals of users who are not registered in the user group and who view the viewing target; A program that executes the following.

6. The processor: A function of displaying a common viewing target on terminals of multiple users belonging to the same user group; A function for displaying multiple operation areas on a user's device; A function of accepting a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users who use a terminal displaying the common viewing target; a function of associating, with the viewing target, a display timing for displaying each of the elements determined based on each of the selection operations, based on an operation timing at which each of the selection operations is accepted; a function of displaying, in a case where the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, alternative elements, the number of which is less than the number of the elements of the same type, instead of the elements of the same type, on the terminals of the users currently displaying the viewing target; Run the command, the predetermined threshold is set according to the processing capabilities of the terminals of the users belonging to the user group; program.

7. Equipped with a server and terminals, The server, Displaying a common viewing target on terminals of a plurality of users belonging to the same user group; Displaying multiple operation areas on a user's device; receiving a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users using a terminal displaying the common viewing target; associate, based on an operation timing at which each of the selection operations is accepted, a display timing at which each of the elements determined based on each of the selection operations is displayed with the viewing target; When the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, instead of the elements of the same type, displaying alternative elements in a number less than the number of the elements of the same type on the terminals of the plurality of users currently displaying the viewing target; Accepting registrations for the user group from a plurality of users; The alternative element is displayed on terminals of a plurality of users who are registered in the same user group, and the alternative element is not displayed on terminals of users who are not registered in the user group and who view the viewing target. system.

8. Equipped with a server and terminals, The server, Displaying a common viewing target on terminals of a plurality of users belonging to the same user group; Displaying multiple operation areas on a user's device; receiving a selection operation for selecting one operation area from the plurality of operation areas from a plurality of users using a terminal displaying the common viewing target; associate, based on an operation timing at which each of the selection operations is accepted, a display timing for displaying each of the elements determined based on each of the selection operations with the viewing target; When the number of the elements of the same type whose display timing is close to each other is equal to or greater than a predetermined threshold value, instead of the elements of the same type, displaying alternative elements in a number less than the number of the elements of the same type on the terminals of the plurality of users currently displaying the viewing target; the predetermined threshold is set according to the processing capabilities of the terminals of the users belonging to the user group; system.

Images