Information Processing Apparatus, Information Processing Method, and Program

Abstract

The present disclosure relates to an information processing device, an information processing method, and a program that make it possible to realize more realistic live streaming of events. A display control unit displays, on the local terminal, a plurality of viewpoint images having been photographed at different viewpoints in one event, and an avatar information acquisition unit acquires avatar information regarding the display of avatars which correspond to the local user viewing the plurality of viewpoint images displayed at the local terminal and other users viewing the plurality of viewpoint images at other terminals, respectively. Meanwhile, the display control unit controls display of the avatar based on avatar information, in correspondence to the viewpoint images displayed at the local terminal. The present disclosure can be applied, for example, to a smartphone, etc.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program, and more particularly to an information processing apparatus, an information processing method, and a program that can realize a more realistic live distribution of events.

Background Art

[0002] Conventionally, there has been a technique for displaying an avatar, which is a character serving as the user's own avatar, in a virtual reality space or an augmented reality space.

[0003] For example, Patent Document 1 discloses a video synthesis method for synthesizing an avatar based on an image of a user captured by an in-camera into a real video captured by an out-camera in a smartphone or the like.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in events such as sports games and music festivals, users can participate in the event from various viewpoints. On the other hand, in services that live-distribute such events, only images captured from one viewpoint are distributed.

[0006] The present disclosure has been made in view of such a situation, and aims to realize a more realistic live distribution of events.

Means for Solving the Problems

[0007] The information processing apparatus of the present disclosure includes a display control unit that displays a plurality of viewpoint images captured from different viewpoints in one event on the self-terminal, a self-user who views the plurality of viewpoint images displayed on the self-terminal, and an avatar information acquisition unit that acquires avatar information regarding the display of an avatar corresponding to each of the other users who are viewing the plurality of viewpoint images on the other terminal. The display control unit is an information processing apparatus that controls the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on the self-terminal.

[0008] The information processing method of the present disclosure is an information processing method in which an information processing apparatus displays a plurality of viewpoint images captured from different viewpoints in one event on the self-terminal, acquires avatar information regarding the display of an avatar corresponding to each of the other users who are viewing the plurality of viewpoint images on the other terminal, and controls the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on the self-terminal.

[0009] The program of the present disclosure causes a computer to display a plurality of viewpoint images captured from different viewpoints in one event on the self-terminal, acquire avatar information regarding the display of an avatar corresponding to each of the other users who are viewing the plurality of viewpoint images on the other terminal, and execute a process of controlling the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on the self-terminal.

[0010] In the present disclosure, a plurality of viewpoint images captured from different viewpoints in one event are displayed on the self-terminal, avatar information regarding the display of an avatar corresponding to each of the other users who are viewing the plurality of viewpoint images on the other terminal is acquired, and the display of the avatar based on the avatar information is controlled corresponding to the viewpoint image displayed on the self-terminal.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Modes for Carrying Out the Invention

[0012] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be made in the following order.

[0013] 1. Overview of the image distribution system 2. Example of the functional configuration of each device 3. Display form of the perspective image 4. Animation display of the avatar 5. Priority display of the group and the avatar 6. Other display variations 7. Example of the computer configuration

[0014] <Overview of the Image Distribution System> FIG. 1 is a diagram showing an overview of an image distribution system according to an embodiment of the present disclosure.

[0015] As shown in FIG. 1, the image distribution system 1 is configured to include a plurality of shooting-side terminals 10-1, 10-2, 10-3, ···, a server 20, and a plurality of viewing-side terminals 30-1, 30-2, 30-3, ···.

[0016] The shooting-side terminals 10-1, 10-2, 10-3, ··· are each operated by a different user, but when there is no need to distinguish between them, they are simply referred to as the shooting-side terminal 10. Similarly, the viewing-side terminals 30-1, 30-2, 30-3, ··· are also each operated by a different user, but when there is no need to distinguish between them, they are simply referred to as the viewing-side terminal 30.

[0017] The shooting-side terminal 10, the server 20, and the server 20 and the viewing-side terminal 30 are each connected to each other via the Internet.

[0018] The shooting-side terminal 10 shoots perspective images from different perspectives (angles) at one event such as sports watching or a music festival, and uploads them to the server 20. In the following, an example in which perspective images are shot at each point in a stadium where a soccer game is being held will be described.

[0019] The server 20 combines a plurality of perspective images uploaded by the shooting-side terminal 10 into one image. Then, the server 20 distributes the combined image to each of the viewing-side terminals 30. Note that the server 20 may distribute the plurality of perspective images uploaded by the shooting-side terminal 10 to each of the viewing-side terminals 30 as individual images as they are.

[0020] This allows each of the users U1, U2, U3, etc. of the shooting terminals 10-1, 10-2, 10-3, etc. to switch and view a viewpoint image of a desired angle at a desired timing among viewpoint images shot from different viewpoints during an event.

[0021] Here, an overview of image delivery in the image delivery system 1 will be described with reference to FIG.

[0022] At the shooting side shown in Fig. 2, six viewpoint images 40A to 40F are shot by six shooting side terminals 10. Viewpoint image 40A is a moving image overlooking the entire stadium, and viewpoint image 40B is a moving image with spectators in the spectator seats as subjects. Viewpoint image 40C is a moving image with a goalkeeper as a subject, and viewpoint image 40D is a moving image with a dribbling player as a subject. Viewpoint image 40E is a moving image with the dribbling player of viewpoint image 40D and a player of the opposing team as subjects, and viewpoint image 40F is a moving image with another player of the same team as the dribbling player of viewpoint image 40D as a subject.

[0023] The viewpoint images 40A to 40F are uploaded to the server 20 in real time and distributed to the image capturing side (image capturing side terminal 10) shown on the right side of FIG.

[0024] On the photographing side (photographing side terminal 10), the viewpoint images 40A to 40F are displayed together on one screen.

[0025] In the example of Fig. 2, among the viewpoint images 40A to 40F, the viewpoint image 40E is displayed large in the center of the screen, and the other viewpoint images 40A to 40D, 40F are displayed small side by side at the top of the screen. When one of the other viewpoint images 40A to 40D, 40F is selected by the viewer (user) from the state of Fig. 2, instead of the selected viewpoint image (selected viewpoint image) being displayed large in the center of the screen, the viewpoint image 40E is displayed small side by side with the other viewpoint images at the top of the screen.

[0026] In this way, a display mode in which any one of a plurality of viewpoint images is selected and displayed largely on the screen of the viewing terminal 30 is called a selection mode.

[0027] On the viewing terminal 30 (own terminal) on which the viewpoint image captured in a certain event is displayed, avatars 50 corresponding to each of the user (hereinafter also referred to as the own user) who is viewing the viewpoint image and the user (hereinafter also referred to as the other user) who is viewing the viewpoint image on another viewing terminal 30 (other terminal) are displayed.

[0028] In the example of FIG. 2, three avatars 50 are displayed at the lower end of the viewpoint image 40E displayed in the selection mode. Basically, the avatar 50 corresponding to the own user (hereinafter also referred to as the own avatar) is always displayed at the center in the left - right direction, and the avatars 50 corresponding to the other users (hereinafter also referred to as the other avatars) are displayed at predetermined positions on the left and right of the own avatar.

[0029] Also, in the lower part of the screen in the selection mode, an evaluation area 60 for inputting an evaluation of the viewpoint image displayed in the selection mode is provided. Specifically, in the evaluation area 60, a Good button 61 for indicating a high - evaluation intention for the viewpoint image displayed in the selection mode and a comment input box 62 for text - inputting a comment on the viewpoint image are provided. Although details will be described later, the comment input in the comment input box 62 is displayed in a balloon shape above the avatar 50 corresponding to the user who input it.

[0030] In this way, by the Good button 61 and the comment input box 62, users who are viewing the same viewpoint image can communicate with each other by evaluating the viewpoint image.

[0031] FIG. 3 is a diagram showing a configuration example of the avatar 50.

[0032] As shown in FIG. 3, the avatar 50 is composed of a body 71 which is a human - shaped image and an icon 72 synthesized with a part corresponding to the head of the body 71.

[0033] The body 71 is controlled in its operation according to the operation of the user corresponding to the avatar 50 on the viewing-side terminal 30. For example, according to the user's operation, a body 71 that walks or a body 71 that high-touches the avatar 50 of another user is animated and displayed. The body 71 may be color-coded so as to distinguish the self-avatar from other avatars, or may be designed according to the event in which the viewpoint image is captured.

[0034] The icon 72 is, for example, a face image capable of identifying the user corresponding to the avatar 50. The face image applied to the icon 72 is a still image, but may be a moving image, for example, when enabling a video call between users who are viewing the same viewpoint image. Further, the icon 72 is not limited to a face image, and may be an image unique to the user corresponding to the avatar 50.

[0035] In this way, through the display and operation of the avatar 50, users who are viewing the same viewpoint image can communicate more intuitively.

[0036] In the following, a specific configuration for realizing the distribution of a plurality of viewpoint images in the image distribution system 1 and the display control of avatars corresponding to the distributed plurality of viewpoint images will be described.

[0037] <2. Functional Configuration Examples of Each Device> (Functional Configuration Example of Server 20) FIG. 4 is a block diagram showing a functional configuration example of the server 20.

[0038] The server 20 is configured as a cloud server that provides a cloud computing service including a web service.

[0039] The server 20 is configured to include a photographing-side communication processing unit 111, an image processing unit 112, a distribution control unit 113, a viewing-side communication processing unit 114, and an event-avatar management unit 115.

[0040] The shooting-side communication processing unit 111 transmits and receives image data and other information to and from the shooting-side terminal 10 via the Internet. Specifically, the shooting-side communication processing unit 111 receives the perspective images from each of the shooting-side terminals 10, or transmits the combined image of the perspective images to each of the shooting-side terminals 10. Further, the shooting-side communication processing unit 111 receives the position information of each of the shooting-side terminals 10, or transmits the avatar information related to the display of the avatar corresponding to the user of the viewing-side terminal 30 to each of the shooting-side terminals 10.

[0041] The image processing unit 112 decodes the perspective images from each of the shooting-side terminals 10 received by the shooting-side communication processing unit 111, combines them into one image, and supplies it to the distribution control unit 113. Further, the image processing unit 112 supplies the unique information for identifying the perspective images from each of the shooting-side terminals 10 to the event-avatar management unit 115.

[0042] The distribution control unit 113 encodes the image combined by the image processing unit 112, and controls the viewing-side communication processing unit 114 to distribute it to each of the viewing-side terminals 30 as a multi-stream. The multi-stream after encoding is also distributed to each of the shooting-side terminals 10.

[0043] The viewing-side communication processing unit 114 transmits and receives image data and other information to and from the viewing-side terminal 30 via the Internet. Specifically, the viewing-side communication processing unit 114 transmits the combined image of the perspective images to each of the viewing-side terminals 30. Further, the viewing-side communication processing unit 114 receives the position information of each of the viewing-side terminals 30 and the information related to each user, or transmits the above-mentioned avatar information to each of the viewing-side terminals 30.

[0044] The event-avatar management unit 115 manages the events in which the perspective images from each of the shooting-side terminals 10 are taken, and the avatars corresponding to the users who are viewing those perspective images in each of the viewing-side terminals 30.

[0045] The event avatar management unit 115 includes an event setting unit 121, an avatar ID generation unit 122, an icon setting unit 123, an angle ID setting unit 124, a group ID setting unit 125, a coordinate information setting unit 126, a comment adding unit 127, and an action waiting state determination unit 128.

[0046] The event setting unit 121 sets an event that can be participated in by the user of the viewing-side terminal 30 by associating a plurality of viewpoint images captured in the event 1 with a specific URL. The user of the viewing-side terminal 30 can view the viewpoint images of the corresponding event by accessing that URL with the viewing-side terminal 30. Note that the viewpoint images from the shooting-side terminal 10 are also uploaded to the server 20 when the shooting-side terminal 10 accesses a specific URL.

[0047] The avatar ID generation unit 122 generates an avatar ID unique to the avatar corresponding to the user for the user of the viewing-side terminal 30 who has accessed the URL corresponding to the event.

[0048] The icon setting unit 123 sets an icon for the avatar specified by the generated avatar ID. The face image applied to the icon may be acquired from the viewing-side terminal 30 that has accessed the above-mentioned URL, or may be held in the server 20 in advance.

[0049] The angle ID setting unit 124 generates an angle ID that specifies the angle of the viewpoint image based on the unique information for specifying the viewpoint image from the image processing unit 112. The angle ID setting unit 124 sets the angle ID of the viewpoint image for the avatar ID corresponding to the user who is viewing the viewpoint image.

[0050] The group ID setting unit 125 sets a group ID that identifies the group created within the event for the avatar ID corresponding to the user participating in one event. The group is created by a predetermined user participating in one event, and the group ID of the group is set for the avatar ID corresponding to the user who created the group. Also, when the user who created the group invites other users to the group and the invited users participate in the group, the group ID of the group is set for the avatar ID corresponding to the invited users.

[0051] Here, with reference to FIG. 5, a configuration example of event data generated for one event will be described.

[0052] In the event data of FIG. 5, avatar IDs 01 to 12 corresponding to 12 users are generated. Also, angles A, B, and C are shown as angle IDs corresponding to three perspective images.

[0053] In the example of FIG. 5, among the 12 users, since the users with avatars 01 to 05 are viewing the perspective image specified by angle A, angle A is set for avatars 01 to 05. Since the users with avatars 06 to 09 are viewing the perspective image specified by angle B, angle B is set for avatars 06 to 09. Since the users with avatars 10 to 12 are viewing the perspective image specified by angle C, angle C is set for avatars 10 to 12.

[0054] Also, among the avatar IDs for which angle A is set, group ID g1 is set as the group ID for avatars 01 to 03. That is, the users with avatars 01 to 03 belong to the same group specified by group g1.

[0055] Similarly, among the avatar IDs with Angle B set, for avatars 06 and 07, Group ID g2 is set as the group ID. That is, the users of avatars 06 and 07 belong to the same group specified by Group g2.

[0056] Note that for avatars 08 and 09 with Angle B set and avatar 12 with Angle C set, across the Angle IDs, Group ID g3 is set as the group ID.

[0057] In this way, users who are viewing the perspective images of the same angle can belong to one group. On the other hand, since the user can switch the perspective image (angle) to be viewed, it is not always the case that the same Angle ID is set for avatar IDs belonging to the same group. Also, one user can belong to multiple groups.

[0058] Now, returning to the description of FIG. 4, the coordinate information setting unit 126 sets coordinate information representing the display position of the avatar corresponding to each user displayed together with the perspective image on the viewing-side terminal 30. The coordinate information indicates the absolute position on the two-dimensional plane corresponding to the screen of the viewing-side terminal 30 and is associated with all avatar IDs included in one event data.

[0059] The comment adding unit 127 adds the comment input on the viewing-side terminal 30 to the avatar corresponding to the user of that viewing-side terminal 30.

[0060] The action waiting state determination unit 128 determines the action waiting state of the avatar based on the waiting state information indicating that the avatar corresponding to the user of the viewing-side terminal 30 is in a waiting state for a specific action (action), and generates information representing the determination result.

[0061] As described above, the information generated and set by each part of the event avatar management unit 115 is transmitted to each viewing-side terminal 30 as avatar information regarding the display of the avatar corresponding to each user.

[0062] (Functional configuration example of the shooting-side terminal 10) FIG. 6 is a block diagram showing a functional configuration example of the shooting-side terminal 10.

[0063] The shooting-side terminal 10 is not limited to a general camera, and a smartphone or a tablet terminal having a shooting function can also be used as a configuration.

[0064] The shooting-side terminal 10 is configured to include a shooting unit 131, a communication processing unit 132, a display unit 133, and a control unit 134.

[0065] The shooting unit 131 is composed of an optical system including a lens and an image sensor, and shoots a moving image that becomes a viewpoint image.

[0066] The communication processing unit 132 transmits and receives image data and other information to and from the server 20 via the Internet. Specifically, the communication processing unit 132 transmits the viewpoint image shot by the shooting unit 131 to the server 20, or receives an image in which a plurality of viewpoint images from the server 20 are combined. In addition, the communication processing unit 132 transmits the position information of the shooting-side terminal 10 to the server 20, or receives avatar information from the server 20 and feedback information representing feedback from viewers including the above-described comments. The feedback information also includes tap information for displaying a tap image described later.

[0067] The display unit 133 is composed of a liquid crystal display, an organic EL (Electro-Luminescence) display, etc., and displays a screen including an image from the server 20.

[0068] The control unit 134 is composed of a processor such as a CPU (Central Processing Unit) and controls each part of the imaging terminal 10. The control unit 134 realizes the imaging control unit 141, the display control unit 142, and the viewer information acquisition unit 143 by operating a predetermined application executed on a browser or installed in the imaging terminal 10.

[0069] The imaging control unit 141 controls the imaging unit 131 to acquire a viewpoint image, and causes the acquired viewpoint image to be transmitted to the server 20 via the communication processing unit 132.

[0070] The display control unit 142 controls the display unit 133 to display on the display unit 133 a screen including the image received by the communication processing unit 132.

[0071] The viewer information acquisition unit 143 acquires avatar information and feedback information received by the communication processing unit 132. The display control unit 142 controls the display of the avatar based on the avatar information acquired by the viewer information acquisition unit 143.

[0072] (Functional configuration example of the viewing terminal 30) FIG. 7 is a block diagram showing a functional configuration example of the viewing terminal 30.

[0073] The viewing terminal 30 is configured as a smartphone, a tablet terminal, a PC (Personal Computer), or the like.

[0074] The viewing terminal 30 is configured to include a communication processing unit 151, a display unit 152, and a control unit 153.

[0075] The communication processing unit 151 transmits and receives image data and other information to and from the server 20 via the Internet. Specifically, the communication processing unit 151 receives an image in which a plurality of viewpoint images from the server 20 are combined. Also, the communication processing unit 151 transmits the position information of the viewing terminal 30 and information related to its user to the server 20 or receives avatar information from the server 20.

[0076] The display unit 152 is composed of a liquid crystal display, an organic EL display, etc., and displays a screen including an image from the server 20.

[0077] The control unit 153 is composed of a processor such as a CPU, and controls each part of the viewing-side terminal 30. The control unit 153 realizes the display control unit 161, the avatar information acquisition unit 162, and the operation detection unit 163 by being executed on a browser or operating a predetermined application installed in the viewing-side terminal 30.

[0078] The display control unit 161 controls the display unit 152 to display on the display unit 152 a screen including the image received by the communication processing unit 151.

[0079] The avatar information acquisition unit 162 acquires the avatar information received by the communication processing unit 151. The display control unit 161 controls the display of the avatar based on the avatar information acquired by the avatar information acquisition unit 162.

[0080] The operation detection unit 163 detects a user's operation on the viewing-side terminal 30. The detection of the user's operation includes not only the detection of operations on buttons, keys (not shown), and a touch panel superimposed on the display unit 152, but also the detection of the inclination of the viewing-side terminal 30 by an acceleration sensor.

[0081] <3. Display form of the viewpoint image> Hereinafter, an example of the display form of the viewpoint image on the viewing-side terminal 30 will be described.

[0082] (Flow of image distribution) First, with reference to FIG. 8, the flow of image distribution in the image distribution system 1 will be described.

[0083] In step S11, the photographing-side terminal 10 accesses a URL for uploading a viewpoint image. As a result, in step S12, a connection is established between the photographing-side terminal 10 and the server 20. Note that an upper limit on the number of accessible photographing-side terminals 10 may be set for this URL.

[0084] In step S13, the photographing-side terminal 10 (photographing control unit 141) starts photographing a viewpoint image, and in step S14, the photographing-side terminal 10 (communication processing unit 132) starts transmitting the photographed viewpoint image to the server 20.

[0085] In step S15, the server 20 (image processing unit 112) combines a plurality of viewpoint images uploaded by a plurality of photographing-side terminals 10 into one image.

[0086] On the other hand, in step S16, the viewing-side terminal 30 accesses a URL for viewing a viewpoint image (in other words, for participating in an event). As a result, in step S17, a connection is established between the viewing-side terminal 30 and the server 20. No upper limit on the number of accessible viewing-side terminals 30 is set for this URL.

[0087] When the connection between the viewing-side terminal 30 and the server 20 is established, in step S18, the server 20 (distribution control unit 113) starts distributing the image obtained by combining a plurality of viewpoint images.

[0088] That is, the server 20 (viewing-side communication processing unit 114) transmits a plurality of viewpoint images in step S19 and transmits avatar information corresponding to each user of the viewing-side terminal 30 in step S20.

[0089] In step S21, the viewing-side terminal 30 (avatar information acquisition unit 162) acquires avatar information from the server 20.

[0090] Then, in step S22, the viewing-side terminal 30 (display control unit 161) controls the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on its own terminal (display unit 152).

[0091] (Display of viewpoint images in panorama mode) FIG. 9 is a diagram showing an example of the display of viewpoint images in panorama mode in which a plurality of viewpoint images are arranged side by side on the viewing-side terminal 30.

[0092] The panorama mode screen 200 shown in FIG. 9 is displayed, for example, when the viewing-side terminal 30 first accesses the URL for participating in a certain event. In the panorama mode, the user of the viewing-side terminal 30 views the viewpoint images of the panorama mode screen 200 with the rectangular display unit 152 horizontal (the longitudinal direction being horizontal).

[0093] In the panorama mode screen 200, the plurality of viewpoint images are arranged side by side in order based on the position information of the shooting-side terminal 10 where each was shot. In the example of FIG. 9, three viewpoint images 40D, 40E, and 40F are displayed, but the display transitions to viewpoint images 40A, 40B, and 40C (not shown) by an operation such as a left-right swipe by the user.

[0094] In the panorama mode screen 200, the display area is divided corresponding to each viewpoint image. In the example of FIG. 9, display areas 210D, 210E, and 210F corresponding to the viewpoint images 40D, 40E, and 40F are shown.

[0095] Each of the display areas 210D, 210E, and 210F (hereinafter also simply referred to as display area 210) is composed of a background area 211 that is the background of the viewpoint image and an evaluation area 212 for inputting an evaluation of the viewpoint image.

[0096] The background area 211 is the background image of the viewpoint image in the panorama mode screen 200 and has different colors and designs for each viewpoint image (angle).

[0097] The evaluation area 212 corresponds to the evaluation area 60 described with reference to FIG. 2, and a Good button and a comment input box are provided. The evaluation area 212 also has different colors and designs for each viewpoint image (angle).

[0098] In each of the display areas 210, other avatars 50s corresponding to other users are displayed at the lower end of the viewpoint image. The other avatars 50s are arranged within the display area 210 corresponding to the viewpoint image being viewed by the corresponding other user. Therefore, for example, the four other users corresponding to the four other avatars 50s displayed in the display area 210E are viewing the viewpoint image 40E.

[0099] Also, in each of the display areas 210, comments 220 text - input by the other users corresponding to the other avatars 50s are displayed in a balloon - like manner above the corresponding other avatars 50s.

[0100] Here, with reference to FIG. 10, the arrangement of comments corresponding to the avatars in the panorama mode screen 200 will be described. Here, the arrangement of the other avatars 50s and the comments 220 in FIG. 9 will be taken as an example for explanation.

[0101] In the figure, the horizontal axis represents the position P of the avatar (other avatars 50s) in the panorama mode screen 200, and the vertical axis represents the time T when the comment 220 was text - input by the user corresponding to the avatar.

[0102] As described above, the position P of the avatar is the position within the display area 210 corresponding to the viewpoint image being viewed by each user. Also, in the panorama mode screen 200, the viewpoint images are arranged side - by - side horizontally in the order based on the positions where they are taken. Therefore, it can be said that the position P of each avatar in the panorama mode screen 200 represents the virtual position of each user participating in one event.

[0103] Since comment 220 is displayed above the avatar corresponding to the user who entered the text, the horizontal position of comment 220 is the same as the position P of the corresponding avatar. Also, the vertical position of comment 220 is higher the earlier the time T at which the text was entered, and lower the later.

[0104] Therefore, comment 220 immediately after the text is entered is located directly above the corresponding avatar and moves upward as time passes.

[0105] In this way, on the panorama mode screen 200, avatars are arranged in the first direction (horizontal axis direction), and comments entered in text by the users corresponding to those avatars are arranged in the second direction (vertical axis direction) orthogonal to the first direction according to the order of input.

[0106] Note that such an arrangement of avatars and comments is not limited to the panorama mode screen 200, and can also be applied to the selection mode screen described later.

[0107] Now, in the example of FIG. 9, on the panorama mode screen 200, other avatars 50s corresponding to other users who are viewing two or more viewpoint images are displayed as avatars. And the user (self-user) of the viewing-side terminal 30 on which the panorama mode screen 200 is displayed can view the viewpoint image in the selection mode described with reference to FIG. 2 by selecting any one of the viewpoint images.

[0108] On the other hand, at the time when the panorama mode screen 200 is displayed, since the self-user has not selected any viewpoint image, it is assumed that the self-avatar corresponding to the self-user is not displayed.

[0109] However, on the panorama mode screen 200, in addition to the other avatars 50s, the self-avatar may be displayed as an avatar.

[0110] For example, as shown in FIG. 11, in the panoramic mode screen 200, the self-avatar 50m is displayed in a region different from the region where other avatars 50s are displayed (in the example of FIG. 11, the lower left of the panoramic mode screen 200). Thereby, it is possible to give the self-user a feeling of participating in the event in which each viewpoint image displayed on the panoramic mode screen 200 is being taken.

[0111] In the above, in the panoramic mode screen 200, a plurality of viewpoint images are arranged side by side horizontally, but it is not limited to this, and a plurality of viewpoint images may be arranged vertically. In this case, the avatar is arranged in the second direction (vertical axis direction), and the comments text-input by the user corresponding to the avatar is arranged in the first direction (horizontal axis direction) according to the input order. Also, similar to the case where a plurality of viewpoint images are arranged side by side horizontally, the avatar may be arranged in the first direction (horizontal axis direction) and the comments may be arranged in the second direction (vertical axis direction).

[0112] Also, in the panoramic mode screen 200, in addition to a plurality of viewpoint images being arranged side by side horizontally or vertically, they may be arranged to correspond to the actual positional relationship of the shooting-side terminal 10, or may be displayed in other regular arrangements such as an arc-shaped arrangement or a zigzag arrangement.

[0113] (Display of viewpoint image in selection mode) FIG. 12 is a diagram showing an example of the display of a viewpoint image in a selection mode in which any one of the viewpoint images is selected on the panoramic mode screen.

[0114] FIG. 12 shows three selection mode screens 300D, 300E, and 300F. The selection mode screen 300D is displayed when the viewpoint image 40D is selected on the panoramic mode screen 200 of FIG. 12. The selection mode screen 300E is displayed when the viewpoint image 40E is selected on the panoramic mode screen 200 of FIG. 12. The selection mode screen 300F is displayed when the viewpoint image 40F is selected on the panoramic mode screen 200 of FIG. 12.

[0115] In each of the selection mode screens 300D, 300E, and 300F (hereinafter, also simply referred to as the selection mode screen 300), the selected viewpoint image (selected viewpoint image) is largely displayed in the center, and the other viewpoint images are displayed small and arranged at the top. Further, an evaluation area 60 for inputting an evaluation of the viewpoint image is provided at the lower part of the selection mode screen 300. Similar to the evaluation area 212 in the panorama mode screen 200 of FIG. 12, the evaluation area 60 has different colors and designs for each viewpoint image (angle).

[0116] Furthermore, in the selection mode screen 300, at the lower end of the viewpoint image, as an avatar corresponding to the user who is viewing the selected viewpoint image, the self-avatar 50m is displayed at the center in the left-right direction, and the other avatars 50s are displayed at predetermined positions on the left and right of the self-avatar 50m.

[0117] In the selection mode screen 300, a background image 311 is displayed on the background layer of the self-avatar 50m and the other avatars 50s. Similar to the background area 211 in the panorama mode screen 200 of FIG. 12, the background image 311 has different colors and designs for each viewpoint image (angle).

[0118] As described with reference to FIG. 2, in the selection mode screen 300, when any one of the other viewpoint images displayed at the top is selected by the user, the viewpoint image largely displayed in the center is switched. At this time, the colors and designs of the background image 311 and the evaluation area 60 are also switched according to the selected viewpoint image (angle).

[0119] Note that the colors and designs of the background image 311 and the evaluation area 60 (background area 211 and evaluation area 212) are different for each viewpoint image. However, for the entire event, it is preferable to adopt colors and designs with similar color tones and atmospheres corresponding to the event. Also, when there are multiple events, it is preferable to adopt colors and designs with different color tones and atmospheres for each event. Of course, in one event, the colors and designs may be unified regardless of the viewpoint image.

[0120] (Flow from event participation to viewing of viewpoint images) FIG. 13 is a diagram for explaining the flow from the participation in the above-described event to the viewing of viewpoint images.

[0121] In step S31, the viewing-side terminal 30 (communication processing unit 151) sends an event participation request to the server 20 by accessing the URL for participating in the event. A face image applied to the icon of the self-avatar may be sent together with the event participation request.

[0122] In step S32, the server 20 (avatar ID generation unit 122) generates an avatar ID corresponding to the viewing-side terminal 30 that has sent the event participation request.

[0123] In step S33, the server 20 (distribution control unit 113) distributes an image obtained by combining a plurality of viewpoint images to the viewing-side terminal 30. As a result, on the viewing-side terminal 30, a panorama mode screen as described with reference to FIG. 9 is displayed.

[0124] In step S34, when any viewpoint image (angle) displayed on the panorama mode screen of the viewing-side terminal 30 is selected, in step S35, the viewing-side terminal 30 (communication processing unit 151) sends angle selection information indicating the selected angle to the server 20.

[0125] In step S36, the server 20 (angle ID setting unit 124) associates (sets) the angle ID corresponding to the angle selection information from the viewing terminal 30 with the avatar ID generated in step S32.

[0126] In step S37, the server 20 (viewing-side communication processing unit 114) transmits avatar information for displaying the self-avatar with the angle ID set and other avatars with the same angle ID set to the viewing terminal 30.

[0127] In step S38, the viewing terminal 30 (display control unit 161) controls the display of the selected angle (viewpoint image) and avatars (self-avatar and other avatars). As a result, on the viewing terminal 30, a selection mode screen as described with reference to FIG. 12 is displayed.

[0128] In step S39, when another angle is selected on the selection mode screen, the processing after step S35 is repeated, and the selected other angle (viewpoint image) and avatars (self-avatar and other avatars) are displayed. On the other hand, in step S39, when another angle is not selected on the selection mode screen, the display of the angle (viewpoint image) and avatars (self-avatar and other avatars) that have been displayed so far continues.

[0129] According to the above processing, the user can view the entire plurality of viewpoint images captured in the event, select and view a desired viewpoint image from among them, or switch to other viewpoint images. And corresponding to the displayed viewpoint image, the display of the avatar also changes. Thereby, a more realistic live distribution of the event can be realized, and the user can participate in and enjoy the event from various viewpoints as in the real world.

[0130] <4. Animation display of avatar> On the viewing terminal 30, the avatar can be displayed as an animation.

[0131] (Avatar walking) For example, on the selection mode screen, the avatar can be made to walk.

[0132] FIG. 14 is a diagram for explaining the flow of avatar walking.

[0133] In step S51, the viewing terminal 30 (operation detection unit 163) determines whether the inclination of the viewing terminal 30 has been detected. Here, the inclination is the inclination in the rotational direction with the normal direction of the screen of the viewing terminal 30 as the axis. This inclination is detected by the output of the acceleration sensor provided in the viewing terminal 30.

[0134] Step S51 is repeated until the inclination is detected, and when the inclination is detected, the process proceeds to step S52.

[0135] In step S52, the viewing terminal 30 (display control unit 161) causes the display unit 152 to perform an animation display in which the avatar (self-avatar) walks.

[0136] For example, as shown in FIG. 15, when the selection mode screen 300 displayed on the viewing terminal 30 is tilted in the counterclockwise rotation direction, the self-avatar 50m moves to the downward slope side (left side in the figure) (the display position is changed) and is animated to walk. Note that the self-avatar 50m does not move to the edge of the selection mode screen 300 in accordance with the inclination of the selection mode screen 300, but after moving a certain extent, it is displayed so as to be positioned again at the center in the left-right direction of the selection mode screen 300.

[0137] In step S53, the viewing terminal 30 (display control unit 161) scrolls and displays the background image 311 in response to the movement of the self-avatar. For example, in the example of FIG. 15, the background image 311 scrolls in the direction opposite to the movement direction (left direction) of the self-avatar 50m (right direction).

[0138] As shown in FIG. 16, the background image 311 may be composed of a plurality of layers L1, L2, and L3. In the example of FIG. 16, layer L1 constitutes the layer closest to the self-avatar 50m, and layer L3 constitutes the layer farthest from the self-avatar 50m. In the background image 311 having such a structure, when the background image 311 scrolls in the direction opposite to the moving direction of the self-avatar 50m, the layer L1 closer to the self-avatar 50m is scrolled faster, and the layer L3 farther from the self-avatar 50m is scrolled more slowly. Thereby, when the self-avatar 50m moves, a background with a sense of depth can be expressed.

[0139] Returning to the description of FIG. 14, in step S54, the viewing-side terminal 30 (communication processing unit 151) transmits movement information representing the movement amount of the self-avatar according to the inclination amount of the viewing-side terminal 30 to the server 20. Note that the movement amount of the self-avatar represented by the movement information is not the movement amount of the self-avatar on the selection mode screen 300, but the absolute movement amount based on the inclination amount of the viewing-side terminal 30.

[0140] In step S55, the server 20 (coordinate information setting unit 126) updates the coordinate information of the avatar corresponding to the viewing-side terminal 30 based on the movement information from the viewing-side terminal 30. The coordinate information of the avatar indicates the position on the two-dimensional coordinates set for each viewpoint image (angle). Thereby, among the avatars corresponding to the users viewing the same viewpoint image, the relative positions of the other avatars with respect to the position of the self-avatar change.

[0141] In step S56, the server 20 (viewing-side communication processing unit 114) transmits avatar information about all avatars including the updated coordinate information to the viewing-side terminal 30. This avatar information is transmitted not only to the viewing-side terminal 30 in which the inclination is detected, but also to the viewing-side terminals 30 of all users viewing the same viewpoint image.

[0142] Note that all the avatars mentioned here may be all the avatars that can be displayed on at least the viewing terminal 30, and the same applies hereinafter. Similarly, all the users may be all the users corresponding to all the avatars that can be displayed, and the same applies hereinafter.

[0143] The avatars that can be display targets include not only the self-avatar but also other avatars associated with the self-avatar (other avatars in the same group as the self-avatar). Also, when there is room for the number of avatars that can be display targets, other avatars not associated with the self-avatar may be included in the avatars to be displayed. Thereby, even if the number of users participating in one event is huge, appropriate numbers of avatar information can be transmitted and received.

[0144] In step S57, the viewing terminal 30 (display control unit 161 and avatar information acquisition unit 162) acquires avatar information from the server 20 and updates the display (arrangement) of other avatars based on the self-avatar on the selection mode screen 300.

[0145] FIG. 17 is a diagram showing an example of the coordinate information of avatars in a certain viewpoint image (angle).

[0146] In FIG. 17, the coordinate information of five avatars is shown on the xy coordinates. The x-axis corresponds to the horizontal position on the selection mode screen 300, and the y-axis corresponds to the vertical position on the selection mode screen 300.

[0147] As shown in FIG. 17, on the selection mode screen 300, the avatars can move not only in the horizontal direction but also in the vertical direction by, for example, animation display such as jumping.

[0148] On the viewing terminal 30 of the user corresponding to each of the avatars shown in FIG. 17, other avatars are arranged based on the self-avatar while maintaining the positional relationship between the avatars shown in FIG. 17.

[0149] (Avatar action display 1) On the selection mode screen, avatars can be made to perform actions according to the distance between the avatars.

[0150] FIG. 18 is a diagram for explaining the flow of avatar action display.

[0151] Here, as described with reference to FIG. 14, it is assumed that the self-avatar moves according to the inclination of the viewing-side terminal 30, but other avatars may also move.

[0152] In step S71, the server 20 (viewing-side communication processing unit 114) transmits avatar information about all avatars including the updated coordinate information to the viewing-side terminal 30.

[0153] In step S72, the viewing-side terminal 30 (avatar information acquisition unit 162) determines whether the distance between the self-avatar and other avatars (distance between avatars) has become closer than a predetermined distance based on the coordinate information included in the avatar information from the server 20.

[0154] Step S72 is repeated until the distance between the avatars becomes closer than the predetermined distance, and when the distance between the avatars becomes closer than the predetermined distance, the process proceeds to step S73.

[0155] In step S73, the viewing-side terminal 30 (display control unit 161) causes specific action displays to be performed between the avatars whose distance between them has become closer than the predetermined distance.

[0156] Specifically, as shown in FIG. 19, on the selection mode screen 300, when the distance between the self-avatar 50m and the other avatar 50s becomes closer than the predetermined distance due to the movement of the self-avatar 50m, the self-avatar 50m and the other avatar 50s are animatedly displayed as if they have collided with each other. Further, after the animated display as if they have collided with each other, the self-avatar 50m and the other avatar 50s may be animatedly displayed as if they are bounced off each other.

[0157] (Avatar action display 2) In the selection mode screen, avatars can be made to perform actions according to the distance between avatars that are waiting to take a specific action.

[0158] Figure 20 is a diagram for explaining the flow of avatar action display.

[0159] In step S91, the viewing-side terminal 30 (operation detection unit 163) determines whether or not the self-user has instructed the self-avatar to take a specific action (action waiting state).

[0160] Step S91 is repeated until the action waiting state is instructed, and when the action waiting state is instructed, the process proceeds to step S92.

[0161] In step S92, the viewing-side terminal 30 (communication processing unit 151) transmits waiting state information indicating that the self-avatar is in the action waiting state to the server 20.

[0162] In step S93, the server 20 (action waiting state determination unit 128 and viewing-side communication processing unit 114) determines the action waiting state of the avatar based on the waiting state information from the viewing-side terminal 30, and transmits the determination result and avatar information about all avatars including coordinate information to the viewing-side terminal 30.

[0163] Here too, as described with reference to FIG. 14, it is assumed that the self-avatar moves according to the inclination of the viewing-side terminal 30, but other avatars may also move.

[0164] In step S94, the viewing-side terminal 30 (viewer information acquisition unit 143) determines whether or not the distance between the self-avatar and other avatars (distance between avatars) has become closer than a predetermined distance based on the coordinate information included in the avatar information from the server 20.

[0165] Step S94 is repeated until the distance between the avatars becomes closer than a predetermined distance, and when the distance between the avatars becomes closer than the predetermined distance, the process proceeds to Step S95.

[0166] In Step S95, the viewing terminal 30 (avatar information acquisition unit 162) determines, based on the standby state information included in the avatar information from the server 20, whether or not the other avatars that have come closer than the predetermined distance have also entered the action standby state.

[0167] Steps S94 and S95 are repeated until the other avatars that have come closer than the predetermined distance also enter the action standby state, and when the other avatars that have come closer than the predetermined distance also enter the action standby state, the process proceeds to Step S96.

[0168] In Step S96, the viewing terminal 30 (display control unit 161) causes specific action displays to be performed between the avatars whose distance from each other is closer than the predetermined distance and who are in the action standby state with respect to each other.

[0169] Specifically, as shown in FIG. 21, on the selection mode screen 300, when the self-avatar 50m in the action standby state moves, the distance from the other avatar 50s becomes closer than the predetermined distance, and the other avatar 50s is also in the action standby state, the self-avatar 50m and the other avatar 50s are animated to high-five each other.

[0170] As described above, on the viewing terminal 30, the avatars can be animated in various ways.

[0171] <5. Priority Display of Group and Avatar> As described above, the user can create a group within an event.

[0172] FIG. 22 is a diagram for explaining the flow of creating and participating in a group in an event.

[0173] In step S111, the viewing-side terminal 30-1 (communication processing unit 151) sends an event participation request to the server 20 by accessing the URL for participating in the event.

[0174] In step S112, the server 20 (avatar ID generation unit 122) generates an avatar ID corresponding to the viewing-side terminal 30-1 that has sent the event participation request.

[0175] In step S113, the viewing-side terminal 30-1 (communication processing unit 151) sends a group creation request to the server 20 in response to an operation for creating a group in the participated event. The operation for creating a group is, for example, pressing a group creation button displayed on the screen of the viewing-side terminal 30-1. At this time, the name of the group or the like may be input.

[0176] In step S114, the server 20 (group ID setting unit 125) generates a group ID in response to the group creation request from the viewing-side terminal 30-1.

[0177] In step S115, the server 20 (group ID setting unit 125) associates (sets) the group ID generated in step S114 with the avatar ID generated in step S112.

[0178] After that, in order for the user of the viewing-side terminal 30-1 to invite the user of the viewing-side terminal 30-2 to the group, the URLs for participating in the event and the group are sent to the viewing-side terminal 30-2 using an email or a predetermined messaging function.

[0179] In step S116, the viewing-side terminal 30-2 (communication processing unit 151) that has received the URL from the viewing-side terminal 30-1 sends an event participation request to the server 20 by accessing the URL.

[0180] In step S117, the server 20 (avatar ID generation unit 122) generates an avatar ID corresponding to the viewing terminal 30-2 that has sent an event participation request.

[0181] In step S118, the viewing terminal 30-2 (communication processing unit 151) sends a group participation request to the server 20 in response to an operation to participate in the group invited to the user of the viewing terminal 30-1. The operation to participate in the group is, for example, pressing a group participation button displayed on the screen of the viewing terminal 30-2.

[0182] In step S119, the server 20 (group ID setting unit 125) associates (sets) the group ID generated in step S114 with the avatar ID generated in step S117.

[0183] As described above, the user can create a group within the event and further invite other users to the group.

[0184] By the way, in the selection mode screen 300 described above, an avatar corresponding to the user viewing the displayed viewpoint image is displayed. However, when the number of users viewing the viewpoint image is extremely large, the display of the avatars becomes complicated.

[0185] Therefore, as shown in FIG. 23, when a large number of other avatars 50s are displayed more than 50m away from the self-avatar, other avatars 50s corresponding to the users who have joined the same group as the self-user are preferentially displayed.

[0186] Referring to FIG. 24, the flow of the priority display of the avatars will be described.

[0187] In step S131, the server 20 (the viewing-side communication processing unit 114) transmits avatar information about all avatars that can be displayed on the viewing-side terminal 30 to the viewing-side terminal 30. The avatar information is transmitted each time a new avatar ID is generated, or when the angle ID, group ID, or coordinate information associated with the avatar ID is updated.

[0188] In step S132, the viewing-side terminal 30 (the avatar information acquisition unit 162) acquires the avatar information from the server 20 and determines whether the number of other avatars with the same angle ID as the angle ID associated with its own avatar exceeds a predetermined number.

[0189] If the number of other avatars with the same angle ID exceeds the predetermined number, the process proceeds to step S133.

[0190] In step S133, the viewing-side terminal 30 (the display control unit 161) preferentially displays its own avatar and other avatars in the same group among the other avatars with the same angle ID.

[0191] At this time, if the total number of its own avatar and other avatars in the same group is less than the predetermined number, in addition to its own avatar and other avatars in the same group, randomly selected other avatars may be displayed within a range not exceeding the predetermined number.

[0192] Also, if the total number of its own avatar and other avatars in the same group is less than the predetermined number, in addition to its own avatar and other avatars in the same group, other avatars selected according to the viewing history of the perspective images of other users may be displayed within a range not exceeding the predetermined number.

[0193] Furthermore, if the total number of its own avatar and other avatars in the same group is less than the predetermined number, in addition to its own avatar and other avatars in the same group, other avatars selected by the user itself may be displayed within a range not exceeding the predetermined number.

[0194] On the other hand, if the number of other avatars with the same angle ID does not exceed a predetermined number, the process proceeds to step S134, and the viewing-side terminal 30 (display control unit 161) displays its own avatar and all other avatars with the same angle ID.

[0195] As described above, when the number of avatars displayed on the viewing-side terminal 30 exceeds a predetermined number, among the other avatars, the display of the other avatars associated with the self-avatar corresponding to the user (self-user) of the viewing-side terminal 30 is prioritized. This can prevent the display of avatars from becoming complicated. Note that the priority display of avatars as described above is not limited to the selection mode screen 300, but can also be applied to the panorama mode screen 200.

[0196] <6. Other Display Variations> In the following, display variations on the viewing-side terminal 30 and the shooting-side terminal 10 will be exemplified.

[0197] (Display of Avatars Linked to Events) On the screen where the viewpoint image is displayed, goods related to the event in which the viewpoint image is taken may be made purchasable by EC (Electronic Commerce).

[0198] For example, as shown on the left side of FIG. 25, on the selection mode screen 300, a purchase screen 410 where related goods of the soccer team playing a soccer game can be purchased may be displayed. In the example of FIG. 25, the soccer team uniform 411 is displayed on the purchase screen 410.

[0199] When the user decides to purchase the uniform 411 on the purchase screen 410, as shown on the right side of FIG. 25, an image 411' corresponding to the uniform 411 is synthesized with the self-avatar 50m.

[0200] In this way, by reflecting the display information related to the event on the avatar, the user can be made to feel the sense of presence of participating in the event.

[0201] (Avatar display switching) In the above, an example has been described in which when the number of displayed avatars exceeds a predetermined number, the display of other avatars associated with the self-avatar is prioritized.

[0202] In addition to this, the avatar to be displayed may be switched according to the user's operation.

[0203] FIG. 26 is a diagram showing an example of avatar display switching.

[0204] In FIG. 26, examples of three patterns of avatar display on the selection mode screen 300 are shown.

[0205] In the figure, on the left selection mode screen 300 (Public View), as the avatar 50 to be displayed, other avatars corresponding to all users who are viewing the same viewpoint image (angle) including the self-avatar are displayed. That is, it can be said that the selection mode screen 300 of Public View corresponds to a space where a large number of unspecified people gather in the event venue.

[0206] In the figure, on the central selection mode screen 300 (Friend View), as the avatar 50 to be displayed, the self-avatar and other avatars corresponding to users in the same group who are viewing the same viewpoint image (angle) are displayed. That is, it can be said that the selection mode screen 300 of Friend View corresponds to a space where only oneself and his / her friends gather in the event venue.

[0207] In the figure, on the selection mode screen 300 (Private View) on the right side, only the self-avatar is displayed as the avatar 50 to be displayed, and no other avatars are displayed. That is to say, the selection mode screen 300 of Private View can be said to correspond to the space where only oneself is in the event venue.

[0208] And the three patterns of avatar display shown in FIG. 26 are switched by operations such as pinch-in / pinch-out on the selection mode screen 300. For example, by pinch-in, the display of the selection mode screen 300 transitions from left to right in the figure, so that the number of displayed avatars decreases, and by pinch-out, the display of the selection mode screen 300 transitions from right to left in the figure, so that the number of displayed avatars increases.

[0209] Note that the operation for switching the avatar to be displayed is not limited to pinch-in / pinch-out, and may be other operations such as swiping in a predetermined direction on the selection mode screen 300 or tapping a predetermined number of times.

[0210] (Evaluation of Comments) It may be possible to evaluate comments on the perspective image input in text by other users.

[0211] For example, on the selection mode screen 300 shown in FIG. 27, as comments 220, comments 220a, 220b, 220c, and 220d input in text by other users are displayed.

[0212] Among them, comment 220d is displayed larger than the other comments 220a, 220b, and 220c because it has been tapped multiple times by users who are viewing the same perspective image. Also, in the speech bubble of comment 220d, in addition to the input text, the number of taps is also displayed. These comments 220 are displayed larger as the number of taps increases.

[0213] In this way, by prominently displaying comments according to the number of taps, it becomes possible to immediately identify comments that resonate with other users.

[0214] (Display of the perspective image on the shooting-side terminal) In the above, the display of the perspective image on the viewing-side terminal 30 has been described. However, on the shooting-side terminal 10 as well, a perspective image is displayed in the same manner as the selection mode screen 300 of the viewing-side terminal 30.

[0215] FIG. 28 is a diagram showing an example of the display of the perspective image on the shooting-side terminal 10.

[0216] The shooting screen 600 shown in FIG. 28 is displayed on the shooting-side terminal 10 (display unit 133) that is shooting the perspective image 40D.

[0217] In the shooting screen 600, the perspective image 40D being shot by the shooting-side terminal 10 is prominently displayed in the center, and the other perspective images 40A to 40C, 40E, 40F are displayed small and arranged at the top.

[0218] Also, in the shooting screen 600, an avatar 50 corresponding to the user viewing the perspective image 40D is displayed at the lower end of the perspective image 40D. The position where the avatar 50 is displayed is based on coordinate information. Further, in the shooting screen 600, a comment 220 text-input by the user corresponding to the avatar 50 is displayed in a balloon shape above the corresponding avatar 50.

[0219] Note that in the shooting screen 600, a recording button 610 for starting / ending the recording of the perspective image 40D is displayed at the center of the lower end of the perspective image 40D.

[0220] In this way, on the shooting screen 600, other viewpoint images are displayed together with the viewpoint image being shot by the shooting-side terminal 10. As a result, the shooter of the shooting-side terminal 10 can shoot his / her own viewpoint image while cooperating with other shooters by checking the viewpoint images shot by other shooters.

[0221] Also, on the shooting screen 600, together with the viewpoint image being shot by the shooting-side terminal 10, an avatar 50 corresponding to the user (viewer) who is viewing the viewpoint image and a comment 220 are displayed. Thereby, the shooter of the shooting-side terminal 10 can know the reaction of the viewer in real time while shooting the viewpoint image.

[0222] Such feedback from the viewer to the shooter contributes to the shooter's motivation to shoot a viewpoint image at a better angle.

[0223] Therefore, the part that the viewer wants to pay more attention to and view in the viewpoint image may be fed back to the shooter in real time.

[0224] For example, as shown in FIG. 29, on the selection mode screen 300 of the viewing-side terminal 30, the user (viewer) taps the part that he / she wants to pay more attention to and view in the viewpoint image with the finger Fg. At this time, on the shooting screen 600 of the shooting-side terminal 10, a tap image Tp representing the part tapped with the finger Fg is superimposed and displayed on the captured viewpoint image.

[0225] Thereby, the shooter can shoot the viewpoint image at an angle centered on the location where many tap images Tp are displayed.

[0226] (Display of Object) As an evaluation of the viewpoint image by the user viewing the viewpoint image, various objects (characters and images) may be displayed in addition to the above-described comment 220.

[0227] FIG. 30 is a diagram showing an example of object display.

[0228] In the selection mode screen 300 shown on the left side of FIG. 30, an object 711 in which a face image and a character string applied to the icon of the self-avatar are combined is displayed on the viewpoint image like a firework launched from the self-avatar. The character string of the object 711 may be preset or may be text-input by the user himself / herself.

[0229] In the selection mode screen 300 shown in the center of FIG. 30, an object 712 imitating a cheering fan is displayed on the viewpoint image like a firework launched from the self-avatar.

[0230] In the selection mode screen 300 shown on the right side of FIG. 30, an object 713 imitating a jet balloon is displayed on the viewpoint image as if it were launched from each avatar.

[0231] In this way, by displaying various objects on the viewpoint image in addition to the comment 220, the user can evaluate the viewpoint image with the same feeling as the real world.

[0232] (Map view) In the above, it is assumed that the user can participate in the event by the viewing terminal 30 accessing the URL for participating in the event.

[0233] Not limited to this, a map view screen 300M as shown in FIG. 31 may be displayed on the viewing terminal 30. The map view screen 300M displays the self-avatar 50m and the virtual map 750.

[0234] Based on the position information of the viewing terminal 30 and the map information of the event venue where the event is held on the virtual map 750, icons 751, 752, 753 representing the event venue are displayed with the self-avatar 50m as a reference.

[0235] In the map view screen 300M, the user can participate in a desired event by searching for and selecting an icon corresponding to the desired event. Also, in the map view screen 300M, by displaying other avatars corresponding to other users, it may be possible to meet up with friends who will participate in the event together.

[0236] <7. Configuration Example of Computer> The above-described series of processes can be executed by hardware or by software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer in which the program is incorporated in dedicated hardware, or a general-purpose personal computer or the like.

[0237] FIG. 32 is a block diagram showing a configuration example of the hardware of a computer that executes the above-described series of processes by a program.

[0238] The photographing-side terminal 10, the server 20, and the viewing-side terminal 30, as information processing apparatuses to which the technology according to the present disclosure can be applied, are realized by a computer having the configuration shown in FIG. 32.

[0239] The CPU 901, the ROM (Read Only Memory) 902, and the RAM (Random Access Memory) 903 are interconnected by a bus 904.

[0240] Further connected to the bus 904 is an input / output interface 905. Connected to the input / output interface 905 are an input unit 906 including a keyboard, a mouse, etc., and an output unit 907 including a display, a speaker, etc. Also connected to the input / output interface 905 are a storage unit 908 including a hard disk, a non-volatile memory, etc., a communication unit 909 including a network interface, etc., and a drive 910 that drives a removable medium 911.

[0241] In the computer configured as described above, for example, the CPU 901 loads and executes a program stored in the storage unit 908 into the RAM 903 via the input / output interface 905 and the bus 904, whereby the above-described series of processes are performed.

[0242] The program executed by the CPU 901 is recorded, for example, on a removable medium 911, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and installed in the storage unit 908.

[0243] Note that the program executed by the computer may be a program in which processing is performed in time series in the order described in this specification, or may be a program in which processing is performed in parallel or at a necessary timing such as when a call is made.

[0244] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present disclosure.

[0245] Also, the effects described in this specification are merely examples and are not limiting, and there may be other effects.

[0246] Furthermore, the present disclosure can be configured as follows. (1) A display control unit that displays a plurality of viewpoint images captured from different viewpoints in an event 1 on the own terminal, The self-user who views the plurality of viewpoint images displayed on the own terminal, and an avatar information acquisition unit that acquires avatar information regarding the display of an avatar corresponding to each of other users who are viewing the plurality of viewpoint images on other terminals Comprising The display control unit controls the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on the own terminal. An information processing apparatus. (2) The display control unit displays, together with the selected viewpoint image selected by the self-user among the plurality of viewpoint images, the self-avatar corresponding to the self-user and the other-avatar corresponding to the other user who is viewing the selected viewpoint image on the other terminal. The information processing apparatus according to (1). (3) The display control unit switches the other-avatar to be a display target according to the operation of the self-user. The information processing apparatus according to (2). (4) The display control unit switches whether to display the other-avatar in the same group as the self-avatar or not to display any of the other-avatars as the other-avatar to be a display target. The information processing apparatus according to (3). (5) The display control unit displays, together with two or more viewpoint images among the plurality of viewpoint images, the other-avatars corresponding to the other users who are viewing the two or more viewpoint images. The information processing apparatus according to (1). (6) The display control unit does not display the self-avatar corresponding to the self-user. The information processing apparatus according to (5). (7) The display control unit displays the self-avatar corresponding to the self-user in a region different from the display region of the other-avatar. The information processing apparatus according to (5). (8) The display control unit arranges the avatar in a first direction, and arranges the comment input by the self-user or the other user corresponding to the avatar in a second direction orthogonal to the first direction according to the input order. The information processing apparatus according to any one of (1) to (7). (9) The display control unit, in addition to the plurality of viewpoint images and the avatar, displays a background image corresponding to the event on the background layer of the avatar. The information processing apparatus according to any one of (1) to (8). (10) The display control unit displays different background images for each of the plurality of viewpoint images. The information processing apparatus according to (9). (11) The display control unit controls the operation of the self-avatar corresponding to the self-user according to the operation of the self-user on the self-terminal. The information processing apparatus according to any one of (1) to (4). (12) The display control unit changes the display position of the self-avatar according to the inclination of the self-terminal. The information processing apparatus according to (11). (13) The display control unit controls the operations of the self-avatar and the other avatar corresponding to the other user according to the distance between the avatars. The information processing apparatus according to (12). (14) When the distance between the avatars becomes closer than a predetermined distance, the display control unit causes the self-avatar and the other avatar to perform a specific operation. The information processing apparatus according to (13). (15) When the self-avatar and the other avatar are in a standby state for a specific operation and the distance between the avatars becomes closer than a predetermined distance, the display control unit causes the self-avatar and the other avatar to perform the specific operation. The information processing apparatus according to (13). (16) The display control unit synthesizes an image unique to the self-user or the other user corresponding to the avatar on a part corresponding to the head of the avatar. The information processing apparatus according to any one of (1) to (15). (17) The display control unit reflects display information related to the event on the avatar. The information processing apparatus according to any one of (1) to (16). (18) When the number of avatars to be displayed on the own terminal exceeds a predetermined number, the display control unit gives priority to the display of the other avatars associated with the own avatar corresponding to the own user among the other avatars corresponding to the other users, and displays the own avatar and the other avatars within a range where the number of displayed avatars does not exceed the predetermined number. The information processing apparatus according to any one of (1) to (17). (19) The display control unit displays, within a range not exceeding the predetermined number, the own avatar, the other avatars associated with the own avatar, and the other avatars randomly selected. The information processing apparatus according to (18). (20) The display control unit displays, within a range not exceeding the predetermined number, the own avatar, the other avatars associated with the own avatar, and the other avatars selected according to the viewing history of the other users. The information processing apparatus according to (18). (21) The display control unit displays, within a range not exceeding the predetermined number, the own avatar, the other avatars associated with the own avatar, and the other avatars selected by the own user. The information processing apparatus according to (18). (22) The information processing apparatus displays a plurality of viewpoint images taken from different viewpoints in one event on the own terminal, acquires avatar information regarding the display of avatars corresponding to each of the own user who views the plurality of viewpoint images displayed on the own terminal and the other users who are viewing the plurality of viewpoint images on the other terminal, controls the display of the avatars based on the avatar information corresponding to the viewpoint images displayed on the own terminal. Information processing method. (23) Causes a computer to display a plurality of viewpoint images taken from different viewpoints in one event on the own terminal, Obtain avatar information regarding the display of avatars corresponding to each of the other users who are viewing the plurality of viewpoint images displayed on the own terminal and the own user who is viewing the plurality of viewpoint images, Control the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on the own terminal A program for causing the execution of processing. (24) A distribution control unit that distributes a plurality of viewpoint images captured from different viewpoints in an event to each of the terminals of a plurality of users, A coordinate information setting unit that sets coordinate information representing the display positions of the avatars corresponding to each of the users, which is displayed together with the plurality of viewpoint videos on the terminal, A communication processing unit that transmits avatar information regarding the display of the avatar, including the coordinate information, to the terminal An information processing apparatus comprising the same.

Explanation of Signs

[0247] 1 Image distribution system, 10 Shooting-side terminal, 20 Server, 30 Viewing-side terminal, 50 Avatar, 50m Own avatar, 50s Other avatar, 71 Body, 72 Icon, 111 Shooting-side communication processing unit, 112 Image processing unit, 113 Distribution control unit, 114 Viewing-side communication processing unit, 115 Event-avatar management unit, 121 Event setting unit, 122 Avatar ID generation unit, 123 Icon setting unit, 124 Angle ID setting unit, 125 Group ID setting unit, 126 Coordinate information setting unit, 127 Comment addition unit, 128 Action standby state determination unit, 131 Shooting unit, 132 Communication processing unit, 133 Display unit, 134 Control unit, 141 Shooting control unit, 142 Display control unit, 143 Avatar information acquisition unit, 151 Communication processing unit, 152 Display unit, 153 Control unit, 161 Display control unit, 162 Avatar information acquisition unit, 163 Operation detection unit

Claims

1. A display control unit that displays a plurality of viewpoint images captured from different viewpoints in Event 1 on the own terminal, The self-user who views the plurality of viewpoint images displayed on the own terminal, and an avatar information acquisition unit that acquires avatar information regarding the display of avatars corresponding to each of the other users who are viewing the plurality of viewpoint images on the other terminal Comprising: The display control unit: Corresponding to the viewpoint image displayed on the own terminal, controls the display of the avatar based on the avatar information, When the number of avatars that can be displayed on the own terminal exceeds a predetermined number, among the other avatars corresponding to the other users, gives priority to the display of the other avatars associated with the self-avatar corresponding to the self-user, and displays the self-avatar and the other avatars within a range where the number of displayed avatars does not exceed the predetermined number An information processing apparatus.

2. The display control unit displays the self-avatar corresponding to the self-user, together with the selected viewpoint image selected by the self-user among the plurality of viewpoint images, and the other avatars corresponding to the other users who are viewing the selected viewpoint image on the other terminal. The information processing apparatus according to Claim 1.

3. The display control unit switches the other avatars to be displayed in accordance with the operation of the self-user. The information processing apparatus according to Claim 2.

4. The display control unit switches whether to display other avatars in the same group as the self-avatar as the other avatars to be displayed, or not to display any of the other avatars. The information processing apparatus according to Claim 3.

5. The display control unit displays other avatars corresponding to other users who are viewing two or more of the plurality of viewpoint images, together with the two or more viewpoint images. The information processing apparatus according to Claim 1.

6. The display control unit does not display the self-avatar corresponding to the self-user. The information processing apparatus according to Claim 5.

7. The display control unit displays the self-avatar corresponding to the self-user in a region different from the display region of the other avatars. The information processing apparatus according to Claim 5.

8. The display control unit arranges the avatar in a first direction, and arranges comments input by the self-user or the other user corresponding to the avatar in a second direction orthogonal to the first direction according to the input order. The information processing apparatus according to Claim 1.

9. In addition to the plurality of viewpoint images and the avatar, the display control unit displays a background image corresponding to the event on a background layer of the avatar. The information processing apparatus according to claim 1.

10. The display control unit displays different background images for each of the plurality of viewpoint images. The information processing apparatus according to claim 9.

11. The display control unit controls the movement of the self-avatar corresponding to the self-user according to an operation of the self-user on the self-terminal. The information processing apparatus according to claim 1.

12. The display control unit changes the display position of the self-avatar according to the inclination of the self-terminal. The information processing apparatus according to claim 11.

13. The display control unit controls the operations of the self-avatar and the other avatar corresponding to the other user according to the distance between the avatars of the self-avatar and the other avatar corresponding to the other user. The information processing apparatus according to claim 12.

14. When the distance between the avatars becomes closer than a predetermined distance, the display control unit causes the self-avatar and the other avatar to perform a specific operation. The information processing apparatus according to claim 13.

15. When the self-avatar and the other avatar are in a standby state of a specific operation and the distance between the avatars becomes closer than a predetermined distance, the display control unit causes the self-avatar and the other avatar to perform the specific operation. The information processing apparatus according to claim 13.

16. The display control unit synthesizes an image unique to the self-user or the other user corresponding to the avatar on a portion corresponding to the head of the avatar. The information processing apparatus according to claim 1.

17. The display control unit reflects display information related to the event on the avatar. The information processing apparatus according to claim 1.

18. An information processing apparatus displays a plurality of viewpoint images captured from different viewpoints in one event on a self-terminal, acquires avatar information regarding the display of avatars corresponding to each of a self-user who views the plurality of viewpoint images displayed on the self-terminal and other users who are viewing the plurality of viewpoint images on other terminals, controls the display of the avatar based on the avatar information corresponding to the viewpoint image displayed on the self-terminal. When the number of avatars of the display target that can be displayed on the own terminal exceeds a predetermined number, among the other avatars corresponding to the other users, the display of the other avatars associated with the own avatar corresponding to the own user is prioritized, and the own avatar and the other avatars are displayed within a range where the number of displayed avatars does not exceed the predetermined number. Information processing method.

19. On a computer, display a plurality of viewpoint images taken from different viewpoints in one event on the own terminal, acquire avatar information regarding the display of avatars corresponding to each of the own user who views the plurality of viewpoint images displayed on the own terminal and the other users who view the plurality of viewpoint images on the other terminals, control the display of the avatars based on the avatar information corresponding to the viewpoint images displayed on the own terminal, When the number of avatars of the display target that can be displayed on the own terminal exceeds a predetermined number, among the other avatars corresponding to the other users, the display of the other avatars associated with the own avatar corresponding to the own user is prioritized, and the own avatar and the other avatars are displayed within a range where the number of displayed avatars does not exceed the predetermined number. A program for causing the processing to be executed.

Images