Information processing device, method, and program

Abstract

There is room for improvement regarding online communication between multiple users. [Solution] The information processing device participates in a session where it can distribute distributed images based on images of a running game application. The information processing device displays a session image that includes at least one of a plurality of distributed images distributed in the session. The session image includes distributed images that are displayed in a first state and / or a second state. The information processing device displays an image indicating another information processing device that is displaying at least one of the distributed images distributed by another information processing device in a second state.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, method, and program.

Background Art

[0002] Chat systems using information communication are known (for example, Japanese Unexamined Patent Application Publication No. 2005-157586 (Patent Document 1)).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is room for improvement regarding online communication between a plurality of users.

Means for Solving the Problems

[0005] (Configuration 1) According to an embodiment, there is provided an information processing apparatus that joins a session capable of distributing a distribution image based on an image of a running game application, and displays a session image. The session image may include at least any one of a plurality of distribution images distributed in the session. The one or more distribution images included in the session image may be displayed in a first state and / or a second state, respectively. A first image including at least an image indicating a further information processing apparatus that is displaying at least any one of the distribution images distributed by another information processing apparatus participating in the session in the second state may be displayed.

[0006] (Configuration 2) In Configuration 1, the first image may be one or more images that are displayed in the vicinity of one or more of the distributed images included in the session image. The first image may also represent another information processing device that displays the distributed images in the vicinity of the first image in a second state.

[0007] (Configuration 3) In Configuration 1 or Configuration 2, the session image may include one or more second images generated at predetermined locations. Each of the second images may be associated with an information processing device participating in the session. Each of the second images may include, at a minimum, an image indicating another information processing device that distributes a distribution image if the associated information processing device is displaying one of the distribution images in a second state.

[0008] (Configuration 4) In Configuration 3, the second image may include the distributed image if the associated other information processing device is distributing the distributed image in the session.

[0009] (Configuration 5) In any of Configurations 1 to 4, the second state may be a state in which the visibility of the distributed image is higher than that of the first state.

[0010] (Configuration 6) In Configuration 5, the state of high visibility may include at least one of the following: a state of large display, a state of high resolution, a state of high frame rate, or a state of high bitrate.

[0011] (Configuration 7) In any of Configurations 1 to 6, the second state may be a state in which the sound quality associated with the playback of the streamed image is higher than that of the first state.

[0012] (Configuration 8) In any of Configurations 1 to 7, the number of distributed images displayed in the second state included in the session image may be 0 or 1.

[0013] (Configuration 9) In any of Configurations 1 to 8, the session image may include all distributed images distributed in the session.

[0014] (Configuration 10) In any of Configurations 1 to 9, the first image may include at least an image showing all information processing devices participating in the session, displaying at least one of the distributed images in the second state.

[0015] (Configuration 11) According to another embodiment, a method is provided that is performed in a game device. The method includes the steps of joining a session in which distribution images can be distributed, and displaying a session image which includes at least one of a plurality of distribution images distributed in the session, which are displayed in a first state and / or a second state, and an image showing a further information processing device that is causing at least one of the distribution images distributed by another information processing device that is participating in the session to be displayed in the second state.

[0016] (Configuration 12) A program according to yet another embodiment causes one or more processors to perform the steps of: joining a session in which distribution images can be distributed; and displaying a session image which includes at least one of a plurality of distribution images distributed in the session and displayed in a first state and / or a second state, and an image showing a further information processing device that is causing at least one of the distribution images distributed by other information processing devices participating in the session to be displayed in the second state. [Brief explanation of the drawing]

[0017] [Figure 1] This is a schematic diagram showing an example of the system configuration according to this embodiment. [Figure 2] This is a schematic diagram showing an example of the hardware configuration of a game device according to this embodiment. [Figure 3] This is a schematic diagram showing an example of the hardware configuration of a server according to this embodiment. [Figure 4] This is a schematic diagram showing an example of the software configuration of a game device according to this embodiment. [Figure 5] This figure illustrates an example of a display image generated in a game device according to this embodiment. [Figure 6] This is a diagram for explaining an example of a display image generated by the game device according to the present embodiment. [Figure 7] This is a diagram for explaining an example of a display image generated by the game device according to the present embodiment. [Figure 8] This is a diagram for explaining an example of a display image generated by the game device according to the present embodiment. [Figure 9] This is a sequence diagram for explaining an example of processing according to the present embodiment. [Figure 10] This is a diagram for explaining an example of a display image generated by the game device according to the present embodiment. [Modes for Carrying Out the Invention]

[0018] The present embodiment will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals, and redundant descriptions thereof will not be repeated.

[0019] Even when data is described using the same terminology, the contents of the data do not need to be completely identical. At least when a certain piece of data and another piece of data both convey specific information, they may be regarded as the same data. In addition, the names assigned to data are for convenience, and do not limit the scope or technical meaning indicated by the data.

[0020] A program that causes a computer to execute processing may be a single program, or may be a program group including a plurality of programs. In the present disclosure, "a certain program" does not necessarily mean a single program, and may include a program group. In addition, the entire "certain program" does not need to be stored in one device. The "certain program" may mean, for example, the entirety of a plurality of programs respectively stored in a plurality of devices included in a computer.

[0021] [A. Example System Configuration] First, an example of the system configuration according to this embodiment will be described. Figure 1 is a schematic diagram showing an example of the configuration of an information processing system 1 according to this embodiment. The information processing system 1 includes one or more game devices 100. Figure 1 shows an example of the system 1 configuration including six game devices 100 connected to a network 4. In the following description, when it is necessary to identify each of the multiple game devices 100, they will be distinguished by letters (for example, 100A, 100B, 100C, 100D, 100E, 100F as shown in Figure 1).

[0022] In System 1 according to this embodiment, users using the game device 100 have their own accounts, and each game device 100 is associated with their account. Users with accounts can have their associated account participate in one or more sessions by making a predetermined input to the game device 100. A game device 100 corresponding to an account participating in any session transmits data used for communication between users (hereinafter sometimes referred to as "communication data 10") to other game devices 100 corresponding to other accounts participating in that session. In this embodiment, "communication" may mean transmitting at least one of images (e.g., game images, camera images, text images) and sounds (which may include voice) to other game devices 100. For example, one game device 100 unilaterally transmits only the game images of the game application being run by the other game device 100 to the other game device 100, and this may also be included in "communication". Note that each game device 100 does not necessarily have an account associated with it. "Account" and "game device" may be read interchangeably.

[0023] Communication data 10 may include, for example, sound data acquired by the game device 100, image data acquired by the game device 100, text data acquired by the game device 100, and information about applications running on the game device 100. Communication data 10 may also include management information 208, which will be described later. Note that not all data needs to be included in a single communication data 10; one piece of data may be split into multiple communication data 10s. A single communication data 10 may contain multiple pieces of data.

[0024] In this specification, “image” includes moving images and one or more still images. Images may be images captured using a camera 124 or the like available on the game device 100, or images of an application running on the game device 100. In this specification, “image” includes data used to generate or output images. Similarly, “sound” and “text” in this specification also include data used to generate or output them.

[0025] System 1 may further include a management server 200. The management server 200 is responsible for processes such as configuring new sessions, processing for one or more game devices to join a session, and relaying communication data 10 sent and received by one or more game devices 100. The management server 200 has management information 208 for managing configured sessions and one or more accounts belonging to each session. The management information 208 may be generated by the management server 200. It may also be generated based on communication data 10 transmitted by one or more game devices 100. Information processing system 1 may include multiple management servers 200. If information processing system 1 includes multiple management servers 200, some and other parts of the processing to realize specific functions handled by the management server may be executed by separate management servers 200. The management server 200 may be a virtual server. Hereafter, "server" may mean the management server 200 or other servers.

[0026] The method by which one game device 100 transmits communication data 10 to one or more other game devices 100 may be a method that transmits the communication data 10 directly to one or more game devices 100, in addition to a method that uses one or more relay entities (e.g., a management server 200). An example of a method that transmits communication data 10 directly is, for example, a P2P method.

[0027] [B. Hardware Configuration Example] Next, an example of the hardware configuration of each device included in the information processing system 1 according to this embodiment will be described.

[0028] (b1: Game device 100) Figure 2 is a schematic diagram showing an example of the hardware configuration of a game device 100 according to this embodiment. The game device 100 is, for example, an information processing device for providing games to a user. The game device 100 may be a device dedicated to games, or it may be a device capable of running programs other than games. The game device 100 may be, for example, a personal computer, a television, a tablet terminal, a smartphone, a wearable device, a server, etc.

[0029] Furthermore, each of the game devices 100 constituting the information processing system 1, or each of the game devices 100 participating in any session, does not have to have the same hardware configuration; some or all of them may have different hardware configurations. The game device 100 only needs to be a computer capable of running game applications. In this specification, "computer" does not necessarily mean a single device, but may refer to an entire system in which multiple devices are connected by wires or wirelessly.

[0030] Referring to Figure 2, the game device 100 includes, for example, one or more processors 102, one or more memories 104, a communication unit 106, storage 108, a camera interface 110, an input unit 116, a speaker 118, a display 120, and a microphone 122. Each component may be detachable from the main body of the game device 100.

[0031] The processor 102 is the processing unit for executing processes in the game device 100. The processor 102 includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 102 loads the programs stored in the storage 108 into the memory 104 and executes them.

[0032] The memory 104 is a volatile storage medium accessible by the processor 102, and includes, for example, DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory).

[0033] The storage 108 is a non-volatile storage medium accessible by the processor 102, and includes, for example, flash memory or a hard disk. The storage 108 may also include, for example, a storage medium that can be attached to or removed from the game device 100, such as a cartridge or an optical disc. The storage 108 stores, for example, one or more programs, such as a system program 112 or a game application 180.

[0034] The system program 112 includes computer-readable instructions for hardware control of the game device 100 and providing a program execution environment. The system program 112 may include one or more programs or program modules. For example, the system program 112 includes a communication application 114.

[0035] The communication application 114 performs processing related to the sending and receiving of communication data 10 between multiple game devices 100 participating in the session.

[0036] The game application 180 includes computer-readable instructions for running the game. Multiple game applications 180 may be installed on the game device 100. The game application 180 may also be read from a separate storage medium. At least part of the game execution by the game application 126 may utilize other game devices 100 or external resources such as a server. The system program 112 may also include at least part of the game application 180.

[0037] In this specification, the term "processor" includes at least a processing circuit that performs processing according to computer-readable instructions, such as a CPU or GPU, and hardwired circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). In this specification, the term "memory" may include at least flash memory and DRAM, and may further include other storage media such as storage 108.

[0038] In the game device 100, a System on Chip (SoC) in which functions such as a processor and memory are integrated may be used. In this case, the processor and memory reside within the same SoC. Therefore, in this specification, the terms "processor" and "memory" include both independent and integrated configurations.

[0039] The communication unit 106 transmits and receives data to and from other game devices 100, management servers 200, etc., via the network 4. As an example of the network 4, the communication unit 106 may communicate data with other game devices 100, etc., via the Internet. The network 4 may also be a wireless network conforming to Bluetooth®, ZigBee®, wireless LAN (IEEE 802.11 standard), etc.

[0040] The camera interface 110 mediates the transmission of data to and from the camera 124. The camera interface 110 may be connected to the camera 124 by a wire or by wireless connection.

[0041] The input unit 116 accepts user input. The input unit 116 includes, for example, a keyboard, mouse, game controller, touch panel, etc. The input unit 116 may also be an interface with a device that accepts user input. Thus, the device that accepts user input may be located outside the game device 100.

[0042] The display 120 displays images or videos generated by processing by the processor 102. The display 120 includes, for example, an LCD (Liquid Crystal Display) or an organic EL display. The display 120 may be located outside the game device 100. In this case, the game device 100 may include an interface circuit to the display 120.

[0043] Microphone 122 is located in the casing of the game device 100 and collects sound to generate a sound signal. Speaker 118 is located in the casing of the game device 100 and generates sound from the sound signal. At least one of microphone 122 and speaker 118 may be located outside the game device 100.

[0044] The camera 124 generates an image of the subject according to commands from the game device 100 or according to predetermined conditions. The camera 124 includes an image sensor such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. A lens may be placed in front of the image sensor.

[0045] Camera 124 may be, for example, a camera for video calls. Camera 124 may be a dedicated camera that can only be connected to a specific device, or it may be a general-purpose camera that can be connected to any device. Multiple cameras 124 may be connected to the game device 100. Furthermore, the game device 100 may utilize the camera of another game device (for example, a smartphone). Thus, the configuration in which the game device 100 utilizes camera 124 is not limited. (b2: Management Server 200) Figure 3 is a schematic diagram showing an example of the hardware configuration of a management server 200 according to this embodiment. Referring to Figure 3, the management server 200 includes, for example, one or more processors 202, one or more memories 204, one or more communication units 206, storage 210, and an input unit 216.

[0046] Processor 202 is the processing unit for executing processes in the management server 200. Processor 202 includes, for example, a CPU or GPU. Processor 202 loads programs stored in storage 210 into memory 204 and executes them.

[0047] Memory 204 is a volatile storage medium accessible by the processor 202, and includes, for example, DRAM or SRAM.

[0048] The storage 210 is a non-volatile storage medium accessible by the processor 202, and includes, for example, a hard disk or flash memory. The storage 210 stores, for example, a system program 212 and a management program 214.

[0049] The system program 212 includes computer-readable instructions for hardware control of the management server 200 and providing a program execution environment.

[0050] The management program 214 includes computer-readable instructions for executing management-related processes that are handled by the management server 200.

[0051] The management information 208 may be stored in the storage 210 or memory 204. The communication unit 206 sends and receives data to and from the game device 100, management server 200, etc., via the network 4. The communication unit 206 may be connected to the network 4 by wire or by wireless connection.

[0052] The input unit 216 accepts user input. The input unit 216 includes, for example, a keyboard or mouse.

[0053] [C. Example of software configuration for game device 100] Next, an example of the software configuration of the game device 100 according to this embodiment will be described. Figure 4 is a schematic diagram showing an example of the software configuration of the game device 100 according to this embodiment. Referring to Figure 2, in the game device 100, a system program 112 including a communication application 114 and a game application 180 can be executed simultaneously. Note that "executed simultaneously" means that multiple applications or programs are executed in parallel, and does not limit the execution timing to the same time.

[0054] The game device 100 includes, as an example of software components, a system module 132 and a game execution unit 186. The system module 132 may be implemented by one or more processors 102 executing a system program 112. The system module 132 may also be referred to as the running system program 112. The game execution unit 186 may be implemented by one or more processors 102 executing a game application 180. The game execution unit 186 may also be referred to as the running game application 180. The communication execution unit 140 may also be referred to as the running communication application 114.

[0055] The game execution unit 186 advances the game based on user input and outputs game images 182 and game sounds 184. The game images 182 may include multiple types of images with different image quality, resolution, frame rates, etc.

[0056] The system module 132 provides the basic functions necessary for the game device 100. The system module 132 includes, for example, a communication execution unit 140.

[0057] The system module 132 does not necessarily have to include at least a part of the communication execution unit 140. At least a part of the communication application 114 may be a separate application from the system program 112, similar to the game application 180. In the following description, the communication execution unit 140 may execute some or all of the processes performed by the system module 132. Therefore, in the context of the processes performed by the system module 132, the system module 132 can be read as the communication execution unit 140, or the system module 132 can be read as both the system module 132 and the communication execution unit 140.

[0058] The system module 132 includes, for example, a microphone control unit 144, a communication control unit 148, an image processing unit 150, and a sound processing unit 152.

[0059] The microphone control unit 144 generates sound 154 based on the sound collected by the microphone 122, etc.

[0060] The image processing unit 150 has access to a storage area 160 for storing images. The storage area 160 may, for example, utilize at least a portion of one or more memory 104 or storage 108. The image processing unit 150 may be capable of image processing on one or more images.

[0061] The memory area 160 may include, for example, an area for storing a display image Sd, a game image 182, one or more distribution images 164, a camera image 156, one or more received distribution images 166, and one or more received camera images 168.

[0062] The communication execution unit 140 performs processing to exchange communication data 10 with other game devices 100 within the session. For example, the communication execution unit 140 may perform processing necessary to join or leave a specified session in response to user operations.

[0063] The communication execution unit 140 may generate and transmit communication data 10 based on at least one of the following: sound 154, a transmitted image 164, and a camera image 156, as processing for distribution. The transmitted image 164 may be generated based on a game image 182. Note that "distribution" may mean, for example, that one game device 100 directly or indirectly transmits some data to one or more other game devices 100. "Distribution" and "transmission" may be interpreted as mutually exclusive.

[0064] The communication execution unit 140 may, as processing for receiving the distribution, store in the storage area 160 one or more received distribution images 166 and received camera images 168, etc., which are included in one or more communication data 10 from one or more other game devices.

[0065] The communication execution unit 140 may, as processing for receiving the data, output the received sound 158 contained in one or more communication data 10 from one or more other game devices 100 to the sound processing unit 152. The sound processing unit 152 may generate a sound signal based on the sound 158 and output the generated sound signal to the speaker 118.

[0066] The image processing unit 150 may generate a display image Sd using at least one of the following: a game image 182, a camera image 156, one or more distributed images 164, one or more received distributed images 166, and one or more received camera images 168, in accordance with a command from the communication execution unit. The display image Sd is used to display an image on the device's display 120.

[0067] The communication control unit 148 issues commands to the communication unit 106 to control data communication with other game devices 100, the management server 200, etc. The communication control unit 148 also receives the distributed images 164 and 156 distributed by each game device 100 as the received distributed image 166 or the received camera image 168, respectively. In this specification, "distributed image 164" and "received distributed image 166" may be interpreted as mutually exclusive. In this specification, "camera image 156" and "received camera image 168" may be interpreted as mutually exclusive.

[0068] [D. Example of displayed image] Figure 5 is a diagram illustrating an example of a display image Sd100A generated in a game device 100A according to this embodiment. The display image Sd100A shown in Figure 5 is just one example, and the screen configuration of the display image Sd generated using the communication data 10 is not limited to this.

[0069] In Figure 5, the display image Sd100A includes region MI where the game image 182A of the game application running on game device 100A is displayed. The display image Sd100A also includes regions SI1 to SI4. Region MI is larger than each of the regions SI. Regions SI1 to SI4 can each display the first distributed image 1641 distributed within the session. The first distributed image 1641 is a type of distributed image 164, and details will be described later. In this embodiment, four game devices 100 can simultaneously distribute distributed images within a session. Therefore, the display image Sd100A may contain up to four regions SI. That is, one game device 100 is associated with each region SI. In this embodiment, game devices 100A to 100D can distribute distributed images 164, and game devices 100A to 100D are associated with regions SI1 to SI4, respectively. For example, in the display image Sd100A, area SI1 is associated with game device 100A, and the first distribution image 1641A based on game image 182A is displayed. In this embodiment, game devices 100A to 100C actually distribute the distribution image 164, while game device 100D does not distribute the distribution image 164. Therefore, in the display image Sd100A, areas SI1 to SI3 display the first distribution images 1641A to 1641C, which are distributed from game devices 100A to 100C, respectively. The first distribution image 1641 is not displayed in area SI4. Area SI4 may also display a non-distribution indicator 174 indicating that distribution is not taking place. The non-distribution indicator 174 may include, for example, information about the associated game device 100, or it may include a message or icon indicating that distribution is not taking place. In this embodiment, the position of each area SI within the display image Sd100A is predetermined. "Predetermined" does not mean that it is always constant, and includes the fact that the position, number, or size of each area SI may differ depending on the number of game devices 100 in the session, the number of game devices 100 that can be distributed, and the predetermined screen layouts that can be selected. Furthermore, not limited to the non-distribution display 174, the area SI to which a game device 100 is associated may include information indicating the associated game device 100.

[0070] The displayed image Sd100A includes regions Wi1 and Wi2. Regions Wi1 and Wi2 are associated with game devices 100E and 100F, respectively, which are not associated with any region SI. Region Wi is associated with game device 100, which is not associated with any region SI. Regions Wi1 and Wi2 display information corresponding to game devices 100E and 100F, such as the account name and icon image associated with each game device 100.

[0071] Camera images 156 distributed by the game device 100 associated with each region SI may be displayed in the vicinity of that region SI. The vicinity includes superposition. Camera images 156 may or may not be displayed in the vicinity of each region Wi. The distribution of camera images 156 may be switched on or off by the user at will, similar to the distribution of distributed images 164.

[0072] In other embodiments, the number of region SIs is not limited; for example, one or more region SIs may be associated with all game devices 100 participating in the session. Furthermore, the number of region SIs included in the display image Sd100A is not limited. Also, while the number of region SIs simultaneously included in the display image Sd100A is limited, each region SI can be scrolled, allowing for the display of more region SIs than those currently shown. Additionally, all game devices 100 in a session can arbitrarily select whether or not to distribute the distribution image 164. Game devices 100 may be associated with region SIs when distribution is enabled and with region Wi when distribution is disabled.

[0073] In this embodiment, game devices 100B to 100F also generate display images Sd100B to Sd100F, which include regions MI, SI, and Wi. In the region MI of each display image Sd100B to 100F, game images 182B to 182F of the game application executed on each game device 100B to 100F are displayed. On the other hand, in each game device 100, regions SI1 to SI4 are associated with game devices 100A to 100D, and region Wi is associated with game devices 100E and 100F. Note that which game device 100 is associated with which region of regions SI1 to SI4 and regions Wi1 to Wi2 may differ or be the same for each game device 100.

[0074] As shown in Figure 5, a user of game device 100A can play a game application running on game device 100A while viewing the first distributed image 1641 of another game device 100. For example, the game image 182 displayed in area MI may have higher visibility than the first distributed image 1641 displayed in display image Sd100. On the other hand, a user may want to view a more visible distributed image 164 of another game device 100. Therefore, in this embodiment, the display image Sd100 may display a second distributed image 1642 that has higher visibility than the first distributed image 1641. In this embodiment, "visibility" may mean at least one of the following parameters: size, resolution, frame rate, and bitrate, or it may mean other parameters. Furthermore, one image having higher visibility than another image may include, for example, a case where at least one of the above-executed parameters in one image is higher than in the other image, and other parameters in one image are lower than in the other image.

[0075] Figure 6 is a diagram illustrating another example of a display image Sd100A generated in a game device 100A according to this embodiment. The scene shown in Figure 6 may be before or after the scene shown in Figure 5, and differs from the scene shown in Figure 5 in that the game device 100A is displaying the second distribution image 1642B of the game device 100B.

[0076] As shown in Figure 6, the second distribution image 1642B is displayed in area MI of the display image Sd100A of the game device 100A. The second distribution image 1642 is a type of distribution image 164 and has higher visibility than the first distribution image 1641. For example, a user of game device 100A may want to view the game image of the game application corresponding to the first distribution image 1641 of game device 100B, which is displayed in area SI2, as a more visible image. In this embodiment, as an example, a user of game device 100A can display the second distribution image 1642B in area MI by selecting game device 100B. The visibility of the game image 182 may be higher than the visibility of the second distribution image 1642. In addition, area MI, which includes the second distribution image 1642B, may display other information. For example, an image showing the source of the second distributed image 1642 displayed in the MI area, or an image guiding the user on how to stop viewing the second distributed image 1642, may be displayed.

[0077] In this embodiment, the game image 182A of the game application running on the game device 100A is hidden when the second distribution image 1642B is displayed in area MI. However, for example, the game image 182A and the second distribution image 1642B may be displayed in different areas. Since area MI is larger than area SI, the user of the game device 100A can view the distribution image 164 of the game device 100B with better visibility. The second distribution image 1642B may have a higher resolution than the first distribution image 1641B, and the sound of the game application may be played in conjunction with the display of the second distribution image 1642B.

[0078] In this embodiment, the game device 100 generates and distributes a first distribution image 1641 and a second distribution image 1642 based on the game image 182. The server distributes either the first distribution image 1641 received from the source game device 100, or both the first distribution image 1641 and the second distribution image 1642, to each destination game device 100. For example, if game device 100B selects to display the first distribution image 1641A of game device 100A, and game device 100C selects to display both the first distribution image 1641A and the second distribution image 1642A of game device 100A, the server may distribute the first distribution image 1641A to game device 100B and the first distribution image 1641A and the second distribution image 1642A to game device 100C.

[0079] The server obtains information from each game device 100 indicating which game device 100 has selected to view or is currently viewing the second distributed image 1642. Based on the obtained information, the server may generate viewing information 2081, which is information regarding the display status of the distributed image 164 on each game device 100. The viewing information 2081 may be included in the management information 208. For example, based on the viewing information 2081, the server decides which destination game device 100 to distribute the first distributed image 1641 and / or second distributed image 1642 from which source game device. For example, based on viewing information 2081 indicating that one game device 100 is requesting to view the second distributed image 1642 of another game device 100, the server distributes the first distributed image 1641 and the second distributed image 1642 of that game device to the other game device. Furthermore, the viewing information 2081 may include information indicating which game device 100 is requesting to view the first distributed image 1641 of. In this embodiment, the viewing information 2081 is transmitted to each game device 100.

[0080] In other embodiments, the server may distribute only one of the first distribution image 1641 and the second distribution image 1642 to each destination game device 100. The server may distribute both the first distribution image 1641 and the second distribution image 1642 to all game devices 100, and the receiving game device 100 may display either or both of the distribution images 164. Furthermore, if all of the destination game devices 100 have selected to display the first distribution image 1641, the source game device 100 may distribute the first distribution image 1641 while not distributing the second distribution image 1642. In addition, only one type of distribution image 164 is distributed from the game device 100, and the destination game device 100 may change the size of the received distribution image 164 or apply upscaling or downscaling processing to the received distribution image 164. Furthermore, when two game devices 100 communicate via P2P, the source game device 100 may distribute either the first distribution image 1641 or the second distribution image 1642 to each destination game device 100 in accordance with the display selection of the destination game device 100.

[0081] As shown in Figure 6, when game device 100A is displaying the second distribution image 1642B of game device 100B, the viewing destination information image 172A is displayed in the region SI1 associated with game device 100A. The viewing destination information image 172A indicates that game device 100A is displaying the second distribution image 1642B distributed by game device 100B. The viewing destination information image 172A may include a message or an icon. As will be described later, the viewing destination information image 172A is also displayed in the display image Sd of other game devices 100. However, the viewing destination information image 172A displayed by game device 100A and other game devices 100 may be different. The viewing destination information image 172 may be generated by the server and sent to each game device 100, or each game device 100 may generate it. For example, each game device may receive viewing information 2081 from the server and generate the viewing destination information image 172 based on the viewing information 2081.

[0082] In addition to, or instead of, the viewing destination information image 172A, an auxiliary image may be displayed near the region SI1 to which the game device 100A is associated. The auxiliary image may indicate the direction in which another region SI is located, which is the source of the second distributed image 1642 being viewed by the game device 100 corresponding to that region SI, and to which another region SI is associated. The auxiliary image is, for example, an icon. For example, the auxiliary image may be an arrow or an icon representing an eye. In the example shown in Figure 7, for example, the eye icon, which is an auxiliary image, may be superimposed on the viewing destination information image 172A, and the eye icon may be displayed looking to the right to indicate the direction that includes region SI2. Note that since regions SI2 to SI4 are located to the right of region SI1, the user cannot determine from the auxiliary image alone whether game device 100A is viewing the second distributed image 1642 of game device 100B or game device 100C.

[0083] Furthermore, the viewing destination information image 172 and / or auxiliary images may also be displayed in the vicinity of each region Wi. For example, when a game device 100E associated with Wi1 is viewing the second distribution image 1642 of game device 100A, a message such as "Viewing Account A's Screen" or an eye icon pointing towards region SI1 may be displayed in the vicinity of region Wi1 as the viewing destination information image 172. As an example, the direction that the icon as the viewing destination information image 172 displayed in the vicinity of Wi1 points will differ depending on which of the second distribution images 1642 of the game devices 100 corresponding to regions SI1 to SI4 the game device 100 is viewing. For example, an icon as an auxiliary image may also be displayed in the vicinity of region Wi1, and this icon may point upwards in the same direction regardless of which of the second distribution images 1642 of the game devices 100 corresponding to regions SI1 to SI4 the game device 100 is viewing.

[0084] When game device 100A is displaying the second distribution image 1642 of another game device 100, the game application running on game device 100A may be stopped. At this time, game device 100A may also stop distributing the distribution image 164 based on the game image 182, and may also stop generating the distribution image 164. Note that stopping the game application may mean that the game application is not running in the foreground, or it may mean that the application is temporarily stopped so that, for example, when game device 100A finishes displaying the second distribution image 1642, the game application will resume from the point when it started displaying the second distribution image 1642.

[0085] Furthermore, in this embodiment, when the second distribution image 1642 of one game device 100 is displayed on another game device 100, a viewing source information image 170 indicating the other game device 100 is displayed near the first distribution image 1641B of that game device 100 displayed in area SI. The viewing source information image 170 may include information indicating multiple other game devices 100. The viewing source information image 170 may include messages or icons. For example, as shown in Figure 6, the first distribution image 1641B of game device 100B is superimposed with a viewing source information image 170B indicating game device 100A displaying the second distribution image 1642B of game device 100B. The viewing source information image 170 may be generated by a server and sent to each game device 100, or each game device 100 may generate it. For example, each game device may receive viewing information 2081 from the server and generate the viewing source information image 170 based on the viewing information 2081.

[0086] In the state shown in Figure 7, game devices 100C to 100F are not displaying the second distribution image 1642. Therefore, the viewing source information image 170 displayed in regions SI1 to SI4 does not include information related to game devices 100C to 100F, which are not displaying the second distribution image 1642. In addition, the viewing destination information image 172 is not displayed in each region corresponding to game devices 100C to 100F.

[0087] Figure 7 shows an example of the display image Sd100B generated by game device 100B when the display image Sd100A shown in Figure 6 is being generated by game device 100A.

[0088] As shown in Figure 7, the MI region in the displayed image Sd100B shows the game image 182B of the game application being run on the game device 100B.

[0089] As shown in Figure 7, area SI1 is associated with game device 100B, and displays the first distributed image 1641B being distributed by game device 100B. Area SI2 is associated with game device 100A, and displays the viewing destination information image 172A indicating the game device 100 being viewed by game device 100A. The other areas SI and area Wi are associated in the same way as the areas in the display image Sd100A of game device 100A are associated with game device 100. In this way, for example, by associating the user's own game device 100 with the leftmost area SI, it is easy for the user operating the game device 100 to understand the image they are distributing. Note that the association between each area SI and game device 100 may be the same for all game devices 100.

[0090] Since the second distributed image 1642B distributed by the game device 100B is viewed by the game device 100A, the viewing source information image 170B is superimposed on the first distributed image 1641B. In this embodiment, the viewing source information image 170B is superimposed on the first distributed image 1641B and is not displayed in the vicinity of, for example, the game image 182B. In other embodiments, the viewing source information image 170B may be displayed in the vicinity of the game image 182B in addition to, or instead of, being displayed in the vicinity of the first distributed image 1641B.

[0091] Figure 8 shows an example of the display image Sd100C generated by the game device 100C when the display image Sd100A shown in Figure 6 is being generated by the game device 100A.

[0092] As shown in Figure 8, region SI1 is associated with game device 100C, and displays the first distributed image 1641C distributed by game device 100C. Region SI2 is associated with game device 100A, and displays the viewing destination information image 172A indicating the game device 100 that game device 100A is viewing. Region SI3 is associated with game device 100B, and displays the first distributed image 1641B distributed by game device 100B. Regions SI4 and Wi are associated in the same way as the association between each region and game device 100 in the display image Sd100A of game device 100A.

[0093] The second distributed image 1642C distributed by game device 100C is not viewed by other game devices 100. Also, game device 100C is not viewing the second distributed image 1642 distributed by other game devices 100. On the other hand, area SI2 displays viewing destination information image 172A indicating that game device 100A is viewing the second distributed image 1642B of game device 100B, and the first distributed image 1641B of game device 100B displayed in area SI3 has viewing source information image 170B superimposed on it, indicating that game device 100A is viewing the second distributed image 1642B of game device 100B.

[0094] As shown in Figure 8, in this embodiment, the image generated by a game device 100 participating in a session includes information indicating the viewing relationship between other game devices 100 and yet other game devices 100 participating in the same session. This can improve the user experience. For example, it can stimulate online communication within a session.

[0095] Furthermore, in the examples shown in Figures 6 to 8, the images generated by each game device 100 participating in the session display information indicating the viewing relationship between all other game devices 100 participating in that session and all other game devices 100. This improves the user experience.

[0096] [E. Processing Example] Figure 9 is a sequence diagram showing an example of processing according to this embodiment. The processing shown in Figure 9 is just one example, and other processing may be included, or some processing may be omitted. Also, the execution order of each processing is just one example, and for example, multiple processing may be executed in parallel, or some processing may be executed in the reverse order of the description. For the sake of explanation, the process is divided into units called steps, but these units of division are arbitrary, and multiple steps may be combined, or one step may be divided into multiple steps. The same applies to the descriptions that follow.

[0097] Figure 9 shows an example of the process until the display image Sd shown in Figures 6 to 8 is generated. Figure 9 shows the processing of game devices 100A to 100D and the management server 200. In the initial state of the example shown in Figure 9, the communication application 114 is already running on game devices 100A to 100D, and game device 100A is participating in the same session as game devices 100B to 100D. Also, game applications 180A to 180C are running on game devices 100A to 100C, and game devices 100A to 100C are distributing the first distribution images 1641A to 1641C and the second distribution images 1642A to 1642C within the same session. Game devices 100E and 100F are also participating in the same session as game devices 100A to 100D, but are omitted in Figure 9. For these game devices 100E and 100F, at least some of the processing performed in relation to the management server 200 is common.

[0098] Furthermore, in Figure 9, the management information 208, the first distribution image 1641, and the second distribution image 1642 included in the communication data 10 are transmitted and received separately, but these may also be transmitted and received together as the communication data 10.

[0099] The management server 200 sends management information 208 to each game device 100 participating in the session (step S101). The transmission of management information 208 to each game device 100 may be performed simultaneously or separately. "Simultaneous execution" means that multiple applications or programs are executed in parallel, and does not limit the execution timing to the same time.

[0100] Although not shown in the figures, in this embodiment, the transmission of management information 208 is repeatedly performed to each game device participating in the session. The transmission of management information 208 may be performed periodically at predetermined time intervals, or when a change in state occurs in each game device 100 or the server. For example, it may be performed in response to an update of management information (S114). Alternatively, the transmission of management information 208 may be performed in response to a request from each game device 100.

[0101] Not limited to the transmission of management information 208, each transmission process in this example may be performed based on a request from the receiving side, periodically at predetermined time intervals, or based on a change in the state of the sending or receiving side. Furthermore, each of the "predetermined time intervals" used herein does not have to be the same value. In other words, the "predetermined time intervals" used herein are not all limited to being the same "predetermined time interval".

[0102] Returning to Figure 9, game device 100A transmits the first distribution image 1641A and the second distribution image 1642A, which were generated based on the game image 182A output by the game execution unit 186, to the management server 200 (step S102). Similarly, game device 100B transmits the first distribution image 1641B and the second distribution image 1642B, which were generated based on the game image 182B, to the management server 200 (step S103), and game device 100C transmits the first distribution image 1641C and the second distribution image 1642C, which were generated based on the game image 182C, to the management server 200 (step S104). Note that game device 100D does not transmit.

[0103] Next, the management server 200 transmits to game device 100A the first distribution image 1641B and the first distribution image 1641C, which are the first distribution image 1641 transmitted by the other game devices 100 participating in the session (step S105). The management server 200 also transmits to the other game devices 100B to 100D, which are the other game devices participating in the session, each of the first distribution images 1641 that were transmitted by the other game devices 100 (steps S106 to S108).

[0104] In this embodiment, the processes in steps S102 to S108 are repeated. Each of steps S102 to S108 may be performed periodically at predetermined time intervals, or it may be performed when a change in state occurs for each game device 100 or each distributed image 164.

[0105] Based on user operation, the game device 100A requests the management server 200 to view the second distributed image 1642B (step S109). The management server 200 updates the management information 208 in response to the viewing request (step S110). In this embodiment, viewing information 2081 indicating that the second distributed image 1642B of the game device 100B is distributed to the game device 100A is generated as the management information 208. Subsequently, the game device 100A receives the second distributed image 1642B transmitted by the management server 200 in response to the viewing request (step S111). Steps S110 and S111 may be executed in reverse order, that is, the management information 208 may be updated after the transmission of the second distributed image 1642B. Furthermore, the processing in step S110 may be performed periodically at predetermined time intervals, or when a change occurs in the state of the management server 200 or the game device 100 participating in the session.

[0106] Next, based at least on the received second distribution image 1642B and the updated management information 208, the game device 100A generates the display image SdA illustrated in Figure 6 (step S112). Also, based at least on the updated management information 208, the game device 100B generates the display image SdB illustrated in Figure 7, and the game device 100C generates the display image SdC illustrated in Figure 8 (steps S113-S114). Similarly, based at least on the updated management information 208, the game device 100D generates the display image SdD (step S115). Game devices 100E and 100F also generate display images in the same manner. In this embodiment, the generation of display images (steps S112-S115) is repeated from before the timings described. Each of steps S112 to S115 may be performed periodically at predetermined time intervals, or it may be performed when a change in state occurs for each game device 100 or each distributed image 164.

[0107] Figure 10 is a diagram illustrating another example of a display image Sd100A generated in a game device 100A according to this embodiment.

[0108] As shown in Figure 10, the MI region displays the game image 182A of the game application currently running on the game device 100A.

[0109] In region SI1, to which game device 100A is associated, the first distributed image 1641A and the viewer source information image 170A superimposed on the first distributed image 1641A are displayed. The viewer source information image 170A indicates game device 100B receiving the second distributed image 1642A distributed by game device 100A.

[0110] The region SI2 to which the game device 100B is associated displays the viewing destination information image 172B. The viewing destination information image 172B indicates that the game device 100B is displaying the second distribution image 1642A distributed by the game device 100A.

[0111] In the region SI3 associated with game device 100C, the first distributed image 1641C and the superimposed viewing source information image 170C are displayed. The viewing source information image 170C indicates game devices 100D, 100E, and 100F that are viewing the second distributed image 1642C distributed by game device 100C. Thus, in this embodiment, if the second distributed image 1642 of a certain game device 100 is displayed by multiple other game devices 100, the viewing source information image 170 includes information indicating all of those game devices 100. Note that, for example, it is not necessary for information indicating game devices 100D, 100E, and 100F to be displayed simultaneously at all times. For example, an icon indicating that the image is being viewed by two or more game devices 100 may be displayed, and when the icon is selected, the tops of all the viewing game devices 100 may be displayed. Note that in other embodiments, for example, there may be an upper limit on the number of game devices 100 on which the information is displayed. For example, if the number of game devices 100 on which information is displayed exceeds the upper limit, a display indicating that there are game devices 100 on which information is not displayed may be shown.

[0112] The region SI4 to which the game device 100D is associated displays the viewing destination information image 172D. The viewing destination information image 172D indicates that the game device 100D is displaying the second distribution image 1642C distributed by the game device 100C.

[0113] Game devices 100E and 100F are viewing the second distributed image 1642C distributed by game device 100C. However, in this embodiment, the region Wi1 to which each game device 100E and game device 100F is associated does not display anything corresponding to the fact that each game device 100 is displaying the second distributed image 1642 distributed by the other game device 100. However, in other embodiments, for example, the region Wi may display a viewing destination information image 172 corresponding to the fact that the associated game device 100 is displaying the second distributed image 1642 distributed by the other game device 100.

[0114] [F. Variant] The display format of the distributed image 164 of one's own game device 100 in the display image Sd may differ from the display format of the distributed image 164 of other game devices 100. For example, one's own first distributed image 1641 may be displayed smaller than the first distributed image 1641 of others, or it may be displayed with a border of a different color. Also, one's own first distributed image 1641 may not be displayed in the display image Sd.

[0115] The position of the region SI where the first distributed image 1641 is displayed is not limited. Multiple region SIs may be arranged vertically, or they may be arranged around the center. Furthermore, the position of region SIs may be moved by user operation.

[0116] The second distributed image 1642 may have better visibility than the first distributed image, or alternatively, better sound quality. Good sound quality includes louder volume and also includes the presence of sound compared to silence.

[0117] The area in the display image Sd on which the second distribution image 1642 is displayed is not limited to area M1. For example, it may be displayed in area SI. When the first distribution image 1641 distributed by a game device 100 is displayed in area SI, the second distribution image 1642 may be displayed in the same area SI based on user operation. Alternatively, the second distribution image 1642 may be displayed in an area other than area MI and area SI. As an example, the second distribution image 1642 may be displayed in full screen. The area on which the second distribution image 1642 is displayed may be in a position and / or size that includes at least a part of area MI and / or area SI.

[0118] The viewing destination information image 172 may be displayed in a region different from region SI in the display image Sd. For example, the viewing destination information image 172 may be displayed near region MI, or it may be displayed in a position that is not near either region MI or region SI. Similarly, the display position of the viewing source information image 170 is not limited to the vicinity of the first distribution image 1641 in the display image Sd, but may also be near region MI, or it may be in a position that is not near either region MI or region SI.

[0119] In the above example, the viewing destination information image 172 was displayed in each region SI associated with each game device 100 that is viewing the second distribution image 1642 of another game device 100. The viewing destination information image 172 may be, for example, a list, which is an image that collectively contains information indicating multiple game devices 100 that are viewing the second distribution image 1642 of another game device 100. Similarly, the viewing source information image 170 may not be displayed in each region SI, but may be, for example, a list, which is an image that collectively contains information indicating which game device 100's second distribution image 1642 is being viewed by which other game device 100.

[0120] The viewing destination information image 172 and / or viewing source information image 170 may be displayed based on user interaction. For example, a list showing the viewing destination information image 172 and / or viewing source information image 170 may be displayed based on user interaction.

[0121] The viewing destination information image 172, which is displayed near a certain region SI, may be displayed when the display of the first distribution image 1641, which was displayed in the region SI, stops when the game device 100 associated with that region SI starts viewing the second distribution image 1642. For example, when the game device 100 is not distributing the first distribution image 1641, and starts viewing the second distribution image 1642 of another game device 100, the viewing destination information image 172 indicating the game device 100 that the game device 100 is viewing does not need to be displayed. Based on the display of the viewing destination information image 172, the user can find out why the display of the first distribution image 1641 on the game device 100 stopped.

[0122] In this embodiment, the source viewing information image 170 and the destination viewing information image 172 are distinguished. Information indicating that one game device 100 is viewing another game device 100 may also be information indicating that the other game device 100 is being viewed by another game device 100. In other words, information indicating the source viewing and information indicating the destination viewing may not need to be distinguished. For example, the game device 100 may display only one of the source viewing information image 170 or the destination viewing information image 172, or it may display other images indicating the viewing relationship between game devices 100.

[0123] Each process may be provided by cloud computing. For example, the generation of game images 182 and the distribution of distribution images 164 may be performed by a server rather than by the game device 100 operated by the user. In such a case, since the distribution is based on user operation on the game device 100, it can still be said that "the game device 100 distributes the distribution images 164." The same applies to the execution of game applications and chat applications.

[0124] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0125] 1 System, 4 Network, 10 Communication Data, 100, 100A~100F Game Device, 102, 202 Processor, 104, 204 Memory, 106, 206 Communication Unit, 108, 210 Storage, 112, 212 System Program, 114 Communication Application, 116, 216 Input Unit, 118 Speaker, 120 Display, 122 Microphone, 132 System Module, 140 Communication Execution Unit, 144 Microphone Control Unit, 148 Communication Control Unit, 150 Video Processing Unit, 152 Sound Processing Unit, 154 Sound, 158 Received Sound, 160 Memory Area, 164, 164A~164D Distributed Image, 1641, 1641A~1641C First Distributed Image, 1642, 1642A~1642C Second Distributed Image, 166 Received and streamed image, 168 Received camera image, 170 Viewing source information image, 172 Viewing destination information image, 174 Display when not streaming, 180 Game application, 182, 182A~182D Game video, 184 Game sound, 186 Game execution unit, 200 Management server, 208 Management information, 214 Management program.

Claims

1. An information processing device that participates in a session capable of distributing images based on images of a running game application, and displays session images, The session image includes at least one of the multiple distribution images distributed in the session, The one or more of the distributed images included in the session image are displayed in a first state and / or a second state, An information processing device that displays a first image which includes at least an image showing another information processing device that is displaying at least one of the distributed images distributed by other information processing devices participating in the session in the second state.

2. The first image is one or more images that are displayed in the vicinity of one or more of the distributed images included in the session image, The information processing apparatus according to claim 1, which shows another information processing apparatus that displays the distribution image in the vicinity of the first image in the second state.

3. The session image includes one or more second images generated at predetermined positions, Each of the second images is associated with the information processing device participating in the session, The information processing apparatus according to claim 1 or 2, which includes, at least, an image indicating another information processing apparatus that distributes a distribution image, when the associated information processing apparatus is displaying any of the distribution images in the second state.

4. The information processing apparatus according to claim 3, wherein the second image includes the distributed image if the associated other information processing apparatus is distributing the distributed image in the session.

5. The information processing apparatus according to claim 1 or claim 2, wherein the second state is a state in which the visibility of the distributed image is higher than that of the first state.

6. The information processing apparatus according to claim 5, wherein the state of high visibility includes at least one of a state of large display, a state of high resolution, a state of high frame rate, and a state of high bit rate.

7. The information processing apparatus according to claim 1 or claim 2, wherein the second state is a state in which the sound quality associated with the playback of the distributed image is higher than that of the first state.

8. The information processing apparatus according to claim 1 or claim 2, wherein the number of distributed images displayed in the second state included in the session image is 0 or 1.

9. The information processing apparatus according to claim 1 or 2, wherein the session image includes all of the distributed images distributed in the session.

10. The information processing apparatus according to claim 9, wherein the first image includes at least an image showing all information processing apparatuses participating in the session, each displaying at least one of the distribution images in the second state.

11. The information processing apparatus according to claim 1 or claim 2, wherein the first image includes at least an image showing all information processing apparatuses participating in the session, each displaying at least one of the distribution images in the second state.

12. A method that is performed on a computer, Steps to join a session where streaming images are available, A method comprising the steps of displaying a session image including at least one of a plurality of distributed images displayed in a first state and / or a second state, which are distributed in the session, and an image showing a further information processing device that is causing at least one of the distributed images distributed by another information processing device participating in the session to display in the second state.

13. A program that runs on one or more processors, Steps to join a session where streaming images are available, A program that performs the steps of displaying a session image which includes at least one of a plurality of distributed images displayed in a first state and / or a second state that are distributed in the session, and an image which shows a further information processing device that is displaying at least one of the distributed images distributed by another information processing device participating in the session in the second state.

Images