Information processing system, information processing method, terminal, and program

Abstract

To suppress an unintended situation for a user regarding handling of a camera image.SOLUTION: An information processing system includes: a communication control section for controlling data communication among multiple terminals including a first terminal and a second terminal that belong to the same group; a camera image acquisition section for acquiring a camera image of a camera corresponding to either terminal that belongs to the same group; a restriction setting section for setting restriction of display of a camera image of a camera corresponding to a terminal other than at least the second terminal in the second terminal according to a user's operation; a game application execution section for executing game processing according to a user's operation in the first terminal and outputting a first game image and first information; and a display mode determination section for determining a display mode of a first game image in the second terminal on the basis of the setting of restriction of display and first information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, a terminal, and a program. [Background technology]

[0002] Game processing using camera images is known (for example, Japanese Patent Application Laid-Open No. 2013-048924 (Patent Document 1)). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-048924 Summary of the Invention [Problem to be solved by the invention]

[0004] It is preferable to prevent situations that are not intended by the user from occurring when handling camera images. [Means for solving the problem]

[0005] (Configuration 1) An information processing system according to one embodiment includes a communication control unit that controls data communication between multiple terminals, including a first terminal and a second terminal, that belong to the same group; a camera image acquisition unit that acquires camera images from a camera corresponding to any of the terminals that belong to the same group; a restriction setting unit that sets restrictions on the display of camera images from a camera corresponding to at least one terminal other than the second terminal in the second terminal in accordance with user operation; a game application execution unit that executes game processing in accordance with user operation on the first terminal and outputs a first game image and first information; and a display mode determination unit that determines the display mode of the first game image in the second terminal based on the display restriction setting and the first information.

[0006] According to configuration 1, when a certain terminal has set restrictions on the display of camera images from cameras corresponding to at least terminals other than the terminal, the display mode of game images sent from other terminals can be determined based on the first information, thereby preventing situations that are not intended by the user from occurring.

[0007] (Configuration 2) In Configuration 1, the game application execution unit may be capable of generating a first game image using at least a first camera image of a first camera corresponding to the first terminal. According to Configuration 2, generating a first game image using the first camera image can promote communication through the game.

[0008] (Configuration 3) In configuration 2, the game application execution unit may set a flag when the first camera image is used. The first information may be based on the flag. According to configuration 3, it is possible to reliably determine that the first camera image is being used based on the first information.

[0009] (Configuration 4) In any of configurations 1 to 3, the game application execution unit may be capable of acquiring camera images from cameras corresponding to terminals other than the first terminal that belong to the same group, and generating the first game image using the acquired camera images. According to configuration 4, the game image is generated using camera images from cameras corresponding to the other terminals, which enables communication using camera images, for example, when multiple users play the same online game.

[0010] (Configuration 5) In any of configurations 1 to 4, the information processing system may further include a chat execution unit that executes a chat using camera images between multiple terminals. The restriction setting unit may restrict the use of camera images when the chat is being executed. According to configuration 5, by setting a display restriction, it is possible to restrict the display of camera images even when a chat is being executed.

[0011] (Configuration 6) In configuration 5, the chat execution unit may acquire camera images while the chat is being executed. The game application execution unit may acquire camera images acquired by the chat execution unit. According to configuration 6, the camera images acquired while the chat is being executed can also be used to generate game images.

[0012] (Configuration 7) In configuration 5 or 6, the chat executing unit may display, on at least another terminal among the plurality of terminals, a game image generated by a game process executed in response to a user operation on at least some of the plurality of terminals. According to configuration 7, communication can be performed with users of other terminals based on a game image generated by a game process executed on a certain terminal.

[0013] (Configuration 8) In any of configurations 1 to 7, the first terminal may include a first communication unit and transmit a data stream based on the first game image and the first information to the second terminal. According to configuration 8, the first game image and the first information can be continuously updated in a relatively short cycle.

[0014] (Configuration 9) In configuration 8, when the image data of the first game image includes a flag, the data stream may include data generated by removing the flag from the image data and data indicating the removed flag. According to configuration 8, the image data generated by the game processing can be converted into an appropriate data stream.

[0015] (Configuration 10) In configuration 9, the data stream may be compliant with the H.264 (MPEG-4 AVC) standard. According to configuration 10, data can be transmitted efficiently.

[0016] (Configuration 11) In any of configurations 1 to 10, when a restriction on the display of camera images is set on the second terminal, the display mode determination unit may determine, based on the first information, not to display the first game image including the camera image, or at least to display the first game image in a mode in which the camera image cannot be distinguished. According to configuration 11, when a restriction on the display of camera images is set on a certain terminal, it is possible to prevent a game image including a camera image from being unintentionally displayed.

[0017] (Configuration 12) An information processing method according to another embodiment includes the steps of performing data communication between multiple terminals including a first terminal and a second terminal belonging to the same group, acquiring camera images from a camera corresponding to any of the terminals belonging to the same group, setting display restrictions on the camera images from a camera corresponding to at least a terminal other than the second terminal in accordance with user operation in the second terminal, executing game processing in accordance with user operation in the first terminal and outputting a first game image and first information, and determining the display mode of the first game image in the second terminal based on the display restriction setting and the first information.

[0018] (Configuration 13) A terminal according to yet another embodiment includes a game image acquisition unit that acquires game images and first information output by other terminals belonging to the same group executing game processing; a camera image acquisition unit that acquires camera images from a camera corresponding to any of the terminals belonging to the same group; a restriction setting unit that sets restrictions on the display of camera images from a camera corresponding to at least terminals other than the terminal in the terminal in accordance with user operation; and a display mode determination unit that determines the display mode of game images on the terminal based on the display restriction setting and the first information.

[0019] (Configuration 14) A program according to yet another embodiment causes one or more processors to execute the steps of acquiring game images and first information output by other terminals belonging to the same group executing game processing, acquiring camera images from a camera corresponding to any of the terminals belonging to the same group, setting restrictions on the display of camera images from cameras corresponding to at least terminals other than the terminal in accordance with user operation, and determining the display mode of the game images on the terminal based on the display restriction settings and the first information. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a schematic diagram showing an example of a configuration of an information processing system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of a terminal according to the present embodiment. [Figure 3] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of a management server according to the present embodiment. [Figure 4] FIG. 2 is a schematic diagram showing an example of a software configuration of a terminal according to the present embodiment. [Figure 5] 10 is a flowchart showing an example of game processing in the terminal according to the present embodiment. [Figure 6] 10 is a flowchart showing an example of a data transmission process in the terminal according to the present embodiment. [Figure 7] 10A to 10C are diagrams illustrating an example of processing of game image data and a camera use flag in the terminal according to the present embodiment. [Figure 8] 10 is a flowchart showing an example of a data reception process in the terminal according to the present embodiment. [Figure 9] 10 is a flowchart showing an example of a process for generating a chat screen in the terminal according to the present embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a chat screen generated in the information processing system according to the present embodiment. [Figure 11]10A and 10B are diagrams for illustrating an example of a state in which a camera image cannot be distinguished in the information processing system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] The present embodiment will be described in detail with reference to the drawings, in which the same or corresponding parts are designated by the same reference numerals and description thereof will not be repeated.

[0022] [A. System configuration example] First, a configuration example of an information processing system according to the present embodiment will be described.

[0023] FIG. 1 is a schematic diagram showing an example of the configuration of an information processing system 1 according to the present embodiment. Information processing system 1 includes one or more terminals 100. Terminal 100 is an example of a computer. FIG. 1 shows an example of the configuration of information processing system 1 including four terminals 100 connected to a network 4. In the following description, when it is necessary to identify each of the multiple terminals 100, they are distinguished by adding a subnumber (for example, terminals 100-1, 100-2, 100-3, and 100-4 shown in FIG. 1).

[0024] In the information processing system 1, the terminal 100 can participate in one or more communication groups (hereinafter also abbreviated as "groups"). In order for the terminal 100 to participate in any of the groups, the account of the user who uses the terminal 100 may be used, or the identification information of the terminal 100 may be used.

[0025] A certain terminal 100 may belong to only one group at a time, or may belong to multiple groups at the same time. As an example, the process when belonging to one group will be described below. An upper limit may be set for the number of terminals 100 that can belong to each group.

[0026] Various information can be exchanged between the terminals 100 belonging to the same group. Data exchanged between the terminals 100 belonging to the same group is also called "communication data 10."

[0027] A session corresponding to the group may be established between terminals 100 belonging to the same group. That is, multiple terminals 100 belonging to the same group may have a common session. Communication data 10 may be transmitted and received in the common session.

[0028] 1 shows, as an example, a case where terminals 100-1 to 100-4 belong to the same group. Terminal 100-1 transmits communication data 10-1 to one or more other terminals 100, and receives communication data 10-2 to 10-4 transmitted by one or more other terminals 100. Terminals 100-2 to 100-4 transmit and receive communication data 10 in the same way as terminal 100-1.

[0029] For example, the communication data 10 transmitted by the terminal 100 may include sounds acquired by the terminal 100, camera images acquired by the terminal 100, game images output by game processing executed in the terminal 100, and the like.

[0030] The audio acquired by the terminal 100 may include, for example, the voice of the user using the terminal 100, acquired using a microphone available on the terminal 100. The camera image acquired by the terminal 100 may be, for example, an image captured using a camera 118 available on the terminal 100. Note that the image includes a moving image and one or more still images. The audio may include not only voice but also environmental sounds.

[0031] Users using terminals 100 belonging to the same group can communicate with each other in various ways using the communication data 10. The exchange of communication data 10 between terminals 100 belonging to the same group and related processes will be collectively referred to as "chat" below. For example, chat using voice and camera images may be carried out between terminals belonging to the same group.

[0032] The information processing system 1 may include a management server 200 and a relay server 300. The management server 200 is responsible for processes such as forming a new group and allowing one or more terminals 100 to join a group. The relay server 300 is responsible for processes such as relaying communication data 10 transmitted and received by one or more terminals 100. The management server 200 or the relay server 300 may manage sessions.

[0033] The management server 200 and the relay server 300 may be implemented on a single physical server. The information processing system 1 may include multiple management servers 200, or multiple relay servers 300. The management server 200 and the relay server 300 do not need to have separate functions. The management server 200 and the relay server 300 may be virtual servers.

[0034] The method by which a terminal 100 transmits communication data 10 to one or more other terminals 100 may be a method of transmitting communication data 10 directly to one or more other terminals 100 (e.g., a P2P method) in addition to or instead of a method of transmitting data via one or more relay entities (e.g., relay server 300).

[0035] From the viewpoint of preventing unintentional viewing of images of others and from the viewpoint of protecting minors, the display of camera images may be restricted on terminal 100 when chatting, etc. For example, in a terminal 100 where the display of camera images is restricted, at least the display of camera images from cameras 118 corresponding to terminals 100 other than the terminal in question is restricted. Note that the display of camera images from cameras 118 corresponding to a terminal 100 where the display of camera images is restricted on terminals 100 other than the terminal in question may also be restricted, or may be restricted by a separate setting. Similarly, the display of camera images from cameras 118 corresponding to a terminal 100 where the display of camera images is restricted on the terminal in question may also be restricted.

[0036] Incidentally, there are cases where game images output by game processing include camera images. In such a case, for example, if a certain terminal 100 transmits a game image including a camera image to a terminal 100 on which display restrictions are set, the game image including the camera image may be displayed on the terminal 100 despite the display restrictions being set. The information processing system 1 of this embodiment employs an example of a mechanism for preventing such a situation.

[0037] FIG. 1 shows an example in which a game image 150 output by a game process executed on terminal 100-1 is transmitted (hereinafter also referred to as "distribution") to other terminals 100-2 to 100-4.

[0038] Here, in a situation where a certain terminal 100 distributes images or audio to another terminal 100, the terminal 100 distributing the images or audio is also referred to as a "distributing terminal," and the terminal receiving the images or audio is also referred to as a "receiving terminal." The distinction between a distributing terminal and a receiving terminal may be based on the image or audio of interest. Therefore, each terminal 100 can be both a distributing terminal and a receiving terminal.

[0039] In a chat, a game image 150 generated by a game process executed in response to user operations on at least some of the multiple terminals 100 (distribution terminals) may be displayed on at least other of the multiple terminals 100 (receiving terminals).

[0040] As will be described later, the distribution terminal may output game image data (sometimes referred to as "game images") for displaying game images 150 and predetermined information. These may be output in any manner, but as an example, the predetermined information may be associated with the game image data. The predetermined information may be, for example, information indicating that the game image 150 uses a camera image. Upon receiving the game image data and the predetermined information from the distribution terminal, the receiving terminal may determine the display mode of the game image 150 based on the predetermined information. For example, if a restriction on the display of camera images is set in the receiving terminal, the receiving terminal may restrict the display of game images that include camera images. FIG. 1 shows an example of such a restriction on the display of camera images.

[0041] Game image 150 distributed by terminal 100-1, which is a distribution terminal, includes camera image 152 acquired by terminal 100-1. Camera image 152 may be captured by camera 118 available to terminal 100-1, or may be acquired by another method.

[0042] No restrictions are set on the display of camera images on terminal 100-4, which is the receiving terminal, so that terminal 100-4 displays game image 150 transmitted by terminal 100-1 as is.

[0043] On the other hand, suppose that restrictions are set on the display of camera images on terminals 100-2 and 100-3, which are receiving terminals, and therefore the display mode of game image 150 on terminals 100-2 and 100-3 is different from the display mode on terminals 100-1 and 100-4.

[0044] As an example, in terminal 100-2, the entire game image 150 is changed to a black image so that it is not substantially displayed. Hereinafter, an image such as a black image or a white image, the content of which cannot be substantially discerned, is also referred to as a "mute image." In this way, the game image 150 is displayed in a state in which it has been changed to a mute image 154. A similar display is also performed in terminal 100-3.

[0045] [B. Hardware configuration example] Next, an example of a hardware configuration of each device included in information processing system 1 according to the present embodiment will be described.

[0046] (b1: Terminal 100) 2 is a schematic diagram showing an example of a hardware configuration of terminal 100 according to the present embodiment. Terminal 100 is a type of computer. Referring to FIG. 2, terminal 100 includes, for example, one or more processors 102, one or more memories 104, a communication unit 106, a camera interface 108, storage 110, an input unit 116, a display 120, a microphone 122, and a speaker 124.

[0047] The processor 102 is a processing entity for executing processes in the terminal 100. The processor 102 includes, for example, a central processing unit (CPU) and a graphics processing unit (GPU). The processor 102 loads a program stored in the storage 110 into the memory 104 and executes the program.

[0048] The memory 104 is a volatile storage medium accessible by the processor 102, and includes, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0049] The storage 110 is a non-volatile storage medium accessible by the processor 102, and includes, for example, a flash memory, a hard disk, etc. The storage 110 may also include a storage medium that is detachable from the terminal 100, such as a cartridge or an optical disk.

[0050] The storage 110 stores, for example, a system program 112 and a game application 180 .

[0051] The system program 112 includes computer-readable instructions for controlling the hardware of the terminal 100, providing a program execution environment, etc. The system program 112 may also include computer-readable instructions necessary for executing a chat.

[0052] The game application 180 includes computer-readable instructions for executing a game. Multiple game applications 180 may be installed on the terminal 100.

[0053] In this specification, the term "processor" encompasses at least processing circuits that perform processing according to computer-readable instructions, such as a CPU or GPU, and hardwired circuits, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0054] As used herein, the term “memory” encompasses at least memory 104 and storage 110 .

[0055] The terminal 100 may employ an SoC (System on Chip) that integrates functions such as a processor and memory. In this case, the processor and memory exist in the same SoC (Integrated Circuit). Therefore, in this specification, the terms "processor" and "memory" include configurations that are independent of each other and configurations that are integrated with each other.

[0056] The communication unit 106 transmits and receives data to and from other terminals 100, the management server 200, the relay server 300, etc. via the network 4. The communication unit 106 may communicate data with other terminals 100, etc. via the Internet, which is an example of the network 4. The network 4 may also be a wireless network conforming to Bluetooth (registered trademark), ZigBee (registered trademark), wireless LAN (IEEE 802.11 standard), etc.

[0057] The camera interface 108 mediates data transmission to and reception from the camera 118. The camera interface 108 may be connected to the camera 118 via a wired connection or wireless connection.

[0058] The input unit 116 accepts user operations. The input unit 116 includes, for example, a keyboard, a mouse, a game controller, buttons, a cross key, etc. The input unit 116 may be an interface with a device that accepts user operations (for example, a game controller). In other words, the device that accepts user operations may be located outside the terminal 100.

[0059] The display 120 displays an image or video generated by processing by the processor 102. The display 120 includes, for example, an LCD (Liquid Crystal Display) or an organic EL display. The display 120 may be located outside the terminal 100. In this case, the terminal 100 may include an interface circuit with the display 120.

[0060] The microphone 122 is disposed on the housing of the terminal 100 or the like, and collects sound and generates a sound signal. The speaker 124 is disposed on the housing of the terminal 100 or the like, and generates sound from the sound signal. At least one of the microphone 122 and the speaker 124 may be located outside the terminal 100.

[0061] Camera 118 generates camera image data (moving image data or still image data) of an object in accordance with a command from terminal 100 or in accordance with predetermined conditions.

[0062] The camera 118 includes an imaging element such as a charge-coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor, etc. A lens may be disposed in front of the imaging element.

[0063] The camera 118 may be, for example, a camera for video calls. The camera 118 may be a dedicated camera that can be connected only to a specific device, or a general-purpose camera that can be connected to any device. Multiple cameras 118 may be connectable to the terminal 100.

[0064] The terminal 100 may include a camera. The terminal 100 may not include a microphone or a speaker, which may be connectable to the terminal 100.

[0065] It should be noted that the multiple terminals 100 belonging to the same group do not need to have the same hardware configuration. For example, some of the terminals 100 may be dedicated game consoles, while other terminals 100 may be personal computers, televisions, smartphones, tablets, etc. capable of running games.

[0066] (b2: Management Server 200) 3 is a schematic diagram showing an example of a hardware configuration of management server 200 according to this embodiment. Referring to FIG. 3, management server 200 includes, for example, one or more processors 202, one or more memories 204, one or more communication units 206, storage 210, and input unit 216.

[0067] The processor 202 is a processing entity for executing processes in the management server 200. The processor 202 includes, for example, a CPU, a GPU, etc. The processor 202 loads a program stored in the storage 210 into the memory 204 and executes the program.

[0068] The memory 204 is a volatile storage medium accessible by the processor 202, and includes, for example, DRAM, SRAM, and the like.

[0069] The storage 210 is a non-volatile storage medium accessible by the processor 202, and includes, for example, a hard disk or flash memory. The storage 210 stores, for example, a system program 212 and a management program 214. The storage 210 may also store account information (not shown).

[0070] The system program 212 includes computer-readable instructions for controlling the hardware of the management server 200 and providing a program execution environment.

[0071] The management program 214 includes computer-readable instructions for executing the management-related processes for which the management server 200 is responsible.

[0072] The communication unit 206 transmits and receives data to and from the terminal 100, the management server 200, and the like via the network 4. The communication unit 206 may be connected to the network 4 via a wired connection or may be connected to the network 4 wirelessly.

[0073] The input unit 216 receives user operations and includes, for example, a keyboard and a mouse.

[0074] (b3: Relay Server 300) An example of the hardware configuration of relay server 300 is similar to the example of the hardware configuration of management server 200 shown in FIG. 3, and therefore detailed description thereof will not be repeated.

[0075] The relay program stored in the storage of the relay server 300 may include computer-readable instructions for transmitting, receiving, and relaying the communication data 10.

[0076] [C. Software Configuration Example of Terminal 100] Next, an example of the software configuration of terminal 100 according to the present embodiment will be described.

[0077] 4 is a schematic diagram showing an example of the software configuration of terminal 100 according to the present embodiment. Referring to FIG. 4, terminal 100 includes a group of software modules 128. Software module group 128 may be realized by one or more processors 102 executing system program 112 and game application 180.

[0078] The software module group 128 includes an image processing unit 130, a microphone control unit 132, an audio processing unit 134, a communication control unit 136, a camera control unit 138, a chat execution unit 140, a camera image acquisition unit 142, an audio acquisition unit 144, a game image acquisition unit 146, a chat screen generation unit 148, and a restriction unit 190. The software module group 128 also includes a game application execution unit 182.

[0079] The image processing unit 130 generates a video signal based on the image data and outputs the generated video signal to the display 120 .

[0080] The microphone control unit 132 generates audio data 164 from the audio signal output by the microphone 122. The microphone control unit 132 may be able to change the microphone 122 to a mute state.

[0081] The audio processing unit 134 generates an audio signal based on the audio data and outputs the generated audio signal to the speaker 124 .

[0082] The communication control unit 136 issues commands to the communication unit 106 to control data communication with other terminals 100. When a chat is to be held, the communication control unit 136 cooperates with the chat execution unit 140 to control data communication between multiple terminals 100, including the terminal itself and other terminals 100 that belong to the same group.

[0083] Camera control unit 138 generates camera image data 160 from the video signal output by camera 118. Camera control unit 138 may be able to control the start and stop of imaging by camera 118, and may also be able to control the imaging conditions of camera 118 (frame rate, white balance, etc.).

[0084] The chat executing unit 140 executes a chat. For example, the chat executing unit 140 executes a chat between a plurality of terminals 100 using camera images.

[0085] The camera image acquisition unit 142 acquires camera images from the camera 118 corresponding to any of the terminals 100 belonging to the same group. During chat, the camera image acquisition unit 142 acquires camera image data 160 for displaying the camera image of its own terminal and received camera image data 162 which is camera image data of one or more other terminals 100.

[0086] The voice acquisition unit 144 acquires voice corresponding to any of the terminals 100 belonging to the same group. During chat, the voice acquisition unit 144 acquires voice data 164 generated from voice collected by its own terminal and received voice data 166 from one or more other terminals 100.

[0087] The game image acquisition unit 146 acquires a game image corresponding to any of the terminals 100 belonging to the same group. During chat, the game image acquisition unit 146 acquires game image data 168 indicating a game image output by the game application execution unit 182 of its own terminal, and received game image data 170 from one or more other terminals 100. The game image data 168 may be associated with a camera usage flag 169. The received game image data 170 may be associated with a camera usage flag 171.

[0088] The game image data 168 may be associated with the camera usage flag 169 that is independent of the camera usage flag 169. The game image data 168 may include the camera usage flag 169 as part of the data. The association between the game image data 168 and the camera usage flag 169 may be implemented in any manner. The same applies to the received game image data 170 and the camera usage flag 171.

[0089] The game image data 168 may be the same as the game image data output by the game application execution unit 182 and used for display on the display 120 of the terminal itself, or may be image data that has undergone predetermined image processing (e.g., reduced resolution, etc.).

[0090] The restriction unit 190 sets restrictions on the display of camera images in response to user operations, etc. The display restrictions may be at least a part of so-called parental control. For example, when a specific account attempts to chat, the display of camera images may be restricted by default, and permission from an account supervising that account may be required to display the images. As described above, the display restrictions may include restricting the display of camera images from cameras 118 corresponding to other terminals 100 on the terminal itself. Furthermore, the display restrictions may be set by user operations for each terminal 100 or for each account associated with the terminal 100.

[0091] Display restrictions for each terminal 100 or for each account associated with the terminal 100 may be set or released by the management server 200 or the like. The terminal 100 may obtain whether or not display restrictions are set on its own terminal and the setting status by accessing the management server 200 or the like as necessary. The restriction unit 190 does not necessarily need to receive user operation, and may simply access the management server 200 or the like to obtain whether or not display restrictions are set and the setting status.

[0092] In this way, at least a part of the processing relating to setting and canceling display restrictions may be performed by the management server 200 or an external server.

[0093] Furthermore, the setting and release of display restrictions may be based on the operation of the input unit 116 of the terminal itself, or may be based on the user operating any other terminal. For example, the display restrictions may be set or released by the user operating an external terminal associated with the terminal 100 or an external terminal associated with the account of the user who uses the terminal 100.

[0094] Users may also voluntarily set display restrictions, for example to avoid displaying private images of others.

[0095] The display restriction may be releasable with the consent of a user with predetermined authority. The display restriction may be releasable for each participating group (session). In this case, when the terminal 100 joins a new group, the display restriction is maintained in the set state for the new group. Alternatively, the display restriction may be lifted semi-permanently. In this case, once the display restriction is lifted, it is maintained in the lifted state even if the terminal 100 joins a new group.

[0096] Game application execution unit 182 corresponds to the currently running game application 180. Game application execution unit 182 executes the game in response to user operations. Game application execution unit 182 outputs game image data 168 and audio data 164 in response to the execution of the game. Game image data 168 may represent not only the currently running game screen, but also a menu screen, a setting screen, or the like.

[0097] The game application execution unit 182 may be capable of acquiring at least one of a camera image (camera image data 160) from the camera 118 corresponding to its own terminal and a camera image (received camera image data 162) from the camera 118 corresponding to the other terminal, and generating a game image (game image data 168) using the acquired image data. When generating a game image using a camera image, the game application execution unit 182 may output information indicating that the camera image is used in the game image (information based on the camera usage flag 169).

[0098] When acquiring game images, the game application execution unit 182 may, for example, directly access the camera image acquisition unit 142 or may request the chat execution unit 140 to acquire camera image data. As an example, when the chat execution unit 140 receives a request to acquire camera image data, the chat execution unit 140 may respond to the game application execution unit 182 with identification information, an address, or the like for identifying the requested camera image data. The game application execution unit 182 may acquire the target image data based on the identification information, the address, or the like. In this way, the game application execution unit 182 may acquire camera images acquired by the chat execution unit 140 while a chat is being executed.

[0099] The chat screen generation unit 148 generates a screen for chatting on the own terminal (hereinafter also referred to as a "chat screen") using at least some of the camera image data 160, the received camera image data 162, the game image data 168, and the received game image data 170. The chat screen generation unit 148 may determine the display mode of the game image of another terminal (received game image data 170) on the own terminal based on the display restriction settings and the game image information (camera use flag 171).

[0100] [D. Game Processing Example] Next, an example of game processing in the terminal 100 will be described. The game processing described below may be executed independently of chat. By executing the game processing while chat is being executed, game images generated in the game processing can be distributed to other terminals 100, and camera images of one or more other terminals 100 can be used in the game processing.

[0101] FIG. 5 is a flowchart showing an example of game processing in terminal 100 according to the present embodiment.

[0102] 5, game application execution unit 182 acquires a user operation corresponding to an operation on input unit 116 (step S100), and executes game processing according to the acquired user operation (step S102). Game application execution unit 182 acquires camera image data 160 from camera image acquisition unit 142 (step S104).

[0103] The game application execution unit 182 determines whether or not to include a camera image in the game image according to the execution result of the game processing (step S106). For example, if an opponent player object (e.g., an opponent's car in a racing game) is included in the game image during execution of a communication game, it may determine that the camera image of the opponent should also be included.

[0104] When a camera image is to be included in the game image corresponding to the execution result of the game processing (YES in step S106), the game application execution unit 182 generates a game image corresponding to the execution result of the game processing and the camera image data (step S108). Then, the game application execution unit 182 associates the camera use flag 169, which is set to "1", for example, with the game image data 168 (step S110). In this way, the game application execution unit 182 sets the camera use flag 169 when a camera image is used.

[0105] If the game image corresponding to the execution result of the game processing does not include a camera image (NO in step S106), the game application execution unit 182 generates a game image corresponding to the execution result of the game processing (step S112). At this time, in this embodiment, the camera use flag 169 is not associated with the game image data 168. Note that the camera use flag 169 set to a value other than "1" may be associated with the game image data 168. Note that the camera image data 160 acquired in step S104 may be discarded.

[0106] The processing of steps S100 to S112 is repeated until the execution of the game processing is completed.

[0107] Note that examples of when the game application executing unit 182 sets the camera use flag 169 to "1" are not limited to the above. Other examples include when the game application executing unit 182 acquires camera image data 160 from the camera image acquiring unit 142 regardless of whether a camera image is included in the game image, when a game mode in which a camera image may be included in the game image is always being executed, or when a game application in which a camera image may be included in the game image is always being executed. Note that whether or not the camera use flag 169 set to "1" is to be associated with the game image data 168 may be determined by the game application executing unit 182, or may be set in advance by an application developer for each scene, for example, to set the camera use flag to "1" for a specific game scene.

[0108] [E. Chat processing example] Next, an example of chat processing (data transmission and data reception) by the chat conducting unit 140 will be described.

[0109] FIG. 6 is a flowchart showing an example of data transmission processing in terminal 100 according to the present embodiment.

[0110] Referring to FIG. 6, the chat executing unit 140 acquires the voice data 164 generated from the voice signal collected by the microphone 122 (step S200), and transmits the acquired voice data 164 (step S202).

[0111] The chat executing unit 140 acquires the game image data 168 (step S204), and determines whether the camera use flag 169 is associated with the acquired game image data 168 (step S206).

[0112] If the camera use flag 169 is associated with the game image data 168 (YES in step S206), the chat executing unit 140 transmits the game image data 168 and the camera use flag 169 (step S208).

[0113] If the camera use flag 169 is not associated with the game image data 168 (NO in step S206), the chat executing unit 140 transmits the game image data 168 (step S210).

[0114] In this embodiment, when a restriction on the display of camera images is set on a certain terminal, not only is the display of camera images received by that terminal restricted, but the display of camera images from that terminal on other terminals is also restricted. The chat executing unit 140 determines whether a restriction on the display of camera images is set (step S212). If a restriction on the display of camera images is not set (NO in step S212), the chat executing unit 140 acquires camera image data 160 based on an image captured by the camera 118 (step S214) and transmits the acquired camera image data 160 (step S216). The camera image data 160 may include the entire image captured by the camera 118, or may include only a portion of the image.

[0115] If a restriction on the display of the camera image is set (YES in step S212), the chat executing unit 140 transmits the mute image as the camera image data 160 (step S218). Note that the chat executing unit 140 does not have to transmit the camera image data.

[0116] While the terminal 100 belongs to any group, the processes of steps S200 to S218 are repeated.

[0117] As a variant, if the camera use flag 169 is not associated with the game image data 168 (NO in step S206), the camera use flag 169 set to "0" (or, for example, "FALSE") may be sent to the game image data 168.

[0118] As a modified example, if a restriction on the display of camera images is set (YES in step S212), camera image data 160 may not be transmitted.

[0119] The transmission periods of the audio data 164, the game image data 168, and the camera image data 160 do not have to be the same. At least a portion of the audio data 164, the game image data 168, and the camera image data 160 may be data streams that comply with the H.264 (MPEG-4 AVC) standard, for example.

[0120] When the game image data 168 and the camera use flag 169 are transmitted in a data stream conforming to H.264, the following data processing may be performed.

[0121] Stream data conforming to the H.264 standard is a sequence of data called NAL (Network Abstraction Layer) units. NAL units can be broadly classified into VCL (Video Coding Layer)-NAL units, which contain video information, and non-VCL-NAL units, which do not contain video information. VCL-NAL units may contain either key frames (frames that are not inter-frame compressed) or difference frames (delta frames).

[0122] FIG. 7 is a diagram illustrating an example of processing of game image data 168 and camera use flag 169 in terminal 100 according to the present embodiment.

[0123] Referring to FIG. 7 , as an example of associating the camera-used flag 169 with game image data 168, when the game image data 168 includes the camera-used flag 169, the camera-used flag 169 is removed from the game image data 168, and a VCL-NAL unit 12 including the game image data 168 is generated. The VCL-NAL unit 12 is data to be transmitted, and includes the game image data as a payload 14 in addition to a header 13. A non-VCL-NAL unit 16 indicating the removed camera-used flag 169 is also generated. The non-VCL-NAL unit 16 is also data to be transmitted, and includes the camera-used flag as a payload 18 in addition to a header 17. The VCL-NAL unit 12 and the non-VCL-NAL unit 16 may constitute one frame of data. In this manner, the communication unit 106 of the terminal 100 may transmit a data stream based on the game image and the camera-used flag 169.

[0124] On the other hand, if the game image data 168 does not include the camera use flag 169, a VCL-NAL unit 12 is generated from the raw game image data 168. The VCL-NAL unit 12 is data to be transmitted, and includes game image data as a payload 14 in addition to a header 13. In this case, the VCL-NAL unit 12 constitutes one frame of data.

[0125] If the game image data 168 does not include the camera use flag 169, a non-VCL-NAL unit 16 whose payload 18 is null or "0" may be generated. That is, regardless of whether the game image data 168 includes the camera use flag 169, one frame of data may be composed of the VCL-NAL unit 12 and the non-VCL-NAL unit 16. Also, if the camera use flag 169 associated with the game image data 168 is "0", a non-VCL-NAL unit 16 does not have to be generated.

[0126] FIG. 8 is a flowchart showing an example of data reception processing in terminal 100 according to the present embodiment.

[0127] Referring to FIG. 8, when the chat conducting unit 140 receives the communication data 10 (YES in step S300), it determines the data type included in the communication data 10 (step S302).

[0128] If the communication data 10 includes voice data ("voice data" in step S302), the chat conducting unit 140 extracts the voice data included in the communication data 10 as received voice data 166 (step S304) and outputs the received voice data 166 to the voice acquisition unit 144 (step S306).

[0129] The voice acquisition unit 144 may sequentially output the acquired one or more pieces of received voice data 166 to the voice processing unit 134. As a result, the voices received from one or more terminals 100 are output in a mixed state from the speaker 124. That is, the user of the terminal 100 can hear the voices transmitted from one or more other terminals 100.

[0130] If the communication data 10 includes camera image data ("camera image data" in step S302), the chat executing unit 140 extracts the camera image data included in the communication data 10 as received camera image data 162 (step S308) and outputs it to the camera image acquiring unit 142 (step S310).

[0131] If the communication data 10 includes game image data ("game image data" in step S302), the chat executing unit 140 extracts the game image data included in the communication data 10 as received game image data 170 (step S312). The chat executing unit 140 determines whether the communication data 10 includes a camera use flag (step S314). In this embodiment, it is determined that the data stream related to the received game image includes a non-VCL-NAL unit 16 whose payload includes, for example, "1" as the camera use flag.

[0132] If the camera use flag is included (YES in step S314), the chat executing unit 140 outputs the received game image data 170 and the camera use flag 171 to the game image acquiring unit 146 (step S316). The camera use flag 171 is associated with the received game image data 170.

[0133] If the camera use flag is not included (NO in step S314), the chat executing unit 140 outputs the received game image data 170 to the game image acquiring unit 146 (step S318).

[0134] While the terminal 100 belongs to any group, the processes of steps S300 to S318 are repeated.

[0135] If the audio data or image data is transmitted after being inter-frame compressed, data restoration processing may be performed. Furthermore, the camera image data included in the communication data 10 may be processed and the result may be output as received camera image data 162, and the game image data included in the communication data 10 may be processed and the result may be output as received game image data 170.

[0136] As a variant example, if the camera use flag is not included (NO in step S314), the camera use flag 171 set to "0" (or, for example, "FALSE") may be output to the game image acquisition unit 146 along with the received game image data 170.

[0137] [F. Example of processing in chat screen generation unit 148] Next, an example of the generation of a chat screen by the chat screen generating unit 148 will be described.

[0138] FIG. 9 is a flowchart showing an example of a process for generating a chat screen in terminal 100 according to the present embodiment.

[0139] 9, chat screen generating unit 148 determines the display layout of the chat screen (step S400). The display layout of the chat screen may include information such as the type of image data to be used for display, the display position, the display size, etc.

[0140] The chat screen generating unit 148 acquires image data to be used for display in accordance with the display layout (step S402).

[0141] The chat screen generating unit 148 determines whether or not a restriction on the display of camera images is set (step S404). If a restriction on the display of camera images is not set (NO in step S404), the process proceeds to step S414.

[0142] If a restriction on the display of camera images is set (YES in step S404), the chat screen generating unit 148 determines whether the acquired image data includes camera image data 160 or received camera image data 162 (step S406).

[0143] If the acquired image data includes camera image data 160 or received camera image data 162 (YES in step S406), chat screen generating unit 148 sets camera image data 160 and received camera image data 162 included in the acquired image data not to be used in the chat screen to be generated (step S408). Note that camera image data 160 and received camera image data 162 may be changed to mute image data or discarded. Alternatively, the camera image may be blurred or changed to an image showing a message such as "Display prohibited."

[0144] If the acquired image data does not include camera image data 160 or received camera image data 162 (NO in step S406), the process of step S408 is skipped.

[0145] The chat screen generating unit 148 determines whether the acquired image data includes game image data 168 associated with the camera use flag 169 or received game image data 170 associated with the camera use flag 171 (step S410).

[0146] If the acquired image includes game image data 168 associated with the camera use flag 169 or received game image data 170 associated with the camera use flag 171 (YES in step S410), the chat screen generating unit 148 performs image processing on the game image data 168 associated with the camera use flag 169 and the received game image data 170 associated with the camera use flag 171 so that camera images are not displayed (step S412). The image processing for not displaying camera images will be described later. Note that the camera use flag 171 associated with the received game image data 170 may or may not be the same as the camera use flag associated with the received game image data 170 by the device that transmitted the received game image data 170. In either case, it can be said that the receiving terminal determines the display mode of the received game image data 170 based on the camera image display restriction setting of the receiving terminal and the camera use flag output by the transmitting terminal.

[0147] In this way, the receiving terminal determines the display mode of the received game image data 170 distributed by the distributing terminal based on the camera image display restriction settings and the camera use flag 171. An image corresponding to the determined display mode may be generated by the chat screen generating unit 148 or the image processing unit 130 of the receiving terminal.

[0148] If the acquired image does not include game image data 168 associated with camera use flag 169 and received game image data 170 associated with camera use flag 171 (NO in step S410), the process of step S412 is skipped, and the process of step S414 is executed.

[0149] In step S414, the chat screen generating unit 148 generates a chat screen based on the acquired image data, the changed image data, and / or the image data after image processing (step S414). A video signal corresponding to the generated chat screen is output to the display 120.

[0150] While the display of the chat screen is enabled, the processes of steps S400 to S414 are repeated.

[0151] As a modification, in step S412, rather than performing image processing so that the camera image is not displayed, the received game image data may simply be processed so that it is not displayed, or the received game image data may be discarded.

[0152] As a modified example, if a restriction on the display of camera images is set on the own terminal, acquisition of camera image data 160 by game application execution unit 182 may be restricted. Because such a restriction prevents game application execution unit 182 from actually using camera images, only received game image data 170 may be processed in steps S410 and S412.

[0153] [G. Chat screen example] Next, an example of a chat screen generated in information processing system 1 according to the present embodiment will be described.

[0154] FIG. 10 is a diagram illustrating an example of a chat screen generated in information processing system 1 according to the present embodiment.

[0155] As an example, assume that terminals 100-1 to 100-4 belong to the same group, and each of terminals 100-1 to 100-4 distributes game images. Furthermore, no restrictions on the display of camera images are set for terminals 100-1, 100-3, and 100-4, while restrictions on the display of camera images are set for terminal 100-2. Each of terminals 100-1, 100-3, and 100-4 distributes its own camera images. Furthermore, a communication game is being played between terminal 100-1 and terminal 100-4, and the game images output by terminal 100-1 and terminal 100-4 are camera images corresponding to terminal 100-1 and terminal 100-4, respectively. FIG. 10(A) shows an example of a chat screen for terminal 100-1, and FIG. 10(B) shows an example of a chat screen for terminal 100-2.

[0156] 10(A), a game image 158-1 output by game processing being executed on terminal 100-1 is displayed, and game images 150-2, 150-3, and 150-4 based on received game image data 170 distributed by terminals 100-2, 100-3, and 100-4 are also displayed. Note that game image 150-1 corresponds to the game image data distributed by terminal 100-1 and is the same image as game image 158-1. Note that game image 150-1 and game image 158-1 may have different image quality or frame rate, for example.

[0157] In terminal 100-1, no restrictions are set on the display of camera images, so that game images 150-2 to 150-4 are displayed as is in the received game image data 170 that is delivered, regardless of whether or not they include camera images.

[0158] Furthermore, camera images 156-1, 156-3, and 156-4 delivered by terminals 100-1, 100-3, and 100-4, respectively, are displayed superimposed on game images 150-1, 150-3, and 150-4, respectively.

[0159] The camera images used in the game image (camera images 152-1 and 152-4 in the example of FIG. 10(A)) and the camera images used in the chat (camera images 156-1 and 156-4 in the example of FIG. 10(A)) may be the same image. In this case, both camera images will look the same.

[0160] However, the camera image used in the game image and the camera image used in the chat may look different. For example, the camera image used in the game image may be an image that includes only the user's face, while the camera image used in the chat may be an image of the entire view including the background captured by the camera.

[0161] Furthermore, when a camera image is used in a game image, the camera image may not be superimposed on the game image.

[0162] On the other hand, since restrictions on the display of camera images are set for terminal 100-2, no camera images are distributed, and no camera images are superimposed on game image 150-2. Note that when no camera images are distributed, an alternative icon or a message indicating that no camera images are distributed may be displayed in the position where the camera image would be superimposed.

[0163] 10(B), game image 158-2 output by game processing being executed on terminal 100-2 is displayed, as well as game images 150-1, 150-3, and 150-4 based on received game image data 170 distributed by terminals 100-1, 100-3, and 100-4, respectively. Game image 150-2 corresponds to the game image data distributed by terminal 100-2, and is the same as game image 158-2.

[0164] The game images delivered by terminal 100-1 and terminal 100-4 each include camera images, so on terminal 100-2, which has restrictions set on the display of camera images, muted images are displayed as game image 150-1 and game image 150-4.

[0165] On the other hand, the game image distributed by terminal 100-3 does not include a camera image, and therefore the game image distributed by terminal 100-3 is displayed as game image 150-3. However, unlike game image 150-3 shown in Fig. 10(A), camera image 156-3 is not superimposed on terminal 100-2, on which restrictions on the display of camera images are set. Note that when restrictions on the display of camera images are set, a mute image or a message indicating that restrictions on the display of camera images are set may be displayed in the position where the camera image would be superimposed.

[0166] In addition, in each game image (image distributed by the distribution terminal) on the chat screen, an account image or account name indicating the account of the transmitting terminal may be superimposed on each game image or displayed near each game image, regardless of whether a mute image is being displayed or not.

[0167] In the example of the chat screen described above, an example in which a mute image is displayed as a game image is shown as an example in which the camera image cannot be distinguished, but another example in which the camera image cannot be distinguished is shown below. The image shown in Fig. 11 may be generated in the image processing of step S412 in Fig. 9.

[0168] Fig. 11 is a diagram for explaining an example of a state in which a camera image cannot be distinguished in information processing system 1 according to the present embodiment. Fig. 11 shows another display example of game image 150-1 included in the chat screen of terminal 100-2 shown in Fig. 10(B). .

[0169] As shown in FIG. 11(A), for example, text such as "Camera use is restricted" or an image indicating that camera use is restricted may be displayed.

[0170] As shown in FIG. 11(B), for example, a blurred game image may be displayed. As shown in FIG. 11(C), a game image may be displayed in which the camera image included in the game image has been changed to a muted image. Typically, a receiving terminal cannot identify the position and range of the camera image included in received game image data 170 distributed by a distribution terminal. Therefore, for example, a distribution terminal may transmit two types of image data for displaying game image 150-1 shown in FIG. 10(A) and game image 150-1 shown in FIG. 11(C). The receiving terminal may determine which image data to use for display.

[0171] For example, if the broadcasting terminal determines, based on information from the management server 200 or the receiving terminal, that a restriction on the display of camera images is set on one of the receiving terminals, the broadcasting terminal may transmit two types of image data. Alternatively, the broadcasting terminal may always transmit two types of image data, one for the terminal on which the restriction on the display of camera images is set and one for the terminal on which the restriction is not set, provided that at least a camera image is included in the game image. Note that, instead of the game image shown in FIG. 11(C), a game image in which a camera image is not used may be transmitted.

[0172] Furthermore, the distribution terminal may display a game image including a camera image on its own screen, while transmitting a different game image, such as one that does not include a camera image.

[0173] In addition, if restrictions on the display of camera images are set on the distribution terminal, the distribution terminal may transmit a game image in which the camera image included in the game image has been changed to a mute image (for example, game image 150-1 shown in Figure 11 (C)).

[0174] Alternatively, the distribution terminal may transmit data indicating the position and size of the camera image within the game image, and the receiving terminal may generate a mute image that covers the camera image based on the data and superimpose it on the game image.

[0175] The display mode shown in FIG. 11 is an example, and any display mode may be adopted as long as the camera image cannot be distinguished.

[0176] In this way, when a restriction on the display of camera images is set on a certain terminal 100, it may be determined based on the camera usage flag 171 that game images (received game image data 170) from other terminals 100 will not be displayed, or that the game images will be displayed in such a manner that the included camera images cannot be identified.

[0177] For convenience of explanation, the description has focused on one game image, but the receiving terminal can receive game image data from multiple distribution terminals and display chat screens based on the received game image data in any layout. Therefore, a different display mode may be determined for each received game image data.

[0178] [H. Variations] In the above explanation, an example of processing has been shown in which terminal 100 determines the display mode and generates an image according to the determined display mode, but at least a part of the processing may be handled by management server 200 or relay server 300. For example, if a camera use flag is associated with game image data from one of the distribution terminals and a display restriction is set on the receiving terminal at the distribution destination, relay server 300 may distribute a muted image instead of the received game image data when relaying the game image data.

[0179] In the above description, a configuration in which independent terminals 100 are connected to a network is exemplified, but at least a part of the terminals 100 may be virtual processing entities. For example, at least a part of the terminals 100 may be virtual machines on a cloud. All of the terminals 100 may be virtual machines on a cloud, or at least a part of the functions of the terminals 100 may be implemented using processing resources on the cloud.

[0180] From another perspective, at least some of the image processing unit 130, microphone control unit 132, audio processing unit 134, communication control unit 136, camera control unit 138, chat execution unit 140, camera image acquisition unit 142, audio acquisition unit 144, game image acquisition unit 146, chat screen generation unit 148, and restriction unit 190 may be implemented using processing resources on the cloud. Also, some of the functions of each unit may be implemented using processing resources on the cloud.

[0181] Furthermore, at least a part of the game processing may be executed using an external resource other than the terminal 100, such as a server.

[0182] The process of including a camera image in a game image may not only simply superimpose the camera image on the game image, but may also include game processing in which the camera image acts on an in-game object. For example, the movement of a user's arm in the camera image may be recognized, and a ball may be thrown by the user displayed as a camera image in the game.

[0183] The game image and the predetermined information may be output separately from the distribution terminal. For example, the game image and the camera use flag may be distributed as separate streaming data. Alternatively, the game image data may be distributed as streaming data, while the camera use flag, which is an example of predetermined information, may be transmitted to the server or the receiving terminal only when it is switched.

[0184] The camera use flag may be set not by the game application execution unit 182 (game application 180) but by the system program 112 or the chat execution unit 140. For example, the system program 112 or the chat execution unit 140 may set the camera use flag based on the camera image acquisition unit 142 causing the game application execution unit 182 to acquire camera images.

[0185] Although a camera use flag has been exemplified as an example of predetermined information for determining the display mode of received game image data 170 received by the receiving terminal, any information may be used. The predetermined information may be information that is used together with the setting of restrictions on the display of camera images when determining the display mode of game image 150 on the receiving terminal. For example, the game application execution unit 182 may set the predetermined flag to "1" when using a screenshot image or an image obtained from a video distribution service as a game image. In other words, the predetermined information may be unrelated to whether or not the game image includes or may include a camera image.

[0186] The predetermined information is not limited to a flag, and is not limited to information indicating one of two values, for example.

[0187] It is not necessary for the receiving terminal to determine the display mode of the received game image data 170. For example, the server (the management server 200 or the relay server 300) may determine the display mode of the received game image data 170 for each receiving terminal based on the settings for camera image display restrictions for each receiving terminal.

[0188] The restrictions on the transmission of camera images and the restrictions on the display of received game images may be set separately, or only one of the restrictions may be set.

[0189] The restriction on the display of camera images on the chat screen and the restriction on the transmission of camera images during chat processing / the restriction on the display of camera images superimposed on game images may be set separately.

[0190] Although an example has been described in which the display of camera images is restricted during chat processing (chat screen), the situations in which the display of camera images is restricted are not limited to this. For example, restrictions on the display of camera images in game images may be set independently of the chat function. Also, for example, the display of camera images may be restricted at all times on a terminal. Note that restrictions on the display of camera images may be set regardless of whether the terminal is offline or online.

[0191] In the chat screen example shown in FIG. 10 , terminal 100-1 corresponding to game image 150-1 and terminal 100-4 corresponding to game image 150-4 are chatting and playing the same online game, respectively displaying their own and their opponent's camera images. In this way, when chatting and playing the same online game, camera images may be included in the game image only between users playing the online game. Note that the "same online game" may be determined, for example, based on identification information unique to the game application. For example, when a game application on a receiving terminal acquires camera images, the system may acquire identification information for the game application and also acquire identification information for the game application being executed on the chat partner's terminal. The system may then identify terminals running the same game application and cause the game application running on the receiving terminal to acquire the received camera images received from the terminals. The game application may then display the acquired received camera images within the game image.

[0192] The game application is not limited to the "same online game." Camera images may also be acquired for display from a terminal running an online game of the same series or a predetermined game. For example, the system of the receiving terminal may acquire identification information for a game application running on the distribution terminal, identify identification information related to the game application (or the acquired identification information), and cause the game application running on the receiving terminal to acquire camera images of the distribution terminal running the game application corresponding to the identified identification information. Here, the "associated ID" may be, for example, one or more identification information preset by a system administrator for the identification information of a specific game application, or an ID based on a predetermined rule, such as "IDs that are the same except for the last character." The "one or more identification information preset by the system administrator" may be stored anywhere in the information processing system 1. For example, the "one or more identification information preset by the system administrator" may be stored by the game application execution unit 182 of the receiving terminal, the management server 200, or the game application execution unit 182 of the distribution terminal.

[0193] Furthermore, regardless of whether the game applications running on each terminal are the same or not, as long as the terminals belong to the same group, the camera images of the terminals may be acquired and displayed. For example, the user of terminal 100-1 shown in FIG. 10 may be playing a communication game with a user other than the user who belongs to the group corresponding to the chat, and game image 150-1 may include a camera image of the other user in the game. In this case, game images including a camera image of the other user who does not belong to the group corresponding to the chat may be distributed within the chat. These examples may also be applied to the acquisition of audio within the chat by a game application.

[0194] In the chat screen example shown in FIG. 10, terminal 100-1 corresponding to game image 150-1 and terminal 100-4 corresponding to game image 150-4 are playing the same online game. However, terminal 100-1 may play the game, and terminal 100-4 may transmit operation data (and a camera image) to terminal 100-1 and receive a screen showing the game execution results from terminal 100-1, thereby allowing the game to be played as if terminal 100-4 were playing the game. In this case, game image 150-1 and game image 150-4 may be the same or different. In the latter case, game image 150-1 and game image 150-4 may be generated separately. Furthermore, when terminal 100-4 receives a game image associated with a camera use flag from terminal 100-1, it may associate the camera use flag with the game image and distribute it within the chat.

[0195] Similar processing may be performed on audio. The game application may include audio acquired from the microphone 122 in the game audio. For example, the game application may set a flag when the game audio includes audio acquired from the microphone 122, and the receiving terminal may restrict the output of the game audio based on the flag and the setting of restrictions on microphone use on the receiving terminal.

[0196] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0197] 1 Information processing system, 4 Network, 10 Communication data, 12 VCL-NAL unit, 13, 17 Header, 14, 18 Payload, 16 Non-VCL-NAL unit, 100 Terminal, 102, 202 Processor, 104, 204 Memory, 106, 206 Communication unit, 108 Camera interface, 110, 210 Storage, 112, 212 System program, 116, 216 Input unit, 118 Camera, 120 Display, 122 Microphone, 124 Speaker, 128 Software module group, 130 Image processing unit, 132 Microphone control unit, 134 Audio processing unit, 136 Communication control unit, 138 Camera control unit, 140 Chat execution unit, 142 Camera image acquisition unit, 144 Audio acquisition unit, 146 Game image acquisition unit, 148 Chat screen generation unit, 150, 158 Game image, 152 camera image, 154 mute image, 160 camera image data, 162 received camera image data, 164 audio data, 166 received audio data, 168 game image data, 169, 171 camera usage flag, 170 received game image data, 180 game application, 182 game application execution unit, 190 restriction unit, 200 management server, 214 management program, 300 relay server.

Claims

1. An information processing system, a communication control unit that controls data communication between a plurality of terminals, including a first terminal and a second terminal, that belong to the same group; a camera image acquisition unit that acquires a camera image from a camera corresponding to any one of the terminals that belong to the same group; a restriction setting unit that sets a restriction on display of a camera image in the second terminal corresponding to at least a terminal other than the second terminal in response to a user operation; a game application execution unit that executes game processing in response to a user operation on the first terminal and outputs a first game image and first information; an information processing system comprising: a display mode determination unit that determines a display mode of the first game image on the second terminal based on the display restriction setting and the first information;

2. The information processing system according to claim 1 , wherein the game application execution unit is capable of generating the first game image using at least a first camera image of a first camera corresponding to the first terminal.

3. the game application execution unit sets a flag when the first camera image is used; The information processing system according to claim 2 , wherein the first information is based on the flag.

4. 2. The information processing system of claim 1, wherein the game application execution unit is capable of acquiring camera images from a camera corresponding to a terminal other than the first terminal that belongs to the same group, and is capable of generating the first game image using the acquired camera images.

5. a chat execution unit that executes a chat using camera images between the plurality of terminals; The information processing system according to claim 1 , wherein the restriction setting unit restricts use of the camera image when the chat is being carried out.

6. the chat execution unit acquires the camera image during the execution of the chat, The information processing system according to claim 5 , wherein the game application execution unit acquires the camera image acquired by the chat execution unit.

7. 6. The information processing system according to claim 5, wherein the chat execution unit displays, on at least another terminal of the plurality of terminals, a game image generated by a game process executed in response to a user operation on at least some of the plurality of terminals.

8. The information processing system according to claim 1 , wherein the first terminal includes a first communication unit and transmits a data stream based on the first game image and the first information to a second terminal.

9. 9. The information processing system according to claim 8, wherein, when the image data of the first game image includes a flag, the data stream includes data generated by removing the flag from the image data and data indicating the removed flag.

10. 10. The information processing system according to claim 8, wherein the data stream complies with the H.264 (MPEG-4 AVC) standard.

11. 2. The information processing system according to claim 1, wherein, when a restriction on the display of camera images is set on the second terminal, the display mode determination unit determines, based on the first information, not to display the first game image including the camera image, or to display the first game image in a manner in which the camera image cannot be distinguished.

12. An information processing method, comprising: performing data communication among a plurality of terminals including a first terminal and a second terminal that belong to the same group; acquiring a camera image from a camera corresponding to any one of the terminals belonging to the same group; setting, in the second terminal, a restriction on display of camera images of cameras corresponding to at least terminals other than the second terminal in response to a user operation; executing a game process in response to a user operation on the first terminal and outputting a first game image and first information; and determining a display mode of the first game image on the second terminal based on the setting of the display restriction and the first information.

13. A terminal, a game image acquisition unit that acquires game images and first information output by other terminals that belong to the same group executing game processes; a camera image acquisition unit that acquires a camera image from a camera corresponding to any one of the terminals that belong to the same group; a restriction setting unit that sets a restriction on display of camera images from cameras corresponding to at least one terminal other than the terminal in response to a user operation; a display mode determination unit that determines a display mode of the game image on the terminal based on the display restriction setting and the first information.

14. A program that, when executed by one or more processors, acquiring a game image and first information output by another terminal belonging to the same group executing a game process; acquiring a camera image from a camera corresponding to any of the terminals belonging to the same group; setting, in the terminal, a restriction on display of camera images from cameras corresponding to at least terminals other than the terminal in response to a user operation; and determining a display mode of the game image on the terminal based on the setting of the display restriction and the first information.

Images