Information processing systems, information processing methods, terminals, and programs

Abstract

To suppress an unintended situation for a user regarding handling of a camera image.SOLUTION: An information processing system includes: a communication control section for controlling data communication among multiple terminals including a first terminal and a second terminal that belong to the same group; a camera image acquisition section for acquiring a camera image of a camera corresponding to either terminal that belongs to the same group; a restriction setting section for setting restriction of display of a camera image of a camera corresponding to a terminal other than at least the second terminal in the second terminal according to a user's operation; a game application execution section for executing game processing according to a user's operation in the first terminal and outputting a first game image and first information; and a display mode determination section for determining a display mode of a first game image in the second terminal on the basis of the setting of restriction of display and first information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing system, an information processing method, a terminal, and a program.

Background Art

[0002] Game processing using a camera image is known (for example, Japanese Patent Application Laid-Open No. 2013-048924 (Patent Document 1)).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Regarding the handling of camera images, it is preferable to suppress the occurrence of situations unintended by the user.

Means for Solving the Problems

[0005] (Configuration 1) An information processing system according to an embodiment includes a communication control unit that controls data communication between a plurality of terminals including a first terminal and a second terminal belonging to the same group, a camera image acquisition unit that acquires a camera image of a camera corresponding to any one of the terminals belonging to the same group, a restriction setting unit that sets a restriction on the display of a camera image of a camera corresponding to at least a terminal other than the second terminal in the second terminal in response to a user operation, a game application execution unit that executes game processing in response to a user operation on the first terminal and outputs a first game image and first information, and a display mode determination unit that determines a display mode of the first game image in the second terminal based on the setting of the display restriction and the first information.

[0006] According to Configuration 1, if a terminal has set restrictions on the display of camera images from cameras compatible with terminals other than that terminal, the display method of game images transmitted from other terminals can be determined based on the first information, thereby preventing situations that the user did not intend.

[0007] (Configuration 2) In Configuration 1, the game application execution unit may be capable of generating a first game image using a first camera image from at least a first camera corresponding to a first terminal. According to Configuration 2, generating a first game image using a first camera image can facilitate communication through the game.

[0008] (Configuration 3) In Configuration 2, the game application execution unit may set a flag when using the first camera image. The first information may be based on the flag. According to Configuration 3, it is possible to reliably determine that the first camera image is being used based on the first information.

[0009] (Configuration 4) In any of Configurations 1 to 3, the game application execution unit may be able to acquire camera images from cameras corresponding to terminals other than the first terminal that belong to the same group, and generate the first game image using the acquired camera images. According to Configuration 4, since the game image is generated using camera images from cameras corresponding to other terminals, communication using camera images becomes possible, for example, when multiple users play the same communication game.

[0010] (Configuration 5) In any of Configurations 1 to 4, the information processing system may further include a chat execution unit that performs chat using camera images between multiple terminals. The restriction setting unit may restrict the use of camera images when the chat is being executed. According to Configuration 5, by setting display restrictions, the display of camera images and the like can be restricted even when the chat is being executed.

[0011] (Configuration 6) In Configuration 5, the chat execution unit may acquire camera images while the chat is running. The game application execution unit may acquire the camera images acquired by the chat execution unit. According to Configuration 6, camera images acquired while the chat is running can also be used to generate game images.

[0012] (Configuration 7) In Configuration 5 or 6, the chat execution unit may display game images generated by game processing performed in response to user operations on at least some of the multiple terminals on at least other terminals among the multiple terminals. According to Configuration 7, communication can be made with users on other terminals based on game images generated by game processing performed on one terminal.

[0013] (Configuration 8) In any of Configurations 1 to 7, the first terminal may include a first communication unit and transmit a data stream based on the first game image and first information to the second terminal. According to Configuration 8, the first game image and first information can be updated continuously at relatively short intervals.

[0014] (Configuration 9) In Configuration 8, when the image data of the first game image includes a flag, the data stream may include data generated by removing the flag from the image data and data indicating the removed flag. According to Configuration 8, the image data generated by the game processing can be converted into an appropriate data stream.

[0015] (Configuration 10) In Configuration 9, the data stream may conform to the H.264 (MPEG-4 AVC) standard. According to Configuration 10, data can be transmitted efficiently.

[0016] (Configuration 11) In any of Configurations 1 to 10, the display mode determination unit may, when a restriction on the display of camera images is set on the second terminal, decide, based on the first information, not to display the first game image including the camera image, or to display the first game image in a manner in which the camera image cannot be identified. According to Configuration 11, when a restriction on the display of camera images is set on a terminal, it is possible to suppress the unintentional display of a game image including the camera image.

[0017] (Configuration 12) An information processing method according to another embodiment includes the steps of: performing data communication between a plurality of terminals, including a first terminal and a second terminal, that belong to the same group; acquiring a camera image of a camera corresponding to any terminal belonging to the same group; setting restrictions on the display of camera images of at least terminals other than the second terminal in the second terminal in response to user operation; executing game processing in response to user operation in the first terminal and outputting a first game image and first information; and determining the display mode of the first game image in the second terminal based on the setting of display restrictions and the first information.

[0018] (Configuration 13) A terminal according to yet another embodiment includes a game image acquisition unit that acquires game images and first information output when other terminals belonging to the same group execute game processing, a camera image acquisition unit that acquires camera images from a camera corresponding to any terminal belonging to the same group, a restriction setting unit that sets restrictions on the display of camera images of at least terminals other than the terminal on the terminal in response to user operation, and a display mode determination unit that determines the display mode of the game images on the terminal based on the display restriction settings and the first information.

[0019] (Configuration 14) A program according to yet another embodiment causes one or more processors to perform steps of: acquiring a game image and first information output by other terminals belonging to the same group by executing game processing; acquiring a camera image of a camera corresponding to any one of the terminals belonging to the same group; setting a restriction on display of the camera image of at least a camera corresponding to a terminal other than the terminal itself in the terminal according to a user operation; and determining a display mode of the game image in the terminal based on the setting of the display restriction and the first information.

Brief Description of Drawings

[0020] [Figure 1] It is a schematic diagram showing a configuration example of an information processing system according to this embodiment. [Figure 2] It is a schematic diagram showing a hardware configuration example of a terminal according to this embodiment. [Figure 3] It is a schematic diagram showing a hardware configuration example of a management server according to this embodiment. [Figure 4] It is a schematic diagram showing a software configuration example of a terminal according to this embodiment. [Figure 5] It is a flowchart showing a processing example of game processing in a terminal according to this embodiment. [Figure 6] It is a flowchart showing a processing example of data transmission in a terminal according to this embodiment. [Figure 7] It is a diagram for explaining a processing example of game image data and a camera use flag in a terminal according to this embodiment. [Figure 8] It is a flowchart showing a processing example of data reception in a terminal according to this embodiment. [Figure 9] It is a flowchart showing a processing example of generation processing of a chat screen in a terminal according to this embodiment. [Figure 10] It is a diagram for explaining an example of a chat screen generated in an information processing system according to this embodiment. [Figure 11]This figure illustrates an example of a configuration in which a camera image cannot be identified in an information processing system according to this embodiment. [Modes for carrying out the invention]

[0021] This embodiment will be described in detail with reference to the drawings. Note that identical or corresponding parts in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[0022] [A. Example System Configuration] First, an example of the configuration of an information processing system according to this embodiment will be described.

[0023] Figure 1 is a schematic diagram showing an example configuration of an information processing system 1 according to this embodiment. The information processing system 1 includes one or more terminals 100. A terminal 100 is an example of a computer. Figure 1 shows an example configuration of the information processing system 1 including four terminals 100 connected to a network 4. In the following description, when it is necessary to identify each of the multiple terminals 100, they will be distinguished by sub-numbers (for example, terminals 100-1, 100-2, 100-3, 100-4 as shown in Figure 1).

[0024] In the information processing system 1, terminal 100 can join one or more communication groups (hereinafter also referred to as "groups"). In order for terminal 100 to join any group, it may use the account of the user using terminal 100, or it may use the identification information of terminal 100, etc.

[0025] A terminal 100 may belong to only one group at a time, or it may belong to multiple groups at the same time. The following describes the process when a terminal belongs to only one group as an example. There may be an upper limit on the number of terminals 100 that can belong to each group.

[0026] Various types of information can be exchanged between terminals 100 belonging to the same group. The data exchanged between terminals 100 belonging to the same group is also referred to as "communication data 10".

[0027] A session corresponding to the group may be established among terminals 100 belonging to the same group. That is, multiple terminals 100 belonging to the same group may have a common session. Communication data 10 may be sent and received in the common session.

[0028] Figure 1 shows an example where terminals 100-1 to 100-4 belong to the same group. Terminal 100-1 transmits communication data 10-1 to one or more other terminals 100 and receives communication data 10-2 to 10-4 transmitted by one or more other terminals 100. Terminals 100-2 to 100-4 send and receive communication data 10 in the same way as terminal 100-1.

[0029] For example, the communication data 10 transmitted by terminal 100 may include sounds acquired by terminal 100, camera images acquired by terminal 100, and game images output by game processing performed on terminal 100.

[0030] The audio acquired by terminal 100 may include, for example, the voice of the user of terminal 100, acquired using a microphone available on terminal 100. The camera images acquired by terminal 100 may be, for example, images captured using a camera 118 available on terminal 100. The images may include moving images and one or more still images. The audio may include not only voices but also ambient sounds.

[0031] Users using terminals 100 belonging to the same group can communicate in various ways using communication data 10. The exchange of communication data 10 and related processing between terminals 100 belonging to the same group will be collectively referred to as "chat" below. For example, chat using voice or camera images may be performed between terminals belonging to the same group.

[0032] The information processing system 1 may include a management server 200 and a relay server 300. The management server 200 is responsible for processes such as creating new groups and processing for one or more terminals 100 to join a group. The relay server 300 is responsible for processes such as relaying communication data 10 sent and received by one or more terminals 100. The management server 200 or the relay server 300 may manage sessions.

[0033] The management server 200 and the relay server 300 may be implemented on a single physical server. The information processing system 1 may include multiple management servers 200 or multiple relay servers 300. The management server 200 and the relay server 300 do not need to have separate functions. The management server 200 and the relay server 300 may be virtual servers.

[0034] The method by which one terminal 100 transmits communication data 10 to one or more other terminals 100 may be, in addition to, a method of transmitting data via one or more relay entities (e.g., relay servers 300), or alternatively, a method of directly transmitting communication data 10 to one or more other terminals 100 (e.g., a P2P method).

[0035] From the perspective of preventing unintentional viewing of other people's images and protecting minors, the display of camera images may be restricted on terminal 100 when chatting or performing other actions. For example, on terminal 100 where the display of camera images is restricted, the display of camera images from cameras 118 corresponding to other terminals 100 is restricted at least. Furthermore, the display of camera images from cameras 118 corresponding to terminal 100 where the display of camera images is restricted may also be restricted on other terminals 100, or it may be restricted by separate settings. Similarly, the display of camera images from cameras 118 corresponding to terminal 100 where the display of camera images is restricted may also be restricted on that terminal.

[0036] Incidentally, game images output by game processing may include camera images. In such cases, for example, if terminal 100 transmits a game image including a camera image to terminal 100 which has display restrictions set, there is a possibility that the game image including the camera image will be displayed on terminal 100 despite the display restrictions being set. The information processing system 1 of this embodiment employs an example of a mechanism to suppress such a situation.

[0037] Figure 1 shows an example in which a game image 150 output by game processing performed on terminal 100-1 is transmitted (hereinafter also referred to as "distribution") to other terminals 100-2 to 100-4.

[0038] Here, in a scenario where one terminal 100 distributes images or audio to another terminal 100, the terminal 100 distributing the images or audio is also referred to as the "distributing terminal," and the terminal receiving the images or audio is also referred to as the "receiving terminal." The distinction between the distributing terminal and the receiving terminal may be based on the images or audio of interest. Therefore, each of the terminals 100 can be both a distributing terminal and a receiving terminal.

[0039] In a chat, game images 150 generated by game processing performed in response to user operations on at least some of the multiple terminals 100 (distributing terminals) may be displayed on at least the other terminals 100 (receiving terminals) among the multiple terminals 100.

[0040] As described later, the distribution terminal may output game image data (sometimes called "game image") for displaying the game image 150, along with predetermined information. These may be output in any way, but as an example, the predetermined information may be associated with the game image data. The predetermined information may be, for example, information indicating that the game image 150 is using a camera image. When the receiving terminal receives the game image data and the predetermined information from the distribution terminal, it may determine the display mode of the game image 150 based on the predetermined information. For example, if a restriction on the display of camera images is set in the receiving terminal, the receiving terminal may restrict the display of game images that include camera images. Figure 1 shows an example of such a restriction on the display of camera images.

[0041] The game image 150 distributed by terminal 100-1, which is a distribution terminal, includes a camera image 152 acquired by terminal 100-1. The camera image 152 may be captured by a camera 118 available on terminal 100-1, or it may be acquired by another method.

[0042] On terminal 100-4, which is the receiving terminal, no restrictions are set on the display of camera images. Therefore, on terminal 100-4, the game image 150 transmitted by terminal 100-1 is displayed as is.

[0043] On the other hand, terminals 100-2 and 100-3, which are receiving terminals, have restrictions set on the display of camera images. Therefore, the display mode of the game image 150 in terminals 100-2 and 100-3 is different from the display mode of terminals 100-1 and 100-4.

[0044] For example, on terminal 100-2, the entire game image 150 is changed to a black image so that it is not effectively displayed. Below, images that are essentially indistinguishable in content, such as black images and white images, will also be referred to as "muted images." In this way, the game image 150 is displayed in a form that has been changed to a muted image 154. The same display is performed on terminal 100-3.

[0045] [B. Hardware Configuration Example] Next, an example of the hardware configuration of each device included in the information processing system 1 according to this embodiment will be described.

[0046] (b1: Terminal 100) Figure 2 is a schematic diagram showing an example of the hardware configuration of a terminal 100 according to this embodiment. The terminal 100 is a type of computer. Referring to Figure 2, the terminal 100 includes, for example, one or more processors 102, one or more memories 104, a communication unit 106, a camera interface 108, storage 110, an input unit 116, a display 120, a microphone 122, and a speaker 124.

[0047] The processor 102 is the processing unit for executing processes on the terminal 100. The processor 102 includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 102 loads programs stored in the storage 110 into the memory 104 and executes them.

[0048] The memory 104 is a volatile storage medium accessible by the processor 102, and includes, for example, DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory).

[0049] The storage 110 is a non-volatile storage medium accessible by the processor 102, and includes, for example, flash memory or a hard disk. The storage 110 may also include, for example, a storage medium that can be attached to or removed from the terminal 100, such as a cartridge or an optical disc.

[0050] The storage 110 stores, for example, a system program 112 and a game application 180.

[0051] The system program 112 includes computer-readable instructions for hardware control of the terminal 100 and providing a program execution environment. The system program 112 may also include computer-readable instructions necessary for executing the chat.

[0052] The game application 180 includes computer-readable instructions for running the game. Multiple game applications 180 may be installed on the terminal 100.

[0053] In this specification, the term "processor" includes, at a minimum, processing circuits that perform processing according to computer-readable instructions, such as CPUs and GPUs, and hardwired circuits, such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays).

[0054] In this specification, the term “memory” includes at least the memory 104 and the storage 110.

[0055] In terminal 100, a System on Chip (SoC) in which functions such as the processor and memory are integrated may be used. In this case, the processor and memory reside on the same SoC (integrated circuit). Therefore, in this specification, the terms "processor" and "memory" include both independent configurations and integrated configurations.

[0056] The communication unit 106 sends and receives data to and from other terminals 100, a management server 200, a relay server 300, etc., via the network 4. As an example of the network 4, the communication unit 106 may communicate data with other terminals 100, etc., via the Internet. The network 4 may also be a wireless network conforming to Bluetooth®, ZigBee®, wireless LAN (IEEE 802.11 standard), etc.

[0057] The camera interface 108 mediates the transmission of data to and from the camera 118. The camera interface 108 may be connected to the camera 118 by a wire or by wireless connection.

[0058] The input unit 116 accepts user input. The input unit 116 includes, for example, a keyboard, mouse, game controller, buttons, a directional pad, etc. The input unit 116 may also be an interface with a device that accepts user input (for example, a game controller). That is, the device that accepts user input may be located outside the terminal 100.

[0059] The display 120 displays images or videos generated by processing by the processor 102. The display 120 includes, for example, an LCD (Liquid Crystal Display) or an organic EL display. The display 120 may be located outside the terminal 100. In this case, the terminal 100 may include an interface circuit to the display 120.

[0060] The microphone 122 is located on the casing of the terminal 100 and collects sound to generate an audio signal. The speaker 124 is located on the casing of the terminal 100 and generates sound from the audio signal. At least one of the microphone 122 and speaker 124 may be located outside the terminal 100.

[0061] The camera 118 generates camera image data (moving image data or still image data) of the subject in accordance with commands from the terminal 100 or according to predetermined conditions.

[0062] The camera 118 includes an image sensor, such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. A lens may be placed in front of the image sensor.

[0063] Camera 118 may be, for example, a camera for video calls. Camera 118 may be a dedicated camera that can only be connected to a specific device, or it may be a general-purpose camera that can be connected to any device. Terminal 100 may be able to connect to multiple cameras 118.

[0064] Terminal 100 may include a camera. Terminal 100 does not include a microphone or speaker, but these may be connectable to terminal 100.

[0065] Furthermore, multiple terminals 100 belonging to the same group do not necessarily have to have the same hardware configuration. For example, some terminals 100 may be dedicated gaming machines, while others 100 may be personal computers, televisions, smartphones, tablets, etc., capable of running games.

[0066] (b2: Management Server 200) Figure 3 is a schematic diagram showing an example of the hardware configuration of a management server 200 according to this embodiment. Referring to Figure 3, the management server 200 includes, for example, one or more processors 202, one or more memories 204, one or more communication units 206, storage 210, and an input unit 216.

[0067] Processor 202 is the processing unit for executing processes in the management server 200. Processor 202 includes, for example, a CPU and a GPU. Processor 202 loads programs stored in storage 210 into memory 204 and executes them.

[0068] Memory 204 is a volatile storage medium accessible by the processor 202, and includes, for example, DRAM or SRAM.

[0069] Storage 210 is a non-volatile storage medium accessible by the processor 202, and includes, for example, a hard disk or flash memory. Storage 210 stores, for example, system programs 212 and management programs 214. Storage 210 may also store account information (not shown).

[0070] The system program 212 includes computer-readable instructions for hardware control of the management server 200 and providing a program execution environment.

[0071] The management program 214 includes computer-readable instructions for executing management-related processes that are handled by the management server 200.

[0072] The communication unit 206 sends and receives data to and from terminals 100, management servers 200, etc., via the network 4. The communication unit 206 may be connected to the network 4 by wire or by wireless connection.

[0073] The input unit 216 accepts user input. The input unit 216 includes, for example, a keyboard or mouse.

[0074] (b3: Relay server 300) The hardware configuration example for the relay server 300 is the same as the hardware configuration example for the management server 200 shown in Figure 3, so a detailed explanation will not be repeated.

[0075] The relay program stored in the storage of the relay server 300 may include computer-readable instructions for sending, receiving, and relaying communication data 10.

[0076] [C. Example software configuration for terminal 100] Next, an example of the software configuration of terminal 100 according to this embodiment will be described.

[0077] Figure 4 is a schematic diagram showing an example of the software configuration of terminal 100 according to this embodiment. Referring to Figure 4, terminal 100 includes a group of software modules 128. The group of software modules 128 may be realized by one or more processors 102 executing a system program 112 and a game application 180.

[0078] The software module group 128 includes an image processing unit 130, a microphone control unit 132, an audio processing unit 134, a communication control unit 136, a camera control unit 138, a chat execution unit 140, a camera image acquisition unit 142, an audio acquisition unit 144, a game image acquisition unit 146, a chat screen generation unit 148, and a restriction unit 190. The software module group 128 also includes a game application execution unit 182.

[0079] The image processing unit 130 generates a video signal based on the image data and outputs the generated video signal to the display 120.

[0080] The microphone control unit 132 generates audio data 164 from the audio signal output by the microphone 122. The microphone control unit 132 may also mute the microphone 122.

[0081] The audio processing unit 134 generates an audio signal based on the audio data and outputs the generated audio signal to the speaker 124.

[0082] The communication control unit 136 issues commands to the communication unit 106 to control data communication with other terminals 100, etc. When a chat is executed, the communication control unit 136 works in cooperation with the chat execution unit 140 to control data communication between multiple terminals 100, including its own terminal and other terminals 100 belonging to the same group.

[0083] The camera control unit 138 generates camera image data 160 from the video signal output by the camera 118. The camera control unit 138 may also control the start and stop of image capture by the camera 118, and may also control the image capture conditions of the camera 118 (such as frame rate and white balance).

[0084] The chat execution unit 140 executes a chat. For example, the chat execution unit 140 executes a chat using camera images between multiple terminals 100.

[0085] The camera image acquisition unit 142 acquires camera images from the camera 118 corresponding to any terminal 100 belonging to the same group. While the chat is running, the camera image acquisition unit 142 acquires camera image data 160 for displaying the camera image of its own terminal, and received camera image data 162, which is camera image data from one or more other terminals 100.

[0086] The voice acquisition unit 144 acquires voice data corresponding to any terminal 100 belonging to the same group. During the execution of a chat, the voice acquisition unit 144 acquires voice data 164 generated from the voice collected by its own terminal and received voice data 166 from one or more other terminals 100.

[0087] The game image acquisition unit 146 acquires game images corresponding to any terminal 100 belonging to the same group. While chat is running, the game image acquisition unit 146 acquires game image data 168 that shows the game image output by the game application execution unit 182 of its own terminal, and received game image data 170 from one or more other terminals 100. The game image data 168 may also be associated with a camera usage flag 169. The received game image data 170 may also be associated with a camera usage flag 171.

[0088] The game image data 168 may be associated with a camera usage flag 169 that is independent of the camera usage flag 169. The game image data 168 may include the camera usage flag 169 as part of the data. The association between the game image data 168 and the camera usage flag 169 can be implemented in any way. The same applies to the received game image data 170 and the camera usage flag 171.

[0089] The game image data 168 may be the same as the game image data output by the game application execution unit 182 and used for display on the terminal's display 120, or it may be image data that has undergone a predetermined image processing (for example, reduction in resolution).

[0090] The restriction unit 190 sets restrictions on the display of camera images in response to user operations, etc. The display restrictions may be at least part of so-called parental controls. For example, when a particular account attempts to initiate a chat, the display of camera images may be restricted by default, and permission from an account supervising that account may be required to display them. As described above, the display restrictions may also include restricting the display of camera images from cameras 118 corresponding to other terminals 100 on the own terminal. Furthermore, the display restrictions may be set by user operation for each terminal 100 or for each account associated with terminal 100.

[0091] Display restrictions for each terminal 100 or for each account associated with terminal 100 may be set or removed by the management server 200, etc. Terminal 100 may, as needed, access the management server 200, etc. to obtain whether or not display restrictions are set on its terminal and their setting status. The restriction unit 190 does not necessarily need to receive user input; it only needs to access the management server 200, etc. to obtain whether or not display restrictions are set and their setting status.

[0092] Thus, at least part of the processing related to setting and removing display restrictions may be performed by the management server 200 or an external server.

[0093] Furthermore, setting and removing display restrictions may be based on operations on the input unit 116 of the terminal itself, or on the user operating any other terminal. For example, the user may set or remove display restrictions by operating an external terminal associated with terminal 100, or an external terminal associated with the account of the user using terminal 100.

[0094] Furthermore, users may voluntarily set display restrictions, for example, to avoid viewing other people's private images.

[0095] Display restrictions may be lifted with the consent of a user with predetermined permissions. Display restrictions may also be lifted on a group (session) basis. In this case, when terminal 100 joins a new group, the display restrictions will remain in place in that new group. Alternatively, the lifting of display restrictions may be semi-permanent. In this case, once lifted, the display restrictions will remain lifted even if terminal 100 joins a new group.

[0096] The game application execution unit 182 corresponds to the running game application 180. The game application execution unit 182 executes the game in response to user input. The game application execution unit 182 outputs game image data 168 and audio data 164 in response to game execution. The game image data 168 may represent not only the running game screen, but also menu screens, settings screens, etc.

[0097] The game application execution unit 182 may acquire at least one of the camera images (camera image data 160) from the camera 118 corresponding to its own terminal and the camera images (received camera image data 162) from the camera 118 corresponding to another terminal, and generate game images (game image data 168) using the acquired image data. When the game application execution unit 182 generates game images using camera images, it may output information (information based on the camera usage flag 169) indicating that camera images are being used in the game images.

[0098] When the game application execution unit 182 needs to acquire game images, it may, for example, directly access the camera image acquisition unit 142, or it may request the chat execution unit 140 to acquire camera image data. For example, when the chat execution unit 140 receives a request to acquire camera image data, it may respond to the game application execution unit 182 with identification information or an address to identify the requested camera image data. The game application execution unit 182 may then acquire the target image data based on the identification information or address. In this way, the game application execution unit 182 may acquire camera images acquired by the chat execution unit 140 while the chat is running.

[0099] The chat screen generation unit 148 generates a chat screen (hereinafter also referred to as the "chat screen") on its own terminal using at least a portion of the camera image data 160, the received camera image data 162, the game image data 168, and the received game image data 170. The chat screen generation unit 148 may also determine how to display the game images (received game image data 170) from other terminals on its own terminal based on the display restriction settings and the game image information (camera usage flag 171).

[0100] [D. Game Processing Example] Next, an example of game processing on terminal 100 will be described. The game processing described below may be executed independently of chat. By executing game processing while chat is running, the game images generated in the game processing can be distributed to other terminals 100, and the camera images of one or more other terminals 100 can be used in the game processing.

[0101] Figure 5 is a flowchart showing an example of game processing in terminal 100 according to this embodiment.

[0102] Referring to Figure 5, the game application execution unit 182 acquires user operations corresponding to operations on the input unit 116 (step S100), and executes game processing according to the acquired user operations (step S102). The game application execution unit 182 acquires camera image data 160 from the camera image acquisition unit 142 (step S104).

[0103] The game application execution unit 182 determines whether or not to include the camera image in the game image according to the execution result of the game processing (step S106). For example, if the opponent's player object (for example, the opponent's car in a racing game) is included in the game image during the execution of a communication game, it may be determined to also include the camera image of that opponent.

[0104] If the game image, which is based on the execution result of the game processing, is to include the camera image (YES in step S106), the game application execution unit 182 generates a game image corresponding to the execution result of the game processing and the camera image data (step S108). Then, the game application execution unit 182 associates the camera usage flag 169, which is set to, for example, "1", with the game image data 168 (step S110). In this way, the game application execution unit 182 sets the camera usage flag 169 when using the camera image.

[0105] If the game image corresponding to the execution result of the game processing does not include the camera image (NO in step S106), the game application execution unit 182 generates a game image corresponding to the execution result of the game processing (step S112). In this embodiment, the camera usage flag 169 is not associated with the game image data 168. However, the camera usage flag 169 set to a value other than "1" may be associated with the game image data 168. The camera image data 160 acquired in step S104 may be discarded.

[0106] The process from steps S100 to S112 is repeated until the game processing is completed.

[0107] The above is not the only example of when the game application execution unit 182 sets the camera usage flag 169 to "1". Other examples include when the game application execution unit 182 acquires camera image data 160 from the camera image acquisition unit 142, regardless of whether the game image includes camera images or not, or at all times while a game mode that may include camera images in the game image is running, or at all times while a game application that may include camera images in the game image is running. Whether or not to associate the camera usage flag 169 set to "1" with the game image data 168 may be determined by the game application execution unit 182, or it may be pre-configured by the application developer for each scene, for example, to set the camera usage flag to "1" in the case of a specific game scene.

[0108] [E. Chat Processing Example] Next, we will explain an example of chat processing (data transmission and data reception) by the chat execution unit 140.

[0109] Figure 6 is a flowchart showing an example of data transmission processing in terminal 100 according to this embodiment.

[0110] Referring to Figure 6, the chat execution unit 140 acquires audio data 164 generated from the audio signal collected by the microphone 122 (step S200), and transmits the acquired audio data 164 (step S202).

[0111] The chat execution unit 140 acquires game image data 168 (step S204) and determines whether or not a camera usage flag 169 is associated with the acquired game image data 168 (step S206).

[0112] If the game image data 168 is associated with a camera usage flag 169 (YES in step S206), the chat execution unit 140 sends the game image data 168 and the camera usage flag 169 (step S208).

[0113] If the game image data 168 is not associated with a camera usage flag 169 (NO in step S206), the chat execution unit 140 sends the game image data 168 (step S210).

[0114] In this embodiment, when a restriction on the display of camera images is set on a terminal, not only is the display of camera images received by that terminal restricted, but the display of the camera images of that terminal on other terminals is also restricted. The chat execution unit 140 determines whether or not a restriction on the display of camera images is set (step S212). If no restriction on the display of camera images is set (NO in step S212), the chat execution unit 140 acquires camera image data 160 based on the image captured by the camera 118 (step S214) and transmits the acquired camera image data 160 (step S216). The camera image data 160 may include the entire image captured by the camera 118, or it may include only a part of the image.

[0115] If restrictions on displaying camera images are set (YES in step S212), the chat execution unit 140 sends the muted image as camera image data 160 (step S218). However, the chat execution unit 140 does not have to send camera image data.

[0116] As long as terminal 100 belongs to any group, the process in steps S200 to S218 is repeated.

[0117] As a variation, if the game image data 168 does not have a camera usage flag 169 associated with it (NO in step S206), the camera usage flag 169 set to "0" (or, for example, "FALSE") may be sent to the game image data 168.

[0118] As an alternative, if restrictions on displaying camera images are set (YES in step S212), the camera image data 160 may not be transmitted.

[0119] The transmission cycles of the audio data 164, game image data 168, and camera image data 160 do not have to be the same. At least a portion of the audio data 164, game image data 168, and camera image data 160 may be, for example, data streams conforming to the H.264 (MPEG-4 AVC) standard.

[0120] When transmitting game image data 168 and camera usage flag 169 in a data stream conforming to H.264, the following data processing may be performed.

[0121] Stream data conforming to the H.264 standard consists of a sequence of data called NAL (Network Abstraction Layer) units. NAL units can be broadly classified into VCL (Video Coding Layer)-NAL units, which contain video information, and non-VCL-NAL units, which do not contain video information. VCL-NAL units may contain either keyframes (frames that have not undergone inter-frame compression) or difference frames (delta frames).

[0122] Figure 7 is a diagram illustrating an example of processing game image data 168 and camera usage flag 169 in terminal 100 according to this embodiment.

[0123] Referring to Figure 7, as an example of how a camera usage flag 169 is associated with game image data 168, when game image data 168 includes the camera usage flag 169, the camera usage flag 169 is removed from the game image data 168, and then a VCL-NAL unit 12 containing the game image data 168 is generated. The VCL-NAL unit 12 is data for transmission and, in addition to the header 13, includes the game image data as a payload 14. A non-VCL-NAL unit 16 indicating the removed camera usage flag 169 is also generated. The non-VCL-NAL unit 16 is also data for transmission and, in addition to the header 17, includes the camera usage flag as a payload 18. The VCL-NAL unit 12 and the non-VCL-NAL unit 16 may constitute the data for one frame. In this way, the communication unit 106 of the terminal 100 may transmit a data stream based on the game image and the camera usage flag 169.

[0124] On the other hand, if the game image data 168 does not contain the camera usage flag 169, the VCL-NAL unit 12 is generated from the game image data 168 as is. The VCL-NAL unit 12 is data for transmission and, in addition to the header 13, includes the game image data as a payload 14. In this case, the VCL-NAL unit 12 constitutes the data for one frame.

[0125] Furthermore, if the game image data 168 does not include the camera usage flag 169, the payload 18 may generate a non-VCL-NAL unit 16 that is null or "0". In other words, regardless of whether the game image data 168 includes the camera usage flag 169 or not, the VCL-NAL unit 12 and the non-VCL-NAL unit 16 may constitute the data for one frame. Also, if the camera usage flag 169 associated with the game image data 168 is "0", the non-VCL-NAL unit 16 does not need to be generated.

[0126] Figure 8 is a flowchart showing an example of data reception processing in terminal 100 according to this embodiment.

[0127] Referring to Figure 8, when the chat execution unit 140 receives the communication data 10 (YES in step S300), it determines the type of data included in the communication data 10 (step S302).

[0128] If the communication data 10 includes voice data (referred to as "voice data" in step S302), the chat execution unit 140 extracts the voice data included in the communication data 10 as received voice data 166 (step S304), and outputs the received voice data 166 to the voice acquisition unit 144 (step S306).

[0129] The voice acquisition unit 144 may sequentially output one or more received voice data 166 to the voice processing unit 134. As a result, the voices received from one or more terminals 100 are mixed and output from the speaker 124. In other words, the user of terminal 100 can hear the voices transmitted from one or more other terminals 100.

[0130] If the communication data 10 includes camera image data (referred to as "camera image data" in step S302), the chat execution unit 140 extracts the camera image data included in the communication data 10 as received camera image data 162 (step S308) and outputs it to the camera image acquisition unit 142 (step S310).

[0131] If the communication data 10 includes game image data (referred to as "game image data" in step S302), the chat execution unit 140 extracts the game image data included in the communication data 10 as the received game image data 170 (step S312). The chat execution unit 140 determines whether or not the communication data 10 includes a camera usage flag (step S314). In this embodiment, it is determined that the data stream relating to the received game image includes a non-VCL-NAL unit 16 that includes, for example, "1" as a camera usage flag in its payload.

[0132] If the camera usage flag is included (YES in step S314), the chat execution unit 140 outputs the received game image data 170 and the camera usage flag 171 to the game image acquisition unit 146 (step S316). The camera usage flag 171 is associated with the received game image data 170.

[0133] If the camera usage flag is not included (NO in step S314), the chat execution unit 140 outputs the received game image data 170 to the game image acquisition unit 146 (step S318).

[0134] As long as terminal 100 belongs to any group, the process in steps S300 to S318 is repeated.

[0135] If audio data or image data is transmitted in an interframe compressed state, data restoration processing may be performed. Furthermore, the results of processing or modifying camera image data included in the communication data 10 may be output as received camera image data 162, and the results of processing or modifying game image data included in the communication data 10 may be output as received game image data 170.

[0136] As a variation, if the camera usage flag is not included (NO in step S314), the camera usage flag 171, set to "0" (or, for example, "FALSE"), may be output to the game image acquisition unit 146 along with the received game image data 170.

[0137] [F. Example of processing in the chat screen generation unit 148] Next, we will explain an example of how the chat screen generation unit 148 generates a chat screen.

[0138] Figure 9 is a flowchart showing an example of the process for generating a chat screen in terminal 100 according to this embodiment.

[0139] Referring to Figure 9, the chat screen generation unit 148 determines the display layout of the chat screen (step S400). The display layout of the chat screen may include information such as the type of image data to be used for display, the display position, and the display size.

[0140] The chat screen generation unit 148 acquires image data to be used for display according to the display layout (step S402).

[0141] The chat screen generation unit 148 determines whether or not a restriction on the display of the camera image is set (step S404). If no restriction on the display of the camera image is set (NO in step S404), the process in step S414 is executed.

[0142] If a restriction on displaying camera images is set (YES in step S404), the chat screen generation unit 148 determines whether the acquired image data includes camera image data 160 or received camera image data 162 (step S406).

[0143] If the acquired image data includes camera image data 160 or received camera image data 162 (YES in step S406), the chat screen generation unit 148 sets the camera image data 160 and received camera image data 162 included in the acquired image data not to be used in the generated chat screen (step S408). Alternatively, the camera image data 160 and received camera image data 162 may be changed to mute image data or discarded. Alternatively, the camera image may be blurred or changed to an image displaying a message such as "Display is prohibited."

[0144] If the acquired image data does not include camera image data 160 or received camera image data 162 (NO in step S406), the processing in step S408 is skipped.

[0145] The chat screen generation unit 148 determines whether the acquired image data includes game image data 168 associated with the camera usage flag 169, or received game image data 170 associated with the camera usage flag 171 (step S410).

[0146] If the acquired image contains game image data 168 associated with camera usage flag 169, or received game image data 170 associated with camera usage flag 171 (YES in step S410), the chat screen generation unit 148 performs image processing on the game image data 168 associated with camera usage flag 169 and the received game image data 170 associated with camera usage flag 171 so that the camera image is not displayed (step S412). The image processing to prevent the camera image from being displayed will be described later. Note that the camera usage flag 171 associated with the received game image data 170 may or may not be the same data as the camera usage flag associated with the received game image data 170 by the terminal that sent the received game image data 170. In either case, the receiving terminal can be said to determine the display mode of the received game image data 170 based on the settings for restricting the display of camera images on the receiving terminal and the camera usage flag output by the transmitting terminal.

[0147] In this way, the receiving terminal determines the display mode of the received game image data 170 distributed by the distributing terminal based on the settings for restricting the display of camera images and the camera usage flag 171. The image corresponding to the determined display mode may be generated by the chat screen generation unit 148 or the image processing unit 130 of the terminal.

[0148] If the acquired image does not contain game image data 168 associated with camera usage flag 169, and received game image data 170 associated with camera usage flag 171 (NO in step S410), the processing in step S412 is skipped. Then, the processing in step S414 is executed.

[0149] In step S414, the chat screen generation unit 148 generates a chat screen based on the acquired image data, the modified image data, and / or the image data after image processing (step S414). A video signal corresponding to the generated chat screen is output to the display 120.

[0150] While the chat screen display is enabled, steps S400 to S414 are repeated.

[0151] As a variation, in step S412, instead of performing image processing to prevent the camera image from being displayed, the received game image data may simply be processed so that it is not displayed, or the received game image data may be discarded.

[0152] As a variation, if restrictions on displaying camera images are set on the user's terminal, the acquisition of camera image data 160 by the game application execution unit 182 may be restricted. Due to such restrictions, the game application execution unit 182 is effectively unable to use the camera images, so in steps S410 and S412, only the received game image data 170 may be processed.

[0153] [G. Example of chat screen] Next, an example of a chat screen generated in the information processing system 1 according to this embodiment will be described.

[0154] Figure 10 is a diagram illustrating an example of a chat screen generated in the information processing system 1 according to this embodiment.

[0155] As an example of the premise, terminals 100-1 to 100-4 belong to the same group, and each of terminals 100-1 to 100-4 is streaming game images. Also, there are no restrictions on the display of camera images for terminals 100-1, 100-3, and 100-4, while restrictions on the display of camera images are set for terminal 100-2. Each of terminals 100-1, 100-3, and 100-4 is streaming the camera image of its own terminal. Furthermore, a communication game is being played between terminals 100-1 and 100-4, and the game images output by terminals 100-1 and 100-4 respectively use the camera images from the cameras corresponding to terminals 100-1 and 100-4. Figure 10(A) is an example of the chat screen of terminal 100-1, and Figure 10(B) is an example of the chat screen of terminal 100-2.

[0156] In the example chat screen of terminal 100-1 shown in Figure 10(A), game image 158-1 output by the game processing running on terminal 100-1 is displayed, as well as game images 150-2, 150-3, and 150-4 based on the received game image data 170 distributed by terminals 100-2, 100-3, and 100-4, respectively. Game image 150-1 corresponds to the game image data distributed by terminal 100-1 and is the same image as game image 158-1. Game image 150-1 and game image 158-1 may differ in, for example, image quality and frame rate.

[0157] On terminal 100-1, since no restrictions are set on the display of camera images, game images 150-2 to 150-4 will display the received game image data 170 as is, regardless of whether or not it includes camera images.

[0158] Additionally, the camera images 156-1, 156-3, and 156-4, streamed by terminals 100-1, 100-3, and 100-4 respectively, are superimposed on the game images 150-1, 150-3, and 150-4.

[0159] The camera images used in the game (camera images 152-1 and 152-4 in the example in Figure 10(A)) and the camera images used in the chat (camera images 156-1 and 156-4 in the example in Figure 10(A)) may be the same image. In this case, both camera images will look the same.

[0160] However, the camera image used in game images and the camera image used in chat may appear differently. For example, the camera image used in game images may only include the user's face, while the camera image used in chat may be a full-view image including the background captured by the camera.

[0161] Furthermore, if camera images are used in the game images, it is acceptable to avoid superimposing the camera images onto the game images.

[0162] On the other hand, terminal 100-2 has restrictions set on displaying camera images, so it does not stream camera images, and the camera image is not superimposed on game image 150-2. Note that if the camera image is not streamed, an alternative icon or a message indicating that the camera image is not streamed may be displayed in the position where the camera image would normally be superimposed.

[0163] On the other hand, in the example chat screen of terminal 100-2 shown in Figure 10(B), game image 158-2 output by the game processing running on terminal 100-2 is displayed, as well as game images 150-1, 150-3, and 150-4 based on the received game image data 170 distributed by terminals 100-1, 100-3, and 100-4, respectively. Game image 150-2 corresponds to the game image data distributed by terminal 100-2 and is the same image as game image 158-2.

[0164] Since the game images distributed by terminal 100-1 and terminal 100-4 both include camera images, terminal 100-2, which has restrictions set on displaying camera images, displays muted images as game image 150-1 and game image 150-4, respectively.

[0165] On the other hand, the game image distributed by terminal 100-3 does not include the camera image, so the game image distributed by terminal 100-3 is displayed as game image 150-3. However, unlike game image 150-3 shown in Figure 10(A), the camera image 156-3 is not superimposed on terminal 100-2, where the display of the camera image is restricted. If the display of the camera image is restricted, a mute image or a message indicating that the display of the camera image is restricted may be displayed in the position where the camera image would be superimposed.

[0166] Furthermore, in each game image on the chat screen (the image streamed by the streaming device), the account image and account name indicating the sending device's account may be superimposed on each game image or displayed near each game image, regardless of whether a mute image is displayed or not.

[0167] In the example of the chat screen described above, an example is shown where a mute image is displayed as a game image, illustrating a scenario in which the camera image cannot be identified. Below, we illustrate another scenario in which the camera image cannot be identified. The image shown in Figure 11 may be generated in the image processing step S412 of Figure 9.

[0168] Figure 11 is a diagram illustrating an example of a state in which the camera image in the information processing system 1 according to this embodiment cannot be identified. Figure 11 shows another display example of the game image 150-1 included in the chat screen of terminal 100-2 shown in Figure 10(B). .

[0169] As shown in Figure 11(A), for example, text such as "Camera use is restricted" or an image indicating that camera use is restricted may be displayed.

[0170] As shown in Figure 11(B), for example, a blurred game image may be displayed. As shown in Figure 11(C), a game image may be displayed in which the camera image included in the game image has been changed to a muted image. Normally, the receiving terminal cannot determine the position and range of the camera image included in the received game image data 170 distributed by the distributing terminal. For this reason, for example, the distributing terminal may send two types of image data to display the game image 150-1 shown in Figure 10(A) and the game image 150-1 shown in Figure 11(C). The receiving terminal may decide which image data to use for display.

[0171] For example, if the distribution terminal determines, based on information from the management server 200 or the receiving terminals, that a restriction on displaying camera images has been set on one of the receiving terminals, the distribution terminal may send two types of image data. Alternatively, the distribution terminal may always send two types of image data, one for terminals with camera image display restrictions and one for terminals without restrictions, as long as the game image includes at least a camera image. Note that instead of the game image shown in Figure 11(C), a game image in which the camera image is not used at all may be sent.

[0172] Furthermore, the streaming device may display a game image that includes the camera image on its own screen, while using a different game image that does not include the camera image as the game image it transmits.

[0173] Furthermore, if the distribution terminal has set restrictions on the display of camera images, it may transmit a game image in which the camera image included in the game image has been changed to a muted image (for example, game image 150-1 shown in Figure 11(C)).

[0174] Alternatively, the distribution terminal may transmit data indicating the position and size of the camera image within the game image. The receiving terminal may, based on this data, generate a mute image to cover the camera image and superimpose it onto the game image.

[0175] The display configuration shown in Figure 11 is just one example; any display configuration may be used if the camera image cannot be identified.

[0176] Thus, when a restriction on the display of camera images is set on one terminal 100, it may be decided, based on the camera usage flag 171, not to display game images (received game image data 170) from another terminal 100, or to display game images in a manner in which the included camera images cannot be identified.

[0177] For the sake of explanation, we have focused on a single game image, but a receiving terminal can also receive game image data from multiple distributing terminals and display chat screens based on each received game image data in any layout. Therefore, different display methods may be determined for each received game image data.

[0178] [H. Variant] In the above description, an example of processing was shown in which terminal 100 determines the display mode and generates an image according to the determined display mode. However, at least some of the processing may be handled by the management server 200 or the relay server 300. For example, if a camera usage flag is associated with game image data from any distribution terminal, and display restrictions are set on the receiving terminal to which the data is distributed, the relay server 300 may distribute a muted image instead of the received game image data when relaying the game image data.

[0179] In the above description, a configuration in which independent terminals 100 are connected to a network is illustrated as an example, but at least a part of terminal 100 may be a virtual processing entity. For example, at least a part of terminal 100 may be a virtual machine on the cloud. The entire terminal 100 may be a virtual machine on the cloud, or at least a part of the functions of terminal 100 may be implemented using processing resources on the cloud.

[0180] From another perspective, at least some of the following may be implemented using cloud-based processing resources: the image processing unit 130, the microphone control unit 132, the audio processing unit 134, the communication control unit 136, the camera control unit 138, the chat execution unit 140, the camera image acquisition unit 142, the audio acquisition unit 144, the game image acquisition unit 146, the chat screen generation unit 148, and the restriction unit 190. Furthermore, some functions of each unit may be implemented using cloud-based processing resources.

[0181] Furthermore, at least a portion of the game processing may be executed using external resources such as servers other than terminal 100.

[0182] The process of including a camera image in a game image may not only involve simply superimposing the camera image onto the game image, but may also include game processing in which the camera image interacts with in-game objects. For example, the game may recognize the movement of the user's arm in the camera image and then perform processing such as having the user, displayed as a camera image, throw a ball.

[0183] Game images and specified information may be output separately from the distribution terminal. For example, game images and camera usage flags may be distributed as separate streaming data. Alternatively, game image data may be distributed as streaming data, while the camera usage flag, which is an example of specified information, may be sent to the server or receiving terminal only when it is switched.

[0184] The camera usage flag may be set by the system program 112 or the chat execution unit 140, rather than by the game application execution unit 182 (game application 180). For example, the system program 112 or the chat execution unit 140 may set the camera usage flag based on the fact that the camera image acquisition unit 142 is causing the game application execution unit 182 to acquire a camera image.

[0185] As an example of predetermined information for determining the display mode of the received game image data 170 received by the receiving terminal, a camera usage flag has been given as an example, but any information may be used. The predetermined information should be information that is used together with the setting of restrictions on the display of camera images when determining the display mode of the game image 150 on the receiving terminal. For example, the game application execution unit 182 may set the predetermined flag to "1" when using screenshot images or images obtained from video distribution services as game images. In other words, the predetermined information does not have to be related to whether or not the game image includes, or could include, camera images.

[0186] The specified information is not limited to flags. The specified information is not limited to information indicating one of two values, for example.

[0187] The receiving terminal does not necessarily have to determine how the received game image data 170 is displayed. For example, a server (management server 200 or relay server 300) may determine how the received game image data 170 is displayed for each receiving terminal based on the settings for restrictions on the display of camera images for each receiving terminal.

[0188] Restrictions on sending camera images and restrictions on displaying received game images may be set individually, or only one of the restrictions may be set.

[0189] Restrictions on the display of camera images in the chat screen and restrictions on the transmission of camera images during chat processing / the display of camera images superimposed on game images may be set separately.

[0190] While we have described an example where the display of the camera image is restricted during chat processing (on the chat screen), the situations in which the display of the camera image may be restricted are not limited to this. For example, it may be possible to set restrictions on the display of the camera image in game images independently of the chat function. Also, for example, it may be possible to restrict the display of the camera image at all times on the device. Furthermore, the restriction on the display of the camera image may be set regardless of whether the device is offline or online.

[0191] In the example chat screen shown in Figure 10, terminal 100-1, corresponding to game image 150-1, and terminal 100-4, corresponding to game image 150-4, are running the same communication game, and are displaying their own and the other party's camera images, respectively. Thus, when chatting is in progress and the same communication game is being played, it may be possible to include camera images in the game image only between users who are playing the same communication game. Note that "the same communication game" may be determined, for example, based on identification information unique to the game application. For example, when a game application acquires a camera image on a receiving terminal, the system may acquire the identification information of the game application and the identification information of the game application being played on the chat partner's terminal. The system may then identify the terminal running the same game application and have the game application running on the receiving terminal acquire the received camera image from that terminal. The game application may then display the acquired received camera image within the game image.

[0192] Furthermore, the game application may acquire camera images for display not only from the "same communication game," but also from communication games of the same series or from terminals running predetermined games. For example, the receiving terminal's system may acquire identification information of the game application running on the distribution terminal, identify identification information related to the game application (or the acquired identification information), and cause the game application running on the receiving terminal to acquire camera images from the distribution terminal running the game application corresponding to the identified identification information. Here, "related ID" may be, for example, one or more pieces of identification information pre-set by the system administrator for the identification information of a particular game application, or it may be an ID based on a predetermined rule such as "the same ID except for the last code." Furthermore, "one or more pieces of identification information pre-set by the system administrator" may be held by any of the information processing systems 1, for example, the game application execution unit 182 of the receiving terminal may hold it, the management server 200 may hold it, or the game application execution unit 182 of the distribution terminal may hold it.

[0193] Furthermore, regardless of whether the game applications running on each terminal are the same, terminals belonging to the same group may acquire and display camera images from those terminals. For example, the user of terminal 100-1 shown in Figure 10 is playing a communication game with a user other than the user belonging to the chat-compatible group, and game image 150-1 may include the camera image of the other player in the game. In this case, the game image including the camera image of the other player who does not belong to the chat-compatible group may be distributed within the chat. These examples may also apply to the acquisition of audio within the chat by the game application.

[0194] In the example chat screen shown in Figure 10, terminal 100-1, which corresponds to game image 150-1, and terminal 100-4, which corresponds to game image 150-4, are running the same communication game. However, terminal 100-1 runs the game, and terminal 100-4 sends operation data (and camera images) to terminal 100-1 and receives the game execution result screen from terminal 100-1, so that terminal 100-4 can play the game as if it were running the game itself. In this case, game images 150-1 and 150-4 may be the same or different. In the latter case, game images 150-1 and 150-4 may be generated separately. Furthermore, when terminal 100-4 receives a game image associated with a camera usage flag from terminal 100-1, it may associate the camera usage flag with the game image and distribute it in the chat.

[0195] Similar processing may be applied to audio. A game application may include audio acquired from microphone 122 in the game audio. For example, a game application may set a flag if the game audio includes audio acquired from microphone 122, and a receiving terminal may restrict the output of the game audio based on that flag and the settings for restricting microphone usage on its own terminal.

[0196] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0197] 1 Information processing system, 4 Network, 10 Communication data, 12 VCL-NAL unit, 13,17 Header, 14,18 Payload, 16 Non-VCL-NAL unit, 100 Terminal, 102,202 Processor, 104,204 Memory, 106,206 Communication unit, 108 Camera interface, 110,210 Storage, 112,212 System program, 116,216 Input unit, 118 Camera, 120 Display, 122 Microphone, 124 Speaker, 128 Software module group, 130 Image processing unit, 132 Microphone control unit, 134 Audio processing unit, 136 Communication control unit, 138 Camera control unit, 140 Chat execution unit, 142 Camera image acquisition unit, 144 Audio acquisition unit, 146 Game image acquisition unit, 148 Chat screen generation unit, 150,158 Game image, 152 Camera image, 154 Mute image, 160 Camera image data, 162 Received camera image data, 164 Audio data, 166 Received audio data, 168 Game image data, 169, 171 Camera usage flag, 170 Received game image data, 180 Game application, 182 Game application execution unit, 190 Restriction unit, 200 Management server, 214 Management program, 300 Relay server.

Claims

1. An information processing system, A communication control unit that controls data communication between multiple terminals, including the first and second terminals, that belong to the same group, A camera image acquisition unit that acquires a camera image from a camera corresponding to any terminal belonging to the same group, A restriction setting unit that, in response to user operation, sets the second terminal to restrict the display of camera images of at least cameras corresponding to terminals other than the second terminal on the second terminal, A game application execution unit that executes game processing in response to user operation on the first terminal and outputs a first game image and first information indicating that the camera image of the camera is used in the first game image, An information processing system comprising: a display mode determination unit that determines the display mode of the first game image on the second terminal based on the settings of the second terminal regarding the display restrictions and the first information.

2. The information processing system according to claim 1, wherein the restriction setting unit sets the second terminal to restrict the display of camera images of at least one camera corresponding to a terminal other than the second terminal, in response to user operations on the terminal relating to the second terminal.

3. The information processing system according to claim 2, wherein the terminal relating to the second terminal is the second terminal, a third terminal associated with the second terminal, or a fourth terminal associated with the account of a user using the second terminal.

4. The information processing system according to claim 1, wherein the game application execution unit is capable of generating the first game image using at least the first camera image of the first camera corresponding to the first terminal.

5. The game application execution unit sets a flag when using the first camera image. The first information is based on the flag, as described in claim 4.

6. The information processing system according to claim 1, wherein the game application execution unit is capable of acquiring camera images from cameras corresponding to terminals other than the first terminal that belong to the same group, and is capable of generating the first game image using the acquired camera images.

7. The system further includes a chat execution unit that performs chat using camera images between the aforementioned multiple terminals. The information processing system according to claim 1, wherein the restriction setting unit restricts the use of the camera image when the chat is executed.

8. The chat execution unit acquires the camera image during the execution of the chat, The information processing system according to claim 7, wherein the game application execution unit acquires the camera image acquired by the chat execution unit.

9. The information processing system according to claim 7, wherein the chat execution unit causes game images generated by game processing performed in response to user operations on at least some of the multiple terminals to be displayed on at least other terminals among the multiple terminals.

10. The information processing system according to claim 1, wherein the first terminal comprises a first communication unit and transmits a data stream based on the first game image and the first information to the second terminal.

11. The information processing system according to claim 10, wherein when the image data of the first game image includes a flag, the data stream includes data generated by removing the flag from the image data and data indicating the removed flag.

12. The data stream conforms to the H.264 (MPEG-4 AVC) standard, as described in claim 10 or 11.

13. The information processing system according to claim 1, wherein, when a restriction on the display of a camera image is set in the second terminal, the display mode determination unit determines, based on the first information, not to display the first game image including the camera image, or to display the first game image in a manner in which the camera image cannot be identified.

14. Information processing method, The steps include performing data communication between multiple terminals, including the first and second terminals, that belong to the same group, The steps include: acquiring a camera image from a camera corresponding to any terminal belonging to the same group; The steps include: configuring the second terminal to restrict the display of camera images of at least the cameras corresponding to terminals other than the second terminal, in response to user operation; The steps include: executing game processing in response to user operation on the first terminal, outputting a first game image and first information indicating that the camera image of the camera is used in the first game image; An information processing method comprising the steps of determining the display mode of the first game image on the second terminal based on the settings of the second terminal regarding the display restrictions and the first information.

15. It is a terminal, A game image acquisition unit acquires a game image output when another terminal belonging to the same group executes game processing, and first information indicating that the camera image of the camera is used in the game image. A camera image acquisition unit that acquires a camera image from a camera corresponding to any terminal belonging to the same group, A restriction setting unit that, in response to user operation, sets the terminal to restrict the display of camera images of at least cameras corresponding to terminals other than the terminal, A terminal comprising: a terminal setting relating to the aforementioned display restrictions and a display mode determination unit that determines the display mode of the game image in the terminal based on the terminal setting and the first information.

16. A program that runs on one or more processors, The steps include obtaining a game image output by another terminal belonging to the same group executing game processing, and first information indicating that the camera image of the camera is used in the game image, The steps include: acquiring a camera image from a camera corresponding to any terminal belonging to the same group; The steps include: configuring the terminal to restrict the display of camera images of at least one camera corresponding to a terminal other than the terminal in a terminal including one or more processors, in response to user operation; A program that causes the terminal to perform the step of determining the display mode of the game image on the terminal based on the terminal settings regarding the restrictions on the display and the first information.

Images