Program product, method and system

Abstract

A program product, a method, and a system are provided. There is room for improvement in a process of displaying an image of a user together with an image of a game. A program causes a computer to perform the following processes: acquire a camera moving image captured by a camera; move a pointer corresponding to each of a plurality of controllers based on input to the controller; set a position of the pointer corresponding to each of the controllers; advance a game in which players corresponding to each of the controllers participate; and display a second camera moving image of each of the players generated based on the set position of the pointer corresponding to each of the controllers and in accordance with the camera moving image at a position corresponding to each of the players within the image of the game.

Description

Technical Field

[0001] This disclosure relates to a procedure, method, and system. Background Technology

[0002] A system that displays motion images of the user captured by a camera along with game images is known (e.g., Japanese Patent Application Publication No. 2012-174237). Summary of the Invention

[0003] There is room for improvement in how the user's image is displayed alongside the game's image.

[0004] (Structure 1) According to a procedure of one embodiment, the computer performs the following processes: acquires a camera motion image captured by a camera; moves a pointer corresponding to each of the multiple controllers based on inputs to each of the multiple controllers; sets the position of the pointer corresponding to each controller; progresses the game played by the players corresponding to each of the multiple controllers; and displays a second camera motion image of each player generated based on the set position of the pointer corresponding to each controller and the camera motion image at the position corresponding to each player in the game image.

[0005] (Structure 2) In Structure 1, the program may also cause the computer to perform the following processes: display a camera motion image; and move pointers corresponding to each of the multiple controllers on the displayed camera motion image based on inputs to each of the multiple controllers.

[0006] (Structure 3) In Structure 1 or 2, the program may cause the computer to perform the following processing: identify the faces of one or more people contained in the camera's moving image and place pointers at positions that show the faces of the identified people.

[0007] (Structure 4) In any of the structures 1 to 3, the program may cause the computer to perform the following process: based on the input to each of the multiple controllers, move each pointer to the position of the face that is closest to the current position of the pointer.

[0008] (Structure 5) In any of the structures 1 to 4, the portion of the displayed camera motion image other than the area containing the face of at least one or more persons within the camera's field of view may be masked.

[0009] (Structure 6) In any of the structures 1 to 5, the program may cause the computer to perform the following processing: change the display range of the displayed camera motion image based on the input to at least one of the multiple controllers; and move the pointers corresponding to each controller on the camera motion image after the display range has been changed.

[0010] (Structure 7) In any of the structures 1 to 6, the program causes the computer to perform the following processing: based on the input to each of the multiple controllers, to change the shape of the range represented by each pointer corresponding to each controller; and based on the position of the pointer corresponding to each controller and the range represented by the corresponding pointer, to generate a second camera motion image for each player according to the camera motion image.

[0011] (Structure 8) In any of the structures 1 to 7, the program may cause the computer to perform the following process: while the pointers corresponding to each of the multiple controllers are moved based on the inputs to each of the multiple controllers, a reference camera motion image showing the range of the camera motion image corresponding to the current position of each pointer is displayed.

[0012] (Structure 9) In any of Structures 1-8, the program may cause the computer to perform the following processing: based on inputs to each of the multiple controllers, change the shape of the range represented by each pointer corresponding to each controller; and while the pointers corresponding to each controller are moved based on inputs to each of the multiple controllers, display a reference camera motion image showing the range corresponding to the current position of each pointer. Alternatively, the size of the displayed reference camera motion image may be constant, regardless of the size of the range represented by the corresponding pointer.

[0013] (Structure 10) In any of the structures 1 to 9, the positions of each pointer can overlap with the positions of other pointers.

[0014] (Structure 11) In any of Structures 1 to 10, the game may also be a game that communicates with other information processing devices. Alternatively, the program may cause the computer to perform the following processing: receive, from other information processing devices, a received camera motion image based on a camera motion image and position information based on the position of a set pointer; and display, within the game's image, a third camera motion image generated based on the position information received from other information processing devices and according to the received camera motion image, at the position corresponding to the player on the other information processing devices.

[0015] (Structure 12) In any of the structures 1 to 11, the program may cause the computer to perform the following processing: based on the input to a controller, move a pointer corresponding to the controller; set the position of the pointer; advance the game played by a player corresponding to the controller; and display a second camera motion image of a player generated based on the set position of the pointer and the camera motion image at the position corresponding to the player in the game image.

[0016] (Structure 13) In any of the structures 1 to 12, the program may cause the computer to perform the following process: when the game in which a player corresponding to a controller is playing progresses, display the camera motion image within a specified range of the camera's shooting range at the position corresponding to the player in the game image.

[0017] (Structure 14) According to one embodiment, a method executed by one or more computers is provided. The method includes the steps of: acquiring a camera motion image captured by a camera; moving a pointer corresponding to each of a plurality of controllers based on inputs to each of the controllers; setting the position of the pointer corresponding to each controller; progressing the game played by players corresponding to each of the plurality of controllers; and displaying a second camera motion image of each player, generated based on the set position of the pointer corresponding to each controller and the camera motion image, at the position corresponding to each player within the game image.

[0018] (Structure 15) The system according to one embodiment performs the following processing: acquiring a camera motion image captured by a camera; moving a pointer corresponding to each of the multiple controllers based on inputs to each of the multiple controllers; setting the position of the pointer corresponding to each controller; advancing the game played by the players corresponding to each of the multiple controllers; and displaying a second camera motion image of each player generated based on the position of the pointer corresponding to each controller and the camera motion image at the position corresponding to each player in the game image. Attached Figure Description

[0019] Figure 1 This is a schematic diagram showing an example of the structure of a game device according to this embodiment.

[0020] Figure 2 This is a schematic diagram illustrating a structural example of a system including a gaming device according to this embodiment.

[0021] Figure 3 This is a schematic diagram illustrating an example of the hardware structure of a game device according to this embodiment.

[0022] Figure 4This is a schematic diagram illustrating an example of the hardware structure of the controller according to this embodiment.

[0023] Figure 5 This is an example of a settings screen provided in a game device according to this embodiment.

[0024] Figure 6 This diagram illustrates an example of changing the display range of the camera's moving image in the settings screen.

[0025] Figure 7 This is a diagram used to illustrate an example of changing the range represented by the pointer in the settings screen.

[0026] Figure 8 This diagram illustrates an example of masking a moving image from a camera in a set frame.

[0027] Figure 9 This is a diagram showing an example of a reference camera motion image displayed in the settings screen.

[0028] Figure 10 This diagram illustrates an example of pointer positioning in the settings screen.

[0029] Figure 11 This is another example of a settings screen provided in a game device according to this embodiment.

[0030] Figure 12 This is a flowchart illustrating a processing example performed by the game device according to this embodiment.

[0031] Figure 13 This is a flowchart illustrating a processing example when the game device according to this embodiment executes a communication-type game.

[0032] Figure 14 This is a diagram illustrating an example of a chat screen generated in a game device according to this embodiment.

[0033] Figure 15 This is a diagram illustrating an example of a display mode in a game device according to this embodiment.

[0034] Figure 16 This is a schematic diagram showing an example of a display mode selection screen in a game device according to this embodiment. Detailed Implementation

[0035] This embodiment will be described in detail with reference to the accompanying drawings. Furthermore, the same or equivalent parts in the drawings are labeled with the same reference numerals and their descriptions are not repeated.

[0036] In this specification, even when the same terminology is used to describe data, the content of the data does not need to be completely identical. At least when one set of data and another conveys specific information, they can be considered the same data. Furthermore, the names given to data are for ease of description and are not intended to limit the scope or technical meaning of the data.

[0037] In this specification, a program that enables a computer to perform processing can be a single program or a group of programs. In this disclosure, "a program" does not necessarily refer to a single program, but can include a group of programs. Furthermore, "a program" does not necessarily need to be entirely stored in a single device. "A program" can, for example, refer to the entirety of multiple programs that are stored separately in multiple devices included in the computer.

[0038] In this specification, "computer" does not necessarily refer to a single device, but can include an assembly of multiple devices connected by wires or wirelessly. When multiple devices constitute an information processing device, it can be an example of multiple devices as a whole forming an information processing device, or it can be an example of multiple devices each being an information processing device.

[0039] In this specification, "system" does not necessarily refer to a collection of multiple information processing devices; it can include a single information processing device. A "system" can refer to a single information processing device or multiple information processing devices collectively.

[0040] [A. Structural Example]

[0041] The following description focuses on a gaming device 100, which is an example of an information processing device. The gaming device 100 is also an example of a computer. The information processing device is not limited to the gaming device 100; for example, it could be a personal computer, a television, a tablet computer, a smartphone, a wearable device, a server, etc.

[0042] Figure 1 This is a schematic diagram showing a structural example of the game device 100 according to this embodiment. (Refer to...) Figure 1 The gaming device 100 is capable of communicating with one or more controllers 200. In the following description, when individual controllers among the specific plurality of controllers 200 are required, branch numbers are assigned for differentiation (e.g., Figure 1 The controllers shown are 200-1 and 200-2.

[0043] The gaming device 100 can be connected to one or more controllers 200 via either a wired or wireless connection. When multiple controllers 200 are connected to the gaming device 100, these controllers can be of the same model or different models. Alternatively, some controllers 200 can be connected to the gaming device 100 via wired connections, while others can be connected wirelessly.

[0044] It can be either one user operating one controller 200, or one user operating two controllers 200.

[0045] The gaming device 100 can utilize the camera 118. For example, the gaming device 100 uses camera data captured by the camera 118 to perform various processes. As an example, the camera data is motion picture data captured at a specified frame rate. The motion picture data can also be described as a collection of still image data. Hereinafter, the motion picture data captured by the camera 118 will also be referred to as "camera motion picture". The camera motion picture can also be referred to as an image captured in real time by the camera 118.

[0046] Camera 118 generates motion images of a subject within a shooting range 119, obtained by shooting according to instructions from the gaming device 100 or according to predetermined conditions. Camera 118 may include, for example, imaging elements such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. A lens may also be configured in front of the imaging element.

[0047] The gaming device 100 can be connected to the camera 118 either via a wired connection or wirelessly. The camera 118 can be located outside the gaming device 100 or be part of the gaming device 100. The gaming device 100 can also connect to multiple cameras 118.

[0048] Camera 118 can be, for example, a camera used for video calls. Camera 118 can be a dedicated camera that can only connect to a specific device, or a general-purpose camera that can connect to any device. Alternatively, the gaming device 100 can utilize the camera of other information processing devices (such as smartphones). Thus, the structure of the gaming device 100 utilizing camera 118 is not limited. The gaming device 100 can acquire camera motion images captured by camera 118 using any method.

[0049] exist Figure 1In the example shown, two users can operate controllers 200-1 and 200-2 respectively to play a game together. If the two users are within the field of view 119 of camera 118, camera 118 generates a camera motion image that includes the two users as subjects. The game device 100 can use this camera motion image including the two users as subjects to perform the processing described later.

[0050] Furthermore, for ease of explanation, two controllers 200 are shown as an example, but the number of controllers 200 connected to the game device 100 is not limited. Additionally, the number of users playing games together using the game device 100 is not limited.

[0051] The game device 100 according to this embodiment can also be configured as shown in the following system 1.

[0052] Figure 2 This is a schematic diagram illustrating a structural example of a system 1 including a game device 100 according to this embodiment. Figure 2 In the system 1 shown, four gaming devices 100 are connected via network 4. In the following description, when individual gaming devices among a specific number of gaming devices 100 are required, branch numbers are assigned to distinguish them (e.g., Figure 2 The game devices shown are 100-1, 100-2, 100-3, and 100-4.

[0053] In System 1 according to this embodiment, a user of a game device 100 has their own account, and each account is associated with a game device 100. By making specified inputs to the game device 100, the user with the account can join one or more chat groups (hereinafter also referred to as "groups"). The game device 100 corresponding to an account belonging to any group sends data (hereinafter also referred to as "communication data 10") for communication between users to other game devices 100 corresponding to other accounts belonging to that group. Thus, communication between users is realized within the group.

[0054] The communication data 10 may include, for example, sound data acquired by the gaming device 100, camera motion images acquired by the gaming device 100, text data acquired by the gaming device 100, information related to the application running on the gaming device 100, etc. Furthermore, it is not necessary to include all the data in one communication data 10; one data point can be divided into multiple communication data 10s. Alternatively, one communication data 10 may contain multiple data points.

[0055] Furthermore, the gaming devices 100 corresponding to each account within the same group may not have the same hardware structure. For example, some gaming devices 100 may be dedicated gaming consoles, while others may be personal computers, televisions, tablets, smartphones, wearable devices, servers, etc., capable of running game applications.

[0056] Alternatively, not all game devices 100 corresponding to each account in multiple accounts belonging to the same group can utilize the camera 118.

[0057] System 1 may also include a management server 300 and a relay server 350. The management server 300 is responsible for processing the formation of new groups, processing the participation of one or more accounts in a group, etc. The relay server 350 relays the communication data 10 sent and received by one or more gaming devices 100. The management server 300 and the relay server 350 may also be installed on a single physical server.

[0058] Regarding the method by which a game device 100 sends communication data 10 to one or more other game devices 100, in addition to the method via one or more relay entities (e.g., relay server 350), or alternatively, it may be the method of directly sending communication data 10 to one or more other game devices 100 (e.g., P2P method).

[0059] exist Figure 2 In the illustrated structural example, a game device 100 can acquire camera motion images from one or more other game devices 100. Each game device 100 can perform the processing described later using at least one of the camera motion images captured by its own camera 118 and the received camera motion images based on the camera motion images captured by the cameras 118 of one or more other game devices 100.

[0060] [B. Hardware Structure Example]

[0061] Figure 3 This is a schematic diagram illustrating an example of the hardware structure of the game device 100 according to this embodiment. (Refer to...) Figure 3 The gaming device 100 includes, for example, one or more processors 102, one or more volatile memories 104, a network interface 106, a camera interface 108, one or more non-volatile memories 110, a local communication unit 116, a display 120, a microphone 122, and a speaker 124. Each component can also be detached from the main body of the gaming device 100.

[0062] Processor 102 is the processing unit used to perform the processing in game device 100. Processor 102 includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). Processor 102 expands and executes the program stored in non-volatile memory 110 in volatile memory 104.

[0063] Volatile memory 104 is a volatile storage medium that the processor 102 can access, such as DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.

[0064] Non-volatile memory 110 is a non-volatile storage medium accessible to processor 102, such as flash memory, hard disk, etc. Non-volatile memory 110 may also include storage media that can be loaded and unloaded relative to game device 100, such as cartridges, optical discs, etc. Non-volatile memory 110 is also referred to as a storage device.

[0065] Non-volatile memory 110, for example, stores one or more programs such as system program 112 and game application 126.

[0066] System program 112 includes computer-readable commands for hardware control of the gaming device 100, provision of a program execution environment, etc. System program 112 may also include one or more programs or program modules. For example, system program 112 includes a chat application 114.

[0067] The chat application 114 performs processing related to the sending and receiving of communication data 10 between multiple gaming devices 100 corresponding to multiple accounts belonging to a group (e.g., refer to the above). Figure 2 ).

[0068] Game application 126 includes computer-readable commands for executing the game. Multiple game applications 126 may also be installed on the game device 100. Game applications 126 may also be read from independent storage media. At least a portion of the game execution based on game application 126 may also utilize other game devices 100 or external resources such as servers. System program 112 may also include at least a portion of game application 126.

[0069] In this specification, "processor" may include at least a CPU, GPU, or other processing circuits that execute processing according to computer-readable commands, as well as hard-wired circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array). In this specification, "memory" may include at least volatile memory 104 and non-volatile memory 110, and may also include other storage media and storage devices.

[0070] In the gaming device 100, a SoC (System on Chip) that integrates functions such as a processor and memory can also be used. In this case, the processor and memory exist in the same SoC (integrated circuit). Therefore, in this specification, "processor" and "memory" can include at least two aspects: independent structures and integrated structures.

[0071] Network interface 106 communicates with other gaming devices 100, management server 300, relay server 350, etc., via network 4. Network interface 106 can also communicate with other gaming devices 100, etc., via the Internet, which is an example of network 4. Network 4 can also be a wireless network conforming to Bluetooth (registered trademark), ZigBee (registered trademark), wireless LAN (IEEE 802.11 standard), etc.

[0072] Camera interface 108 relays data transmission to and data reception from camera 118. Camera interface 108 can be connected to camera 118 via either a wired or wireless connection.

[0073] The local communication unit 116 relays data communication with one or more controllers 200. The local communication unit 116 receives data corresponding to inputs to one or more controllers 200 and sends instructions to one or more controllers 200. The local communication unit 116 can also relay data communication with one or more other gaming devices 100. The local communication unit 116 can also utilize wireless communication conforming to Bluetooth (registered trademark), ZigBee (registered trademark), wireless LAN (IEEE 802.11 standard), etc.

[0074] Display 120 displays images or videos generated by processing by processor 102. Display 120 may include, for example, an LCD (Liquid Crystal Display), an organic EL display, etc. Display 120 may also be external to the gaming device 100. In this case, the gaming device 100 may also include interface circuitry with display 120.

[0075] The microphone 122 is disposed, for example, in the housing of the gaming device 100, to collect sound and generate sound signals. The speaker 124 is disposed, for example, in the housing of the gaming device 100, to generate sound based on the sound signals. At least one of the microphone 122 and the speaker 124 may also be located outside the gaming device 100.

[0076] Figure 4 This is a schematic diagram illustrating an example of the hardware structure of the controller 200 according to this embodiment. (Refer to...) Figure 4 The controller 200 includes, for example, one or more processors 202, one or more volatile memories 204, one or more non-volatile memories 210, a local communication unit 216, an input unit 220, a sensor 222, and an actuator 224.

[0077] The processor 202, volatile memory 204, and non-volatile memory 210 are the same as the processor 102, volatile memory 104, and non-volatile memory 110 described above.

[0078] Local communication unit 216, like local communication unit 116 described above, relays data communication with game device 100.

[0079] The input unit 220 accepts input to the controller 200. The input unit 220 may be, for example, a cross key, a button, a lever, a stick, etc.

[0080] Sensor 222 detects information related to the controller 200 itself (e.g., acceleration) and information related to the surroundings of the controller 200 (e.g., temperature).

[0081] The actuator 224 may be, for example, a visually verifiable device such as an LED, an audible device such as a speaker or buzzer, or a device that provides tactile information to the user such as a vibrator.

[0082] [C. Example of using camera motion image processing]

[0083] Next, an example of processing using camera motion images performed in the game device 100 according to this embodiment will be described.

[0084] Figure 5This is a diagram showing an example of a setup screen 150 provided in a game device 100 according to this embodiment. Figure 5 The setup screen 150 shown is used, for example, in a game provided by executing the game application 126 to set up the use of camera motion images. When the game application 126 generates the setup screen 150, the setup screen 150 can also be considered a game image.

[0085] As an example, consider the following scenario: two users are playing a game together, each operating controller 200-1 and 200-2 respectively. In this game, for instance, controller 200-1 is assigned to the first player (hereinafter referred to as "1P"), and controller 200-2 is assigned to the second player (hereinafter referred to as "2P"). Figure 5 In the scenario shown, user USR1 is operating controller 200-1, and user USR2 is operating controller 200-2. User USR1 identifies itself as player 1, and user USR2 identifies itself as player 2. Furthermore, the number of players who can participate can be appropriately determined based on the game.

[0086] In this specification, the term "player" can also be used to refer to the user participating in the game. The term "player" can include the subject participating in the game and their role in the game. For example, "player" can include game objects or game characters operated by users or computers.

[0087] exist Figure 5 The setup screen 150 shown displays moving images captured by camera 118. Figure 5 In the example shown, there are two users within the shooting range 119 of camera 118.

[0088] exist Figure 5 In the setup screen 150 shown, one or more pointers 151 are displayed along with the camera motion image. Pointers 151 can also be used to specify the range of the camera motion image. In the following description, when individual pointers among a specific set of pointers 151 are needed, branch numbers are assigned to distinguish them (e.g., Figure 5 Pointers 151-1 and 151-2 are shown.

[0089] As an example, pointer 151-1 is used for 1P, and pointer 151-2 is used for 2P. Pointers 151-1 and 151-2 represent at least a portion of the camera's moving image. Figure 5 As shown, pointer 151 can also be displayed as an overlay on the camera's moving image.

[0090] As an example, each pointer 151 displays a text object (e.g., "1P" and "2P") indicating its correspondence with the controller 200 or the player. Alternatively, in addition to or in place of the text object, an object indicating the appearance of the corresponding controller 200 may be displayed near each pointer 151, or the display mode of the pointer 151 (e.g., display color) may be set to correspond to the corresponding controller 200 or player.

[0091] Since 1P is assigned to controller 200-1, pointer 151-1 is established to correspond with controller 200-1. The first user USR1 operates controller 200-1 so that the range represented by pointer 151-1 matches the range of the camera motion image to be used in the game. Game device 100 moves pointer 151-1 corresponding to controller 200-1 based on input to controller 200-1 (e.g., pressing a joystick corresponding to the direction of movement).

[0092] Similarly, since 2P is assigned to controller 200-2, pointer 151-2 is established to correspond with controller 200-2. Second user USR2 operates controller 200-2 such that the range represented by pointer 151-2 matches the range of the camera motion image desired for use in the game. Game device 100 moves pointer 151-2 corresponding to controller 200-2 based on the input to controller 200-2.

[0093] exist Figure 5 In the setup screen 150 shown, the game device 100 moves the pointers 151 corresponding to each controller 200 on the displayed camera motion image based on inputs to each controller 200. (As described later...) Figure 11 As shown, pointer 151 can also move to places other than the displayed camera motion image.

[0094] Alternatively, each pointer 151 can move such that the range represented by that pointer 151 at least partially overlaps with the ranges represented by other pointers 151. Furthermore, each pointer 151 can move such that the range represented by one pointer 151 becomes the same as the ranges represented by other pointers 151. That is, the positions of each pointer 151 can at least partially overlap with the positions of other pointers 151. For example, multiple pointers 151 can move to arbitrary positions independently of each other. By not restricting the movement of each pointer 151, even in cases where one user operates multiple controllers 200 to move multiple pointers 151, settings can be configured according to the user's intention.

[0095] Alternatively, for example, it is also possible to prevent the range represented by a certain pointer 151 from overlapping with the range represented by other pointers 151, and to set a limit on the area of ​​allowed overlap.

[0096] When any user performs a specified operation, the positions of each pointer 151 are set. The game device 100 generates position information (e.g., 1P position information and 2P position information) representing the set positions of each pointer 151. In the game, camera motion images are used based on the generated position information.

[0097] The process of setting the position of pointer 151 can be any process, such as the following.

[0098] The position of pointer 151 can also be set for each pointer 151 (or each controller 200). For example, when a user operates controller 200 and presses the decision button, the current position of pointer 151 corresponding to that controller 200 at the moment the decision button is pressed is set as the final position.

[0099] Alternatively, the positions of all pointers 151 can be set simultaneously based on the input to the representative controller 200. For example, when the user corresponding to 1P (as an example) operates the controller 200 and presses the decision button, the current position of each pointer 151 at the moment the decision button is pressed is set to its respective final position. Alternatively, when a timer starting from a certain reference reaches its time limit, the current position of each pointer 151 at the time the time limit is reached is set to its respective final position.

[0100] The gaming device 100 enables the game progress of multiple players corresponding to various controllers among the multiple controllers 200. The gaming device 100 displays local camera motion images of each player, generated based on position information corresponding to each controller 200 and from camera motion images, within the game's image at the positions corresponding to each player. For example, the gaming device 100 generates local camera motion images of each player based on the position of a pointer 151 corresponding to each controller 200 and the range represented by that pointer 151.

[0101] Figure 5 The image shows a game image 160 depicting two players, corresponding to controllers 200-1 and 200-2, participating in a game. In game image 160, the first user, USR1, uses controller 200-1 to operate game object 162-1, and the second user, USR2, uses controller 200-2 to operate game object 162-2.

[0102] In game image 160, a local camera motion image 161-1, generated based on 1P position information and camera motion images, is displayed at the position corresponding to game object 162-1. Similarly, a local camera motion image 161-2, generated based on 2P position information and camera motion images, is displayed at the position corresponding to game object 162-2. Since local camera motion images 161-1 and 161-2 are based on camera motion images captured by camera 118, if local camera motion image 161 includes a user, the expressions of each user can be displayed in real time on the game image.

[0103] Regarding the processing of generating local camera motion images based on location information and camera motion images, it can be either directly cropping out a portion of the camera motion image, performing certain image processing on a cropped portion of the camera motion image, or cropping out a portion based on certain image processing performed on the camera motion image.

[0104] The running game application 126 can also handle the following processes: acquiring camera motion images, generating partial camera motion images 161, and displaying the generated partial camera motion images 161 at positions corresponding to each player within the game image. However, the running game application 126 may not be responsible for all the processing. For example, the running game application 126 may provide the game image to be generated and the position where the partial camera motion images 161 will be displayed to the running system program 112, which will then generate the partial camera motion images 161 and overlay them onto the game image at the specified positions, thereby outputting the final display image. In this way, the game application 126 can also cooperate with other running programs to output a game image 160 containing the partial camera motion images 161.

[0105] exist Figure 5 In the example shown, a circular box representing the range of a local camera motion image is shown as a pointer, but the shape of the box can be arbitrary. The pointer can also be, for example, an arbitrary polygon (e.g., a quadrilateral), an ellipse, or a partially open box of any shape.

[0106] When using a frame as a pointer, the area represented by the pointer may not be exactly the same as the area of ​​the local camera motion image. For example, the local camera motion image may be a range cut inward from the frame at a predetermined distance, or it may be a range cut outward from the frame at a predetermined distance. Furthermore, even if the pointer is determined to enclose a person's face, the local camera motion image may include the person's face and upper body below the head.

[0107] The pointer doesn't have to be a box; it can be an arrow or a face marker.

[0108] The location information only needs to contain the information required to generate a local camera motion image based on the camera's motion image, and can contain any data. For example, the location information may include coordinates (e.g., a set of X and Y coordinates) indicating the position of the pointer (e.g., the center point of the frame), and may also include multiple coordinates specifying the frame indicated by the pointer. The location information may also include information indicating the size of the frame (e.g., the radius of the frame). The location information may also include information indicating the shape of the frame (e.g., quadrilateral, hexagon, etc.).

[0109] The location information set in the settings screen 150 may not be updated during game progress, but it may be updated under certain conditions or user requests.

[0110] Each player's local camera motion image only needs to be shown corresponding to that player and can be displayed anywhere. Several specific examples will be described later.

[0111] Consider the following scenario: In a game with multiple players, the face of the user operating each controller 200 is displayed at a position corresponding to the game object operated by each controller 200. However, when multiple users are within the shooting range 119 of the camera 118, it may be difficult to identify which controller 200 each user is operating. Therefore, a position of the pointer corresponding to each controller is set, and a partial camera motion image of the area within the shooting range 119 of the camera 118 corresponding to that set position is displayed. Thus, for example, even if each user does not move, their face can be reflected in the appropriate position corresponding to the player within the game image. Furthermore, for example, if the user operating the controller 200 changes, it is easy to predict that the new user will be displayed in the game image simply by giving up the seat to the new user, resulting in excellent usability.

[0112] As an example, in setting screen 150, the display range of the camera motion image can be fixed, but it can also be changed according to user operation. Figure 6 This is a diagram illustrating an example of a change in the display range of the camera's moving image in the setting screen 150.

[0113] like Figure 6 As shown, for example, the camera motion image displayed on the setting screen 150 can be magnified by any user operating the controller 200. Alternatively, the camera motion image can be reduced by any user operating the controller 200. Besides magnification or reduction, changes to the display range of the camera motion image can also be made in the vertical or horizontal direction.

[0114] The change of the display range of the camera's moving image can also be achieved using the zoom function of the camera 118 (such as an optical zoom mechanism, digital zoom function, etc.), or by using the horizontal pan mechanism or vertical tilt mechanism of the camera 118 or its peripheral device.

[0115] Furthermore, the display range of the camera motion image can be changed by arbitrarily setting the range of the camera motion image captured by the camera 118 to be displayed on the setting screen 150 within the game device 100. The range set for the camera motion image can also be arbitrarily enlarged or reduced and displayed on the setting screen 150.

[0116] It can be that all controllers 200 connected to the game device 100 can change the display range of the camera's moving image, or it can be that a specific controller 200 can change the display range of the camera's moving image.

[0117] exist Figure 6 In the example shown, the pointer 151 does not change even if the display range of the camera's moving image is changed. However, it is also possible that the position of the pointer 151 or the range represented by the pointer 151 changes according to the change of the display range of the camera's moving image.

[0118] Furthermore, if any user is outside the shooting range 119 of the camera 118, the user can either adjust the shooting range 119 of the camera 118 or move into the shooting range 119 of the camera 118.

[0119] like Figure 6 As shown, the game device 100 can also change the display range of the displayed camera motion image based on input to at least one of the plurality of controllers 200. The game device 100 moves the pointer 151 corresponding to each controller 200 on the camera motion image after the display range has been changed. By changing the display range of the displayed camera motion image, the setting of the pointer 151 can be simplified. This improves usability.

[0120] As an example, in the setting screen 150, the shape of the range represented by the pointer 151 can be fixed, or it can be changed according to the user's operation. Figure 7 This is a diagram illustrating an example of a change in the shape of the range indicated by the pointer in the setting screen 150.

[0121] like Figure 7 As shown, for example, the range represented by the pointer 151 displayed on the setting screen 150 can be zoomed in or out by any user operating the controller 200. That is, the size of the range represented by the pointer 151 can also be changed.

[0122] Regarding the change of the shape of the range represented by pointer 151, in addition to changing the size of the range, or instead of changing the size of the range, the shape of the range represented by pointer 151 can also be changed from a circular frame to a quadrilateral frame.

[0123] like Figure 7 As shown, the game device 100 can also change the size of the range represented by each pointer 151 corresponding to each of the multiple controllers 200 based on the input to each controller 200. By changing the size of the range represented by the pointer 151, the setting of the pointer 151 can be simplified. This improves usability.

[0124] As an example, in setting screen 150, the area outside the user contained in the camera's moving image can also be masked. Figure 8 This is a diagram illustrating an example of mask processing for camera motion images in setting screen 150.

[0125] like Figure 8 (A) and Figure 8 As shown in (B), for example, it could also be that parts of the camera's moving image other than the person are masked, and only the person is cropped out for display. Figure 8 (A) shows an example where the background surrounding a person in a camera motion image is set to white or transparent. Figure 8 (B) shows an example where the background surrounding a person in a camera motion image is set to an arbitrary image or pattern.

[0126] exist Figure 8 (A) and Figure 8In (B), it can also be said that the entire body of the person within the shooting range 119 of the camera 118 is displayed. Alternatively, the parts of the person other than the upper body can be masked, or the parts of the person other than the face can be masked. For example, in the latter case, a camera motion image with the parts of the person other than the face masked can be displayed. That is, it can also be a display where the person's face is cropped out.

[0127] In this way, a camera motion image can be displayed that masks the portion of the image except for the area containing at least one or more faces of persons present in the shooting range 119 of camera 118. By displaying a masked camera motion image, at least the faces of persons are clearly visible, thus simplifying the setting of pointer 151. This improves usability.

[0128] In addition, the type of background or whether to perform masking can be changed by any user operating the controller 200.

[0129] As an example, the camera motion image within the range indicated by pointer 151 can also be displayed on the setting screen 150. Figure 9 This is a diagram showing an example of a reference camera motion image displayed on the setting screen 150. It can also be as follows: Figure 9 As shown, for example, when pointers 151-1 and 151-2 are configured, reference camera motion images 152-1 and 152-2 corresponding to the ranges represented by pointers 151-1 and 151-2 are displayed respectively.

[0130] The reference camera motion image 152 can be displayed either continuously before the position of pointer 151 is set, or after the position of pointer 151 is set. Alternatively, the reference camera motion image 152 can be turned off when the position of pointer 151 is set.

[0131] Alternatively, the display of the reference camera motion image 152 in the setting screen 150 can be toggled by any user operating the controller 200. The toggling of the display of the reference camera motion image 152 can also be set per pointer 151.

[0132] Alternatively, the size of the displayed motion image 152 from the reference camera can be constant, regardless of the shape of the area indicated by the pointer 151. For example, in Figure 9In this case, the range represented by pointer 151-1 is different from the range represented by pointer 151-2, but the shape of the reference camera moving image 152-1 is the same as the shape of the reference camera moving image 152-2. In this case, the magnification or reduction rate of the moving images displayed as reference camera moving image 152-1 and reference camera moving image 152-2 can also be different.

[0133] Alternatively, the size of the reference camera moving image 152 can be determined in relation to the size of the range indicated by pointer 151. For example, the size of the reference camera moving image 152 can also be proportional to the size of the range indicated by pointer 151.

[0134] The reference camera motion image 152 can be configured at a pre-defined position on the setting screen 150, or at a position selected by the user. The reference camera motion image 152 can also be configured based on the position of the corresponding pointer 151. For example, the reference camera motion image 152 can be configured above, below, or similarly positioned on the corresponding pointer 151. In this case, the reference camera motion image 152 can also move in tandem with the pointer 151.

[0135] The reference camera motion image 152 can be configured either as an overlay on the camera motion image displayed in the setting screen 150, or it can be configured at a different position from the camera motion image.

[0136] like Figure 9 As shown, the game device 100 can also display a reference camera motion image 152 showing the range of the camera motion image corresponding to the current position of each pointer 151, while the pointers 151 corresponding to each of the multiple controllers 200 are moved based on inputs to each of the multiple controllers 200. Users can adjust the position of each pointer 151 while referring to the reference camera motion image 152, thus improving usability. By displaying the reference camera motion image 152 at a position different from the position of the corresponding pointer 151, users can confirm the current position of the pointer 151 and the reference camera motion image 152 at the current position together.

[0137] As an example, in setting screen 150, pointer 151 can also be located based on the facial recognition results of the person. Figure 10 This is a diagram illustrating an example of the positioning process of pointer 151 in the setting screen 150.

[0138] The gaming device 100 can also recognize the faces of one or more people contained in a camera's moving image, and arrange pointers 151 at various positions showing the recognized faces of the people. The recognition of the faces of the people can also be based on any recognition algorithm.

[0139] like Figure 10 As shown, for example, by having any user operate the controller 200 (e.g., press a specific button), the pointer 151 or all pointers 151 that were originally configured in any position corresponding to the controller can be automatically configured to the position of the person's face.

[0140] With the pointers 151 automatically configured, the gaming device 100 can also move each pointer 151 to a position indicating the face closest to the current position of each pointer 151 based on input to the controller 200. Figure 10 In the example shown, before the movement, pointer 151-1 was located on the left side of the screen, and pointer 151-2 was located on the right side of the screen. In this state, for example, when a specified operation is performed by any user, pointer 151-1 moves to a position showing the face of the user on the left side of the screen closest to its current position before the movement, and pointer 151-2 moves to a position showing the face of the user on the right side of the screen closest to its current position before the movement. By moving pointer 151 to a position showing the face closest to its current position, the likelihood of moving pointer 151 according to the user's intention is increased.

[0141] When pointer 151 is automatically positioned on the character's face, multiple pointers 151 can be positioned on the same character's face or on the faces of different characters. For example, if the faces of the characters closest to each of the multiple pointers are the same, multiple pointers can be positioned on that character's face. Alternatively, one pointer can be positioned on the character's face, while the other pointers are positioned on the faces of other characters, such as the characters whose faces are next to the other pointer.

[0142] The game device 100 can also position the pointer 151 at the position of the identified person's face when a person's face is identified in the acquired camera motion image, even without user operation. For example, when the setting screen 150 is displayed, if a person's face can be identified in the acquired camera motion image, the position of the identified person's face can be determined as the initial position of the pointer 151.

[0143] The gaming device 100 can also present the position of the identified person's face as a candidate for the movement destination of the pointer 151 when it recognizes the face of a person in the acquired camera motion image, and move the pointer 151 to the selected candidate according to the user's operation.

[0144] In addition to positioning the pointer 151 at the location of the person's face, or alternatively, the shape of the area represented by the pointer can be changed. For example, when the user's face displayed in the setting screen 150 is large, the shape of the area represented by the pointer can be set to be larger compared to when the user's face is small.

[0145] After the pointer 151 is positioned on the face of the person, based on further input to the controller 200, the pointer 151 can be moved further or the shape of the area represented by the pointer can be changed.

[0146] After pointer 151 is positioned on the face of a character, it can also be positioned on the face of another character based on further input to the controller 200. For example, pointer 151 can also be positioned on the face of a character to the right of the currently positioned character. Alternatively, pointer 151 can be positioned on the face of the character corresponding to the larger face number included in the setting screen 150, whenever input is made to the controller 200.

[0147] By moving or configuring pointer 151 based on facial recognition results, the movement or configuration of pointer 151 can be made easier, thus improving usability.

[0148] like Figures 5-10 As shown, the game device 100, based on the display of a camera motion image, moves each pointer 151 corresponding to each of the multiple controllers 200 on the displayed camera motion image according to the input to each controller 200. By moving the pointers 151 on the camera motion image, usability can be improved. Alternatively, each pointer 151 corresponding to each controller 200 can also be moved to an area different from the displayed camera motion image.

[0149] Figure 11 This is a diagram showing another example of the setup screen 150A provided in the game device 100 according to this embodiment. Figure 11 The setting screen 150A shown includes, for example, a motion image display unit 154 that displays camera motion images, a setting receiving unit 156 that receives settings of one or more pointers 151, and one or more reference camera motion images 152 corresponding to the ranges represented by one or more pointers 151.

[0150] User Reference Figure 11The setting screen 150A shown allows adjustment of the position, size, etc., of the pointers 151 displayed in the setting receiver 156. In this way, the game device 100 can also move the pointers 151 corresponding to each controller 200 in areas different from the displayed camera motion image based on input to each controller 200. The setting receiver 156 can display either the same camera motion image as the motion image display unit 154 or not.

[0151] In the setting screen 150A, the reference camera motion image 152 can be displayed in an area other than the motion image display unit 154 and the setting reception unit 156, or it can be displayed superimposed on the setting reception unit 156. Alternatively, the reference camera motion image 152 may not be displayed.

[0152] Referring to the above Figures 5-11 The implementation examples described can be combined appropriately, or some implementation examples may not need to be combined.

[0153] Figure 12 This is a flowchart illustrating a processing example performed by the game device 100 according to this embodiment. Figure 12 The steps shown can also be implemented by executing at least one of the system program 112 and the game application 126 by one or more processors 102.

[0154] The processing described in this embodiment may include other processing, or it may not include some processing. Furthermore, the execution order of each processing is an example; for example, multiple processes may be executed in parallel, or several processes may be executed in the reverse order described. For ease of explanation, the process is divided into units of steps, but these units of division are arbitrary; multiple steps may be integrated, or one step may be divided into multiple steps. As an example, Figure 12 and Figure 13 The processing is performed repeatedly at specified intervals (e.g., a processing frame interval performed every 1 / 60th of a second). The same applies in the description below.

[0155] Reference Figure 12 When the game device 100 is instructed to start the game ("Yes" in step S2), it accepts the number of players participating in the game (step S4). Based on the number of players accepted, the game device 100 determines the correspondence between each player and the controller 200 (step S6). Furthermore, the processes in steps S4 and S6 can also be executed as a series of processes. Through these processes, it is determined which controller 200 each player will operate.

[0156] The game device 100 acquires a camera motion image captured by the camera 118 (step S8). The game device 100 determines the number of pointers 151 to be displayed and the initial position of each pointer 151 based on the number of players (step S10). The game device 100 displays the camera motion image and one or more pointers 151 (step S12).

[0157] The game device 100 changes the display range of the camera's moving image based on the input to the controller 200 (step S14). If there is no input to the controller 200, the processing of step S14 can be skipped.

[0158] The game device 100 changes the position of the corresponding pointer 151 based on the input to the controller 200 (step S16). Furthermore, the game device 100 can also change the shape of the area represented by the corresponding pointer 151 based on the input to the controller 200. If there is no input to the controller 200, step S16 can be skipped.

[0159] The game device 100 determines whether the specified conditions are met (step S18). When the specified conditions are met (yes in step S18), the game device 100 sets the position of the pointer 151 that meets the specified conditions (step S20). That is, the game device 100 generates position information about the pointer 151 that meets the specified conditions.

[0160] For example, the specified conditions may include the setting of a position indicated by the controller 200 corresponding to any pointer, or the elapsed of a specified time. Furthermore, whether the specified conditions are met can be determined either for each pointer 151 or for all the displayed pointers 151.

[0161] If the specified conditions are not met ("No" in step S18), the processing in step S20 is skipped. The game device 100 determines whether position information has been generated for all pointers 151 (step S22).

[0162] If no position information has been generated for any pointer 151 ("No" in step S22), the game device 100 updates the display using a new camera motion image acquired from the camera 118 (step S24). Then, the process from step S14 onwards is repeated.

[0163] If position information is generated for all pointers 151 ("Yes" in step S22), the game device 100 advances the game (step S26). During the game, the game device 100 generates a local camera motion image 161 based on the generated position information and the camera motion image, and displays the generated local camera motion image 161 at the position corresponding to the player.

[0164] [D. Examples of using camera motion images in game applications]

[0165] Not limited to the above Figure 5 The game image 160 shown illustrates that, in a game application, camera motion images can also be utilized in any way.

[0166] Games that utilize camera motion images can be any type of game, such as racing games, card games, board games, FPS (First Person Shooter) games, TPS (Third Person Shooter) games, action games, role-playing games, etc.

[0167] Games that utilize camera motion images can be either games executed by a single gaming device 100 or communication-based games based on internet or local communication.

[0168] The timing, position, and method of displaying the local camera motion image generated based on location information and camera motion images can be arbitrary. For example, during game progress, the local camera motion image can be displayed at a designated position or near game objects such as characters operated by the controller 200. Alternatively, the object operated by the controller 200 may itself be the local camera motion image.

[0169] The motion image from a local camera can also be displayed near the game object operated by the controller 200. Alternatively, the motion image from the local camera can be displayed to indicate the location of the game object. For example, the motion images from each player's local camera can be displayed in correspondence with the positions of each game object on the map.

[0170] The motion image from a local camera can also be displayed near a fixed user interface that displays parameters of game objects operated by the controller 200. For example, the motion image from a local camera can also be displayed near a fixed user interface that displays each player's HP (HitPoint: health points).

[0171] For example, in games such as mahjong or poker where specific characters are not manipulated, local camera motion images can be displayed near various locations corresponding to each controller 200 (e.g., seating positions, user interfaces, and objects). Furthermore, even in games where specific characters or objects are not manipulated through each controller 200, the concept of a player can still exist.

[0172] Additionally, when showing the middle or end of a game, corresponding partial camera motion images can be displayed at the locations indicating the players. In scenes where game results are displayed in ranking format, partial camera motion images representing each player can also be displayed at the locations corresponding to their respective rankings.

[0173] When selecting other players as targets for attack, assistance, etc., partial camera motion images corresponding to the candidate players can be displayed near the option or as options themselves. For example, partial camera motion images representing each player can also be displayed on the wheel used to determine which player to award a reward or penalty.

[0174] The above-mentioned display method of local camera motion images is one example. Depending on the progress of the game, the design of the game application, etc., the local camera motion images can be displayed in various ways at the positions corresponding to each player.

[0175] In a game application, the local camera motion image used can be the same as or different from the camera motion image contained within the range indicated by the pointer. For example, a camera motion image covering a narrower range than the range indicated by the pointer can also be generated as a local camera motion image.

[0176] Local camera motion images can also be the result of image processing on local motion images generated based on positional information and from camera motion images (e.g., cropping along facial contours, facial retouching, etc.). As another example of image processing, local motion images cropped from camera motion images can be processed to add crowns, hats, or beards. As another example of image processing, local motion images cropped from camera motion images can be tilted, rotated, stretched, or flipped horizontally.

[0177] You can also display a local camera motion image surrounded by a bounding box. The way the bounding box is displayed can also change depending on the game situation.

[0178] [E. Examples of the use of camera motion images in communication-based games]

[0179] The processing described in this embodiment can also be applied to communication-based games that use internet or local communication. Hereinafter, an example of processing will be described in the case of a game that communicates with other game devices 100.

[0180] For example, a communication-type game is provided that allows multiple players to participate through communication between multiple game devices 100. This can be a structure where one player participates via one game device 100, or a structure where multiple players can participate via one game device 100.

[0181] As an example, suppose players 1 and 2 participate in a communication-type game from game device 100-1, and players 3 and 4 participate in a communication-type game from game device 100-2. In this case, game images are generated in game device 100-1 and game device 100-2 respectively. Alternatively, the game image generation can occur in one game device 100, and the generated game image can be sent to the other game device 100. Local camera motion images are used in communication-type games. Game device 100-1, for example, is based on... Figure 12 The processing example shown generates two local camera motion images corresponding to player 1 and player 2 respectively, based on the camera motion images. Similarly, game device 100-2 generates two local camera motion images corresponding to player 3 and player 4 respectively, based on the camera motion images. Then, game devices 100-1 and 100-2 each generate game images using the local camera motion images corresponding to players 1-4, based on their own generated local camera motion images and the local camera motion images received from the other game device 100. In this way, in the case of executing a communication-type game, game control information and local motion images cropped from camera motion images can be sent and received between multiple game devices 100, thereby generating game images in each game device 100.

[0182] As another example, in the case of a communication-type game, the camera motion image can be sent and received without pointer-based clipping, and the game device 100 that receives the camera motion image can cut out a portion of the camera motion image. For example, when a communication-type game played over the internet is executed together with a chat application 114, the executing game application 126 can also obtain received camera motion images from other game devices 100 via the executing chat application 114. The received camera motion image is the motion image used to display on the chat screen in the executing chat application 114. Furthermore, the game application 126 can also cut out a portion of the camera motion image from the obtained received camera motion image for use in the game. In this case, sometimes it is not possible to obtain received camera motion images from game devices 100 that are not in the same group.

[0183] The received camera motion image is based on the camera motion image captured by camera 118 acquired by other gaming devices 100. It can be the same as the camera motion image acquired by other gaming devices 100, or it can be the result of some processing. For example, the received camera motion image can also be a motion image after image processing has been applied according to the display mode described later.

[0184] Game application 126 acquires received camera motion images and one or more location information from other gaming devices 100, and uses the acquired location information to utilize the received camera motion images from other gaming devices 100. In this embodiment, the received camera motion images are received via the communication path used by chat application 114, and the location information is received via the communication path used by game application 126. Additionally, game application 126 sends at least one or more location information to other gaming devices 100.

[0185] In this way, the game device 100 displays the position corresponding to the player of the other game device 100 within the game image based on the position information received from the other game device 100 and a received partial camera motion image generated from the received camera motion image. Compared to receiving both the received camera motion image and the received partial camera motion image, the amount of communication can be suppressed. In addition, the amount of data for position information may be smaller than that for the partial camera motion image, thus suppressing the amount of data flowing in the communication path used by the game application 126.

[0186] Alternatively, even when the chat application 114 is not running but the game application 126 is running, the game device 100 may also receive camera motion images and position information from other game devices 100, and cut out partial camera motion images from the camera motion images for use in the game.

[0187] Figure 13 This is a flowchart illustrating a processing example when the game device 100 according to this embodiment executes a communication-type game. Furthermore, it is set as follows: Figure 12 The processing shown is performed in advance to generate position information of each player who operates the controller 200 of the game device 100.

[0188] Reference Figure 13 The game device 100 sends one or more location information of its own device to one or more other game devices 100 participating in the communication-type game being executed (step S50).

[0189] The gaming device 100 sends camera motion images captured by its own camera 118 to other gaming devices 100 belonging to the same group. This processing can also be handled by the running chat application 114.

[0190] The game device 100 receives one or more location information from one or more other game devices 100 participating in a communication-type game (step S52). The game device 100 acquires one or more motion images from receiving cameras from one or more other game devices 100 belonging to the same group (step S54). The running game application 126 can also acquire one or more motion images from receiving cameras from other game devices 100 via the running chat application 114.

[0191] The game device 100 generates one or more local camera motion images based on one or more location information of the device and according to the camera motion images of the device (step S56), and generates one or more received local camera motion images based on one or more location information from one or more other game devices 100 and according to one or more received camera motion images from one or more other game devices 100 (step S58).

[0192] The gaming device 100 displays the game image based on the game image generated according to the game's progress, the local camera motion image generated in step S56, and the received local camera motion image generated in step S58 (step S60).

[0193] The process of steps S50 to S60 is repeated until the game ends (in step S62, it is "Yes").

[0194] Furthermore, since it is assumed that the location information is unlikely to be changed during the execution of the game, the execution cycle for sending or receiving location information shown in step S50 or step S52 can also be longer than the display cycle of the game image.

[0195] [F. Display modes in chat applications]

[0196] In the game device 100 according to this embodiment, if a chat application 114 is running, the camera motion image captured by the camera 118 is processed by the running chat application 114 and sent as part of the communication data 10 to other game devices 100. In this case, the running game application 126 acquires the camera motion image processed by the running chat application 114.

[0197] Figure 14 This is a diagram illustrating an example of a chat screen generated in the game device 100 according to this embodiment. Figure 14 The chat screen shown is primarily generated by the running chat application 114.

[0198] As an example, such as Figure 2 As shown, the four accounts belong to the same group, and these four accounts correspond to game devices 100-1 to 100-4 respectively. Figure 2 In this example, game devices 100-1 to 100-3 distribute camera motion images, while game device 100-4 does not. Game devices 100-1 and 100-2 are engaged in a communication-based game. Two players from each of game devices 100-1 and 100-2 are participating in the communication-based game.

[0199] Figure 14 The image shows an example of a chat screen displayed on the monitor 120 of the gaming device 100-1. Figure 14 The chat screen shown is an example; the screen structure of the chat screen created using communication data 10 is not limited.

[0200] like Figure 14As shown, the chat screen example of game device 100-1 includes a game image 190-1 from the game application 126 running on game device 100-1. When two players are participating, the game application running on game device 100-1 splits the screen horizontally, displaying each player and their surroundings in each split screen. The chat screen example of game device 100-1 includes distribution motion images 192-1 to 192-4 distributed by game devices 100-1 to 100-4 respectively. In this example, game device 100-1 distributes the distribution motion image 192-1 containing game image 190-1 to other game devices 100-2 to 100-4.

[0201] In other gaming devices 100-2 to 100-3, game images 190-1 and distribution motion images 192-1 to 192-4 of the game application being executed by their own gaming device 100 are also displayed. Furthermore, the display order of distribution motion images 192-1 to 192-4 can be different or the same depending on the gaming device 100.

[0202] The motion images 192-1 to 192-3 are superimposed with camera motion images 194-1 to 194-3 distributed by game devices 100-1 to 100-3, respectively. On the other hand, since game device 100-4 does not distribute camera motion images, an icon image 194-4 representing the account corresponding to game device 100-4 is superimposed on the motion image 192-4.

[0203] The camera motion images 194-1 to 194-3 each become a representation corresponding to the display mode set in the game device 100, which serves as the distribution source.

[0204] Figure 15 This diagram illustrates an example of a display mode in the game device 100 according to this embodiment. Figure 15 Here, as an example, three display modes are shown.

[0205] Figure 15 (A) shows an example of camera motion image 158 when the panorama mode is selected as a display mode. Figure 15 The camera motion image 158 shown in (A) is substantially the same as the camera motion image captured by camera 118.

[0206] Figure 15 (B) shows an example of camera motion image 158 when the clipping mode is selected as an example of the display mode. Figure 15The camera motion image 158 shown in (B) only cuts out the people contained in the camera motion image captured by camera 118. If the camera motion image contains multiple people, multiple people can also be cut out.

[0207] Figure 15 (C) shows an example of camera motion image 158 when the face mode is selected as a display mode. Figure 15 The camera motion image 158 shown in (C) only cropes the periphery of the face of the person contained in the camera motion image captured by camera 118. Furthermore, when the face moves within the camera motion image, the cropping position follows the face. In face mode, the camera image can be cropped either along the contour of the face or cropped into a circle, for example, with the background including the face. In cases where the camera motion image contains multiple people, only one person's face can be cropped.

[0208] Furthermore, in the above Figure 14 In the chat screen shown, the user of game device 100-1 selected panoramic mode, while the users of game devices 100-2 and 100-3 selected cut mode.

[0209] When chat application 114 and game application 126 are running simultaneously, the running game application 126 can also obtain information according to... Figure 15 The camera motion image 158 generated in any of the display modes shown. At this time, it can also be displayed in the setting screen 150 according to... Figure 15 The camera motion image 158 generated in any of the display modes shown.

[0210] When multiple users play a game using a single gaming device 100, a pointer 151 corresponding to each player is displayed on the settings screen 150. This is achieved by selecting... Figure 15 The panorama mode and cropping mode shown can display multiple users, thus allowing the display position of different users to be specified as the cropping position of the local camera motion image corresponding to each player.

[0211] On the other hand, in the case of one user playing 100 games on one gaming device, it is also possible to select... Figure 15 The facial pattern shown omits the position specification using pointer 151 in the settings screen 150.

[0212] Furthermore, even when only one user participates in the game via a single gaming device 100, the position of pointer 151 can be set as described above. For example, in Figures 5-11In the image, a pointer 151 is displayed. Additionally, the position of pointer 151 is set for the camera motion image captured by camera 118, thus generating a local camera motion image based on the camera motion image.

[0213] Furthermore, when one user participates in the game using one gaming device 100, a defined range of camera motion image within the shooting range 119 of camera 118 can be used as a partial camera motion image. For example, the defined range can be set to the central portion of the shooting range 119 of camera 118. That is, when the gaming device 100 progresses the game played by one player corresponding to one controller 200, it can display the defined range of camera motion image within the shooting range 119 of camera 118 at the position corresponding to that player within the game image. In addition, the gaming device 100 can also perform processing such as cropping a character's face from the defined range of camera motion image within the shooting range 119. By using the method of using the defined range of camera motion image within the shooting range 119, the user does not need to adjust the position of pointer 151, etc.

[0214] The game application 126 can also provide a user interface for changing the display mode managed in the chat application 114, depending on the number of players participating in the game, etc.

[0215] Figure 16 This is a schematic diagram showing an example of a display mode selection screen in the game device 100 according to this embodiment. Figure 16 The selection screen 180 shown can also be displayed based on a request from the game application 126. In the selection screen 180, for example, it is possible to select three display modes (e.g., panorama mode 181, cutout mode 182, and face mode 183) that are selectable in the chat application 114.

[0216] Figure 16 (A) shows an example of a selection screen 180 conceived in a scenario where multiple users need to be captured by the camera 118 of one gaming device 100. In this example, an object 184 (e.g., the text "Game Recommendation") is displayed to recommend a panoramic mode 181 to the user. Alternatively, a radio button 185 corresponding to the recommended display mode may be selected as an initial value (initial value 186).

[0217] Figure 16(B) shows an example of a selection screen 180 conceived as being captured by a camera 118 of a gaming device 100 on a user. In this example, an object 184 is displayed for recommending facial modes 183 to the user (e.g., the text "Game Recommendation"). Alternatively, a radio button 185 corresponding to the recommended display mode may be selected as an initial value (initial value 186).

[0218] By selecting the OK button 187, the game application 126 provides a display mode selection instruction to the chat application 114. Upon receiving the display mode selection instruction, the display mode in the chat application 114 is set, and the game application 126 acquires camera motion images according to the set display mode.

[0219] [G. Other Implementation Methods]

[0220] Alternatively, camera motion images can be utilized in applications other than gaming applications. For example, in video conferencing applications, it is possible to use motion images from individual camera segments cropped based on location information.

[0221] The embodiments disclosed herein should be considered illustrative in all respects and not restrictive. The scope of the invention is not shown by the foregoing description but by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

Claims

1. A program product containing a program that causes a computer to perform the following processes: Acquire camera motion images captured by the camera; The pointer corresponding to each controller is moved based on the input to each of the multiple controllers. Set the position of the pointer corresponding to each controller; To advance the game for the player corresponding to each of the plurality of controllers; as well as The second camera motion image of each player, generated based on the position of the pointer corresponding to each controller and the camera motion image, is displayed in the game image at the position corresponding to each player.

2. The program product according to claim 1, wherein, The program causes the computer to perform the following processes: Display the moving images from the camera; and Based on the inputs to each of the plurality of controllers, the pointers corresponding to each controller are moved on the displayed camera motion image.

3. The program product according to claim 1, wherein, The program causes the computer to perform the following processing: identify the faces of one or more people contained in the camera's moving image, and position pointers at various locations showing the faces of the identified people.

4. The program product according to claim 3, wherein, The program causes the computer to perform the following processing: based on input to each of the plurality of controllers, to move each pointer to the position of the face closest to the current position of the pointer.

5. The program product according to claim 2, wherein, The portion of the displayed camera motion image other than the area containing the face of at least one or more persons within the camera's field of view is masked.

6. The program product according to claim 2, wherein, The program causes the computer to perform the following processes: The display range of the displayed camera motion image is varied based on input to at least one of the plurality of controllers; and This causes the pointers corresponding to each controller to move on the camera motion image after the display range has been changed.

7. The program product according to claim 2, wherein, The program causes the computer to perform the following processes: Based on the inputs to each of the plurality of controllers, the shape of the range represented by each pointer corresponding to each controller is changed; and Based on the position of the pointer corresponding to each controller and the range represented by the corresponding pointer, the second camera motion image of each player is generated according to the camera motion image.

8. The program product according to claim 2, wherein, The program causes the computer to perform the following processing: while the pointers corresponding to each of the plurality of controllers are moved based on inputs to each of the controllers, a reference camera motion image showing the range of the camera motion image corresponding to the current position of each pointer.

9. The program product according to claim 2, wherein, The program causes the computer to perform the following processes: Based on the inputs to each of the plurality of controllers, the shape of the range represented by each pointer corresponding to each controller is changed; and In a state where pointers corresponding to each of the plurality of controllers are moved based on inputs to each of the controllers, a reference camera motion image is displayed showing the range of the camera motion image corresponding to the current position of each pointer. The size of the displayed motion image from the reference camera is constant and independent of the size of the range indicated by the corresponding pointer.

10. The procedure product according to any one of claims 1 to 9, wherein, The positions of each pointer can overlap with the positions of other pointers.

11. The procedure product according to any one of claims 1 to 9, wherein, The game in question is one that communicates with other information processing devices. The program causes the computer to perform the following processes: Receive from the other information processing device the received camera motion image based on the camera motion image and the position information based on the position of the set pointer; as well as The third camera motion image, generated based on the location information received from the other information processing device and according to the received camera motion image, is displayed in the game image at the position corresponding to the player of the other information processing device.

12. The program product according to any one of claims 1 to 9, wherein, The program causes the computer to perform the following processes: Based on the input to a controller, move a pointer corresponding to that controller; Set the position of the one pointer; To advance the game for the player corresponding to the one controller; as well as The second camera motion image of the player, generated based on the position of the set pointer and the camera motion image, is displayed in the game image at the position corresponding to the player.

13. The program product according to claim 1, wherein, The program causes the computer to perform the following processing: when the game is progressing for a player corresponding to a controller, display a camera motion image within a specified range of the camera's field of view at the position corresponding to the player within the game's image.

14. A method, executed by one or more computers, comprising the following steps: Acquire camera motion images captured by the camera; The pointer corresponding to each controller is moved based on the input to each of the multiple controllers. Set the position of the pointer corresponding to each controller; To advance the game for the player corresponding to each of the plurality of controllers; as well as The second camera motion image of each player, generated based on the position of the pointer corresponding to each controller and the camera motion image, is displayed in the game image at the position corresponding to each player.

15. A system that performs the following processing: Acquire camera motion images captured by the camera; The pointer corresponding to each controller is moved based on the input to each of the multiple controllers. Set the position of the pointer corresponding to each controller; To advance the game for the player corresponding to each of the plurality of controllers; as well as The second camera motion image of each player, generated based on the position of the pointer corresponding to each controller and the camera motion image, is displayed in the game image at the position corresponding to each player.

Images