Game programs, methods, and systems

Abstract

There is room for improvement in the process of displaying user images alongside game images. [Solution] A game program is provided that causes a computer to execute game processing. The game program causes the computer to acquire a captured video based on a camera video captured by a camera, and to execute one or more games that change the game state of multiple players corresponding to each of the multiple first regions, based on the fact that each of the multiple first regions contains at least a part of the user's body, and to execute a preparatory process before executing the one or more games. The preparatory process includes displaying a preparatory image on a display camera video based on the captured video, in which multiple guide images indicating each of the multiple second regions, which are arranged horizontally and correspond to each of the multiple first regions, are placed, and determining that each of the multiple second regions contains at least a part of the user's body.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a game program, a method, and a system.

BACKGROUND ART

[0002] A system is known that displays a moving image that may include a user captured by a camera together with a game image (for example, Japanese Patent Laid-Open No. 2012-174237 (Patent Document 1)).

RELATED ART DOCUMENTS

PATENT DOCUMENTS

[0003]

PATENT DOCUMENT 1

SUMMARY OF INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] There is room for improvement in the process of displaying a user's image together with a game image.

MEANS FOR SOLVING THE PROBLEMS

[0005] (Configuration 1) A game program according to an embodiment causes a computer to execute game processing. The game program causes the computer to acquire a captured moving image based on a camera moving image captured by a camera, and for a plurality of first regions arranged side by side in an image of the captured moving image, change the game state of each of a plurality of players respectively corresponding to the plurality of first regions based on at least a part of the user's body included in each of the plurality of first regions to execute one or a plurality of games, and execute a preparation process before the execution of the one or plurality of games. The preparation process includes displaying a preparation image in which a plurality of guide images respectively indicating a plurality of second regions arranged side by side respectively corresponding to the plurality of first regions are arranged on a display camera moving image based on the captured moving image, and determining that at least a part of the user's body is included in each of the plurality of second regions.

[0006] (Configuration 2) In Configuration 1, the game may include multiple games in which the position or size of the first region differs in at least the vertical direction. The game program may cause the computer to perform specific game pre-preparation processing after the preparation processing and before at least one of the one or more games is executed. The specific game pre-preparation processing may include displaying a specific game pre-preparation image on a display camera video based on captured video, in which a plurality of second guide images are placed, each showing a plurality of horizontally arranged third regions corresponding to a plurality of first regions in at least one game before execution, and determining that each of the third regions includes at least a part of the user's body.

[0007] (Configuration 3) In Configuration 1 or 2, the preparation process may include determining that the state in which at least part of the user's body is included in all of the multiple second regions continues for more than 1 hour. The specific pre-game preparation process may include determining that at least part of the user's body is included in all of the multiple third regions, or determining that the state in which at least part of the user's body is included in all of the multiple third regions continues for more than 2 hours, which is shorter than 1 hour.

[0008] (Configuration 4) In any of Configurations 1 to 3, if the specific game pre-preparation process determines that at least a part of the user's body is included in any of the multiple third regions, it may include displaying the same object that is displayed during the execution of at least one game prior to execution at a position corresponding to at least a part of the user's body included in the third region.

[0009] (Configuration 5) In any of Configurations 1 to 4, the preparation process may include determining that the state in which at least a part of the user's body is included in all of the multiple second regions continues for more than 1 hour.

[0010] (Configuration 6) In any of Configurations 1 to 5, the preparation process may include displaying the guide image in a different display manner than in other cases when at least a part of the user's body is included in all of the multiple second regions.

[0011] (Configuration 7) In any of Configurations 1 to 6, the preparation process may include changing the combination of correspondences between multiple first areas and multiple players in response to user operations.

[0012] (Configuration 8) In any of Configurations 1 to 7, the game may include multiple games in which the position or size of the first region differs in at least the vertical direction. The game program may cause the computer to perform specific game pre-preparation processing after the preparation process and before at least one of the one or more games is executed. The specific game pre-preparation processing may include displaying a specific game pre-preparation image on a display camera video based on captured video, in which multiple second guide images are placed, each showing multiple horizontally arranged third regions corresponding to multiple first regions in at least one game before execution, and determining that each of the third regions includes at least a part of the user's body. The game program may cause the computer to display an image after the preparation process and before at least one of the one or more games is executed, in which multiple partial videos corresponding to multiple first regions extracted from captured video are placed in positions corresponding to the changed combination of correspondences.

[0013] (Configuration 9) In any of Configurations 1 to 8, the game program may cause the computer to run one or more games that change the player's game state corresponding to one first region based on at least a part of the user's body contained in one first region of the captured video image. The preparation process may include displaying a preparation image on a display camera video based on the captured video, in which a guide image indicating one second region corresponding to one first region is placed, and determining that at least a part of the user's body is included in one second region.

[0014] (Configuration 10) In any of Configurations 1 to 9, at least a part of the user's body may be the user's face.

[0015] (Configuration 11) In any of Configurations 1 to 10, the preparation process may include displaying a pointer indicating the position of the user's face when the user's face is not included in any of the multiple second regions.

[0016] (Configuration 12) In any of Configurations 1 to 11, the preparation process may include displaying the pointer in a different display manner than when the user's face is not included in the second area, when the user's face is included in one of the multiple second areas.

[0017] (Configuration 13) Configuration 1 to 12 may include determining the vertical position of each user's video in the image displayed after the preparation process, based on the vertical position of the user in each of the multiple first regions detected in the preparation process.

[0018] (Configuration 14) In any of Configurations 1 to 13, the vertical position of the partial video corresponding to each of the multiple first regions in the image displayed after the preparation process may be determined based on the vertical position of the user in each of the multiple first regions detected in the preparation process.

[0019] (Configuration 15) According to one embodiment, a method is provided for a computer to perform. The computer acquires a captured video based on a camera video captured by a camera, and for a plurality of first regions arranged horizontally in the image of the captured video, it performs one or more games that change the game state of a plurality of players corresponding to each of the plurality of first regions based on at least a part of the user's body contained in each of the plurality of first regions, and performs preparation processing before performing one or more games. The preparation processing includes displaying a preparation image on a display camera video based on the captured video, in which a plurality of guide images indicating a plurality of second regions arranged horizontally, each corresponding to the plurality of first regions, are arranged, and determining that each of the plurality of second regions contains at least a part of the user's body.

[0020] (Configuration 16) A system capable of executing a game program according to one embodiment acquires a captured video based on a camera video captured by a camera, and executes one or more games that change the game state of multiple players corresponding to each of the multiple first regions based on at least a part of the user's body included in each of the multiple first regions arranged horizontally in the image of the captured video, and performs preparation processing before executing one or more games. The preparation processing includes displaying a preparation image on a display camera video based on the captured video, in which multiple guide images indicating multiple second regions arranged horizontally corresponding to each of the multiple first regions are placed, and determining that at least a part of the user's body is included in each of the multiple second regions. [Brief explanation of the drawing]

[0021] [Figure 1] This is a schematic diagram illustrating an example of the application of a game device according to this embodiment. [Figure 2] This is a schematic diagram showing an example of the hardware configuration of a game device according to this embodiment. [Figure 3] This flowchart shows an example of the overall processing performed by a game device according to this embodiment. [Figure 4]This is a diagram for explaining an example of the preparation processing of the game device according to the present embodiment. [Figure 5] This is a schematic diagram showing an example of the preparation image 160 displayed in the preparation processing of the game device according to the present embodiment. [Figure 6] This is a flowchart showing an example of the processing procedure of the preparation processing in step S2 shown in FIG. 3. [Figure 7] This is a diagram for explaining the processing example of step S38 shown in FIG. 6. [Figure 8] This is a diagram for explaining the processing of changing the combination of correspondences between the first region 154 and the player. [Figure 9] This is a schematic diagram showing another example of the preparation image 160 displayed in the preparation processing of the game device according to the present embodiment. [Figure 10] This is a diagram for explaining the partial moving image adjustment processing of the game device according to the present embodiment. [Figure 11] This is a schematic diagram showing an example of adjusting the position at which a partial moving image is clipped from a captured moving image. [Figure 12] This is a diagram for explaining the pre-specific-game preparation image displayed in the pre-specific-game preparation processing. [Figure 13] This is a schematic diagram showing an example of the pre-specific-game preparation image displayed in the pre-specific-game preparation processing of the game device according to the present embodiment. [Figure 14] This is a flowchart showing an example of the processing procedure of the pre-specific-game preparation processing in step S5 shown in FIG. 3. [Figure 15] This is a schematic diagram showing an example of the game processing executed by the game device according to the present embodiment. [Figure 16] This is a schematic diagram showing another example of the game processing executed by the game device according to the present embodiment. [Mode for Carrying Out the Invention]

[0022] This embodiment will be described in detail with reference to the drawings. Note that identical or corresponding parts in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[0023] In this specification, even if the same terminology is used to describe data, the content of the data does not need to be exactly the same. At the very least, if one piece of data and another piece of data both convey specific information, they may be considered the same piece of data. Furthermore, the names assigned to the data are for convenience only and do not limit the scope or technical meaning of the data.

[0024] In this specification, a program that causes a computer to perform a process may be a single program or a group of programs. In this disclosure, “a program” does not have to mean a single program and may include a group of programs. Furthermore, “a program” does not have to be stored entirely in a single device. “A program” may mean, for example, the entirety of multiple programs stored in multiple devices included in a computer.

[0025] In this specification, “computer” does not necessarily mean a single device, but may include a whole set of multiple devices connected by wire or wireless. If an information processing device consists of multiple devices, the whole set of multiple devices may be an example of an information processing device, or each of the multiple devices may be an example of an information processing device.

[0026] In this specification, "system" does not necessarily mean a collection of multiple information processing devices, but may include a single information processing device. A single information processing device may be a "system," or a collection of multiple information processing devices may be a "system."

[0027] [A. Application example] This section describes a game device 100, which is an example of an information processing device. The game device 100 is also an example of a computer. The information processing device is not limited to the game device 100, and may include, for example, a personal computer, a television, a tablet device, a smartphone, a wearable device, a server, etc.

[0028] Figure 1 is a schematic diagram showing an application example of the game device 100 according to this embodiment. Referring to Figure 1, the game device 100 can utilize a camera 118. For example, the game device 100 performs various processes using camera data captured by the camera 118. Camera data is, as an example, moving image data captured at a predetermined frame rate. Moving image data can also be described as a collection of still image data. Hereinafter, the moving image data captured by the camera 118 will also be referred to as "camera video". Camera video can also be described as images captured by the camera 118 in real time.

[0029] The camera 118 generates a camera video of a subject within the imaging range 119, in accordance with commands from the game device 100 or according to predetermined conditions. The camera 118 includes an image sensor, such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. A lens may be placed in front of the image sensor.

[0030] The game device 100 may be connected to the camera 118 by a wire or by wireless connection. The camera 118 may be located outside the game device 100 or may be part of the game device 100. Multiple cameras 118 may be connected to the game device 100. The camera 118 may be mounted on the game device 100 so as to include a user facing the game device 100 in its imaging range 119, for example, as shown in Figure 1. Alternatively, the camera 118 may be connected to the game device 100 by a cable, and its position and orientation may be adjusted so as to include a user facing a separate display 120, from which images are output, in its imaging range 119.

[0031] Camera 118 may be, for example, a camera for video calls. Camera 118 may be a dedicated camera that can only be connected to a specific device, or it may be a general-purpose camera that can be connected to any device. Alternatively, the game device 100 may utilize the camera of another information processing device (for example, a smartphone). Thus, the configuration in which the game device 100 utilizes camera 118 is not limited. The game device 100 may acquire camera video captured by camera 118 in any way.

[0032] In the example shown in Figure 1, if two users, USR1 and USR2, are within the imaging range 119 of camera 118, camera 118 generates a camera video that includes both users as subjects. The game device 100 can perform the processing described later using the camera video that includes one or more users as subjects. There is no limit to the number of users that can be included in the camera video processed by the game device 100. Also, there is no limit to the number of users who play the game together using the game device 100.

[0033] Each frame contained in a camera video or a video based on a camera video represents a rectangular image. In this specification, the terms “vertical” and “horizontal” (or “vertical” and “horizontal”) may be used in reference to the image (see, for example, Figure 4(A) below). “Vertical” may also be interpreted as “up and down direction,” and “horizontal” may also be interpreted as “left and right direction.”

[0034] In this specification, the terms "horizontal direction," "vertical direction," and "distance direction" may be used to describe the position or movement of the camera 118 within the imaging range 119. The horizontal direction of the imaging range 119 corresponds to the horizontal direction within the image of the camera video captured by the camera 118, and the vertical direction of the imaging range 119 corresponds to the direction parallel to the vertical direction within the image of the camera video captured by the camera 118. The distance direction of the imaging range 119 corresponds to the direction parallel to the optical axis of the camera 118.

[0035] The game device 100 may be able to communicate with one or more controllers. In this case, the user can operate the controllers to give various commands to the game device 100. The game device 100 may be connected to one or more controllers by wire or by wireless connection. When multiple controllers are connected to the game device 100, the multiple controllers may be of the same model or different models. Some controllers may be connected to the game device 100 by wire, and other controllers may be connected to the game device 100 by wireless connection.

[0036] [B. Hardware Configuration Example] Figure 2 is a schematic diagram showing an example of the hardware configuration of a game device 100 according to this embodiment. Referring to Figure 2, the game device 100 includes, for example, one or more processors 102, one or more volatile memory 104, a network interface 106, a camera interface 108, one or more non-volatile memory 110, a local communication unit 116, a display 120, a microphone 122, and a speaker 124. Each component may be detachable from the main body of the game device 100.

[0037] The processor 102 is the processing unit for executing processes in the game device 100. The processor 102 includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 102 loads programs stored in the non-volatile memory 110 into the volatile memory 104 and executes them.

[0038] The volatile memory 104 is a volatile storage medium accessible by the processor 102, and includes, for example, DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory).

[0039] The non-volatile memory 110 is a non-volatile storage medium accessible by the processor 102, and includes, for example, flash memory or a hard disk. The non-volatile memory 110 may also include, for example, a storage medium that can be attached to or removed from the game device 100, such as a cartridge or an optical disc. The non-volatile memory 110 is also referred to as storage.

[0040] The non-volatile memory 110 stores one or more programs, such as a system program 112 and a game program 126.

[0041] The system program 112 includes computer-readable instructions for hardware control of the game device 100 and providing a program execution environment. The system program 112 may include one or more programs or program modules. For example, the system program 112 may include a chat application 114.

[0042] The chat application 114 performs processing related to the sending and receiving of communication data 10 between multiple game devices 100 corresponding to multiple accounts belonging to a chat group.

[0043] The game program 126 includes computer-readable instructions for executing game processing. Multiple game programs 126 may be installed on the game device 100. The game program 126 may be read from a separate storage medium. At least part of the game execution by the game program 126 may utilize other game devices 100 or external resources such as a server. The system program 112 may include at least part of the game program 126.

[0044] In this specification, “processor” may include at least processing circuits that perform processing according to computer-readable instructions, such as a CPU or GPU, and hardwired circuits, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). In this specification, “memory” may include at least volatile memory 104 and non-volatile memory 110, and may also include other storage media and storage devices.

[0045] In the game device 100, a System on Chip (SoC) in which functions such as a processor and memory are integrated may be used. In this case, the processor and memory reside on the same SoC (integrated circuit). Therefore, in this specification, "processor" and "memory" may include at least both configurations that are independent of each other and configurations that are integrated with each other.

[0046] The network interface 106 communicates data with other game devices 100, a management server 300, a relay server 350, etc., via network 4. The network interface 106 may also communicate data with other game devices 100, etc., via the internet, which is an example of network 4. Network 4 may be a wireless network conforming to Bluetooth®, ZigBee®, wireless LAN (IEEE 802.11 standard), etc.

[0047] The camera interface 108 mediates the transmission of data to and from the camera 118. The camera interface 108 may be connected to the camera 118 by a wire or by wireless connection.

[0048] The local communication unit 116 mediates data communication with one or more controllers. The local communication unit 116 receives data corresponding to inputs to one or more controllers and transmits commands to one or more controllers. The local communication unit 116 may also mediate data communication with one or more other game devices 100. The local communication unit 116 may use wireless communication such as Bluetooth®, ZigBee®, or wireless LAN (IEEE 802.11 standard).

[0049] The display 120 displays images or videos generated by processing by the processor 102. The display 120 includes, for example, an LCD (Liquid Crystal Display) or an organic EL display. The display 120 may be located outside the game device 100. In this case, the game device 100 may include an interface circuit to the display 120.

[0050] The microphone 122 is located, for example, in the casing of the game device 100, and collects sound to generate a sound signal. The speaker 124 is located, for example, in the casing of the game device 100, and generates sound from the sound signal. At least one of the microphone 122 and the speaker 124 may be located outside the game device 100.

[0051] [C. Example of overall processing] Next, an example of the overall processing performed in the game device 100 according to this embodiment will be described. The game device 100 according to this embodiment is capable of running one or more games. The game device 100 can utilize camera video while running one or more games.

[0052] Figure 3 is a flowchart showing an example of the overall processing performed by a game device 100 according to this embodiment. Each step shown in Figure 3 may be achieved by one or more processors 102 executing at least one of the system program 112 or the game program 126.

[0053] The processes described in this embodiment may include other processes, or some processes may be omitted. Furthermore, the execution order of each process is merely an example; for example, multiple processes may be executed in parallel, or some processes may be executed in the reverse order of the description. For the sake of explanation, the process is divided into units called steps, but these divisions are arbitrary; multiple steps may be combined, or one step may be divided into multiple steps. The same applies to the following description.

[0054] Referring to Figure 3, when the game device 100 is instructed to start game processing (YES in step S1), it performs preparation processing (step S2). The game device 100 performs preparation processing before running one or more games. In the preparation processing, the game device 100 assists with adjusting the user's position and other necessary actions for game processing that utilizes camera video.

[0055] Next, the game device 100 performs a game introduction process (step S3) and selects a game to play (step S4). The game to play may be selected randomly from a group of games. The selection of the game to play may or may not be based on user input.

[0056] When the first game is selected (referred to as "first game" in step S4), the game device 100 performs a specific game pre-preparation process (step S5). In the specific game pre-preparation process, the game device 100 acquires setting information and other necessary information for the first game that utilizes camera video. Then, the game device 100 performs the game processing for the first game (step S6).

[0057] Thus, the game device 100 may perform specific game pre-preparation processing after the preparation processing and before at least one of the one or more games is executed.

[0058] When the second game is selected (referred to as "second game" in step S4), the game device 100 executes the game processing for the second game (step S7). For example, the second game can execute game processing that utilizes camera video based on the setting information acquired in the preparation process.

[0059] If the system is instructed to run the game process again (YES in step S8), the processes from step S3 onward are repeated. Note that the specific game preparation process in step S5 may be omitted if it has already been executed once. Also, if the system is instructed to run the game process again (YES in step S8), the processes from step S2 onward may be repeated.

[0060] If no instruction is given to execute the game process again (NO in step S8), the process terminates. The process in step S8 may also be a process to determine whether or not the termination of the game process has been instructed.

[0061] For the sake of explanation, we will show an example where there are two options, Game 1 and Game 2, but the number of game types is not limited. Also, the presence or absence of specific pre-game preparation processes, and the content of those pre-game preparation processes, may differ for each game.

[0062] [D. Preparation Process] The preparation process in step S2 shown in Figure 3 includes, for example, a process to help each user participating in the game adjust their position in the horizontal direction within the imaging range 119. For example, in the preparation process, the user moves primarily in the horizontal direction so that the subsequent game process can be performed properly. The horizontal movement of the user changes the position of the user's image in the horizontal direction of the camera video. The preparation process may also include a process to help adjust the horizontal position of each user within the image.

[0063] Figure 4 is a diagram illustrating an example of the preparation process for a game device 100 according to this embodiment. As an example, we assume that two users USR1 and USR2 play the game together. The game device 100 acquires an image video 152 based on the camera video 150 captured by the camera 118.

[0064] For example, the camera video 150 shown in Figure 4(A) includes two users, USR1 and USR2, as subjects. The captured video 152 is a video based on the camera video 150. The captured video 152 may be the same as the camera video 150, or it may be a processed version of the camera video 150. The captured video 152 shown in Figure 4(B) is a video in which everything except the people included in the camera video 150 is masked, and only the people are cut out. The captured video 152 shown in Figure 4(B) can also be described as a video cut out using the outline of the people included in the camera video 150. In this case, the background around the people included in the camera video 150 may be set to white or transparent, or it may be set to a specific color or pattern.

[0065] The captured video 152 may be generated by the processor 102 of the game device 100, by at least one of the camera 118 or the camera interface 108, or by another game device 100 or an external server. The method by which the game device 100 acquires the captured video 152 is not limited.

[0066] In this embodiment, one or more first regions 154 are set within the image of the captured video 152. In the following description, when it is necessary to identify each of the multiple first regions 154, they are distinguished by sub-numbers (for example, first regions 154-1, 154-2 as shown in Figure 4).

[0067] When multiple first regions 154 are set within the image of the captured video 152, the multiple first regions 154 are arranged side by side. In this embodiment, the arrangement of multiple first regions 154 side by side may mean that one first region 154 is positioned in the horizontal direction of the captured video 152 in a location that does not overlap with other first regions 154. The horizontal size of one first region 154 in the captured video 152 may be the same as or different from that of other first regions 154. Similarly, the vertical position and vertical size of one first region 154 in the captured video 152 may be the same as or different from that of other first regions 154. The same applies when multiple other regions are arranged side by side.

[0068] The number, position, and shape of the first region 154 may be determined or changed depending on the content of the game or the number of players participating in the game. The shape of the first region 154 is not limited to a rectangle; it may be a circle, any polygon, a discontinuous shape, etc. For the sake of explanation, Figure 4(B) shows an example where two first regions 154 are set up, but for example, if four users play the game together on one game device 100, four first regions 154 may be set up.

[0069] Note that in some cases, only one first region 154 may be set within the image of the captured video 152. Below, we will mainly explain an example of processing when multiple first regions 154 are set, but the same processing may be performed when only one first region 154 is set.

[0070] Each of the first region 154 may be associated with a player participating in the game. In the example shown in Figure 4(B), for example, the first region 154-1 is associated with the first player (hereinafter also referred to as "1P"), and the first region 154-2 is associated with the second player (hereinafter also referred to as "2P"). Assume that user USR1 has selected 1P and user USR2 has selected 2P.

[0071] In this specification, the term "player" may also be used to refer to a user participating in the game. The term "player" may include the entity participating in the game and its role in the game. For example, "player" may include objects or characters controlled by the user or computer.

[0072] The game device 100 executes game processing in which each player participates, based on at least a portion of the image processing results in the first region 154 corresponding to each player. The first region 154 can also be described as the region of interest for each player in the game processing. Details of the game processing will be described later. The position and size of the first region 154 during the execution of the game processing may be set based on the results of the preparation processing. The first region 154 does not need to be set during the execution of the preparation processing. The first region 154 shown in Figure 4(B) may, for example, represent the first region 154 in a subsequent game processing.

[0073] To guide each user to the horizontal range of the first area 154, a second area 156 corresponding to the first area 154 may be set up. In the following description, when it is necessary to identify each of multiple second areas 156, they will be distinguished by sub-numbers (for example, second areas 156-1, 156-2 as shown in Figure 4).

[0074] When multiple first regions 154 are set up side by side, multiple second regions 156 corresponding to each of the multiple first regions 154 are also set up side by side. In the preparation process, the game device 100 determines that each of the multiple second regions 156 contains at least a part of the user's body. For example, the game device 100 determines whether each of the second regions 156 contains the user's face or upper body from the neck down.

[0075] The game device 100 may terminate the preparation process when the determination result for each of the second areas 156, regarding whether or not at least a part of the user's body is included, satisfies predetermined conditions.

[0076] The second region 156 is configured, for example, to determine that at least a part of each user's body is included in the first region 154. The second region 156 may be configured, for example, near the center of the first region 154. The position and shape of the second region 156 may be determined or changed depending on the shape of the corresponding first region 154 and the number of players participating in the game. The shape of the second region 156 is not limited to a rectangle, but may be a circle, an arbitrary polygon, a discontinuous shape, etc.

[0077] The game device 100 may display a preparation image 160 to guide the user to the area corresponding to the second area 156. For example, the game device 100 may display a preparation image 160 on the display 120 as shown in Figure 4(C).

[0078] The preparation image 160 includes a display camera video 162 based on the captured video 152. The display camera video 162 may be the same as the captured video 152, or it may be an image processed version of the captured video 152.

[0079] The preparation image 160 may be an image in which a guide image 164 showing the second region 156 is placed on the display camera video 162. In the following description, when it is necessary to identify each of the multiple guide images 164, they will be distinguished by adding a sub-number (for example, guide images 164-1, 164-2 as shown in Figure 4).

[0080] For example, guide image 164-1 shows the second region 156-1, and guide image 164-2 shows the second region 156-2. Guide image 164 can be displayed in any form or shape as long as it shows the second region 156. In this embodiment, the second region 156 itself is not displayed. The second region 156 in preparation image 160, shown by a dashed line in Figure 4(C), is for reference only and is not actually displayed. However, the second region 156 may be displayed. The same applies to the first region 154.

[0081] As shown in Figure 4(C), the guide image 164 may be a frame indicating a range. The frame indicated by the guide image 164 may be the same as the outer perimeter of the second region 156, or it may be outside or inside the outer perimeter of the second region 156. For example, it is sufficient that at least a part of the frame indicated by the guide image 164 overlaps with the second region 156.

[0082] The shape of the frame shown in guide image 164 is not limited to a rectangle; it may also be a circle, an arbitrary polygon, a discontinuous shape, etc. The shape of the frame shown in guide image 164 may correspond to the shape of the corresponding second region 156.

[0083] The frame shown in guide image 164 may represent the outline of a face or the outline of a body. Alternatively, guide image 164 may represent an object such as an arrow. In this case, the user may move their face or body to the vicinity of the position indicated by the arrow in guide image 164.

[0084] Figure 5 is a schematic diagram showing an example of a preparation image 160 displayed during the preparation process of a game device 100 according to this embodiment. As shown in Figure 5(A), the guide image 164 may be a frame that roughly shows the outline of the body. As shown in Figure 5(B), the guide image 164 may be a dividing line that shows the boundary between the second region 156-1 and the second region 156-2. As shown in Figure 5(C), the guide image 164 may be an arrow that shows the position where the second region 156 is set.

[0085] Figure 6 is a flowchart showing an example of the processing procedure for the preparation process in step S2 shown in Figure 3. Referring to Figure 6, the game device 100 accepts the number of players to participate in the game (step S21). The game device 100 sets up one or more first areas 154 and one or more second areas 156 according to the accepted number of players (step S22). Note that the setting process in step S22 may involve reading a setting corresponding to the number of players from a set of pre-prepared settings, or it may involve determining the number, position, and range of the first areas 154 and second areas 156 each time according to the number of players.

[0086] The game device 100 acquires camera video 150 captured by camera 118 (step S23) and identifies the outline of a person included in the acquired camera video 150 (step S24). Then, based on the information of the identified person's outline, the game device 100 generates captured video 152 from the camera video 150 (step S25). The game device 100 displays a preparation image 160 including a display camera video 162 based on the captured video 152 and a guide image 164 (step S26).

[0087] The game device 100 determines for each of the second regions 156 whether or not at least a part of the user's body is included in the second region 156 (step S27). The process in step S27 may be a process using the result of the process in step S24, or it may be a determination of whether or not the outline of a person can be identified in each of the second regions 156.

[0088] The game device 100 determines whether the preparation process is complete based on the result of determining whether at least a part of the user's body is included in each of the second areas 156 (step S28). If it is determined that the preparation process is not complete (NO in step S28), the processes from step S23 onward are repeated.

[0089] If it is determined that the preparation process is complete (YES in step S28), the process proceeds to step S3 in Figure 3.

[0090] The processes described in steps S23 to S28 above may be repeated, for example, at predetermined intervals (for example, at processing frame intervals of 1 / 60th of a second).

[0091] As an example of the process for determining whether the preparation process is complete (step S28 shown in Figure 6), if the state in which at least a part of the user's body is included in all of the second regions 156 continues for more than 1 hour, it may be determined that the preparation process is complete.

[0092] Figure 7 is a diagram illustrating an example of the process in step S28 shown in Figure 6. In the preparation image 160 shown in Figure 7(A), guide images 164-1 showing the second region 156-1 and 164-2 showing the second region 156-2 are displayed on the camera video 162. User USR1 is located within the second region 156-1, while user USR2 is located outside the second region 156-2. In other words, Figure 7(A) shows a state in which at least a part of the user's body is included only within the second region 156-1.

[0093] Figure 7(B) shows the state in which user USR2 has moved into the second region 156-2. In this state, at least a part of the user's body is included in all of the second regions 156 (in this example, second regions 156-1 and 156-2). When at least a part of the user's body is included in all of the second regions 156, the game device 100 may display the guide image 164 in a different display mode than in other states in order to inform the user of this state.

[0094] By varying the display mode of the guide image 164, for example, each user can understand that at least a part of their body is included in all of the second regions 156, and that they only need to maintain their current state.

[0095] For example, as shown in Figure 7(B), while the above state continues, the object 166 decorating the frame may be displayed as extending sequentially along the frame indicated by the guide images 164-1 and 164-2, respectively. In this case, the length of the object 166 indicates the duration of the state in which at least a part of the user's body is included in all of the second regions 156. Therefore, by displaying the object 166 so as to complete one circuit of the frame in the first time, the user can easily predict the time until the preparation process is completed from the length of the object 166. In addition, the user can easily understand that the preparation process has not been completed by visually observing that the object 166 does not complete one circuit of the frame or that the object 166 disappears midway.

[0096] The display mode for a state in which at least a part of the user's body is included in all of the second regions 156 is not limited to the object 166 shown in Figure 7(B), but various other display modes may be adopted. For example, a message indicating the state may be displayed, or the shape of the frame shown in the guide image 164, the color of the frame, the thickness of the frame, the pattern of changes in the shape of the frame, the blinking pattern of the frame, etc., may be changed depending on the state.

[0097] The first hour may be a predetermined value, or it may be determined each time depending on the number of players, the number of second areas 156, the size of the second areas 156, etc.

[0098] If each user's horizontal position is determined individually beforehand, inconveniences may arise when readjustments are needed due to positional relationships with other users. Therefore, by having all users who plan to participate in the game agree on their positions and then maintaining those positions for a predetermined period of time to finalize them, adjustments can be made in accordance with the intentions of all users.

[0099] The game device 100 may not be able to pre-determine which side of the camera 118's imaging range 119 the user corresponding to each player is located on. Therefore, the game device 100 may change the combination of correspondence between the first area 154 and the player in response to user operations or other factors.

[0100] For example, in the example shown in Figure 4(B), the first region 154-1, which is set on the left side of the captured video 152, is associated with 1P, and the first region 154-2, which is set on the right side of the captured video 152, is associated with 2P. In this state, the game device 100 may associate the first region 154-1 with 2P and the first region 154-2 with 1P.

[0101] By changing the correspondence between the first area 154 and the player in response to user input, the user can change the corresponding player without changing their current position. Alternatively, the corresponding player can be determined based on the user's position.

[0102] In accordance with changes in such correspondence combinations, the display content of the preparation image 160 may also be changed. Figure 8 is a diagram illustrating the process of changing the correspondence combination between the first region 154 and the player. In the upper state of Figure 8, the second region 156-1, which is set in correspondence with the first region 154-1, and the guide image 164-1 showing the second region 156-1 are associated with 1P. Also, the second region 156-2, which is set in correspondence with the first region 154-2, and the guide image 164-2 showing the second region 156-2 are associated with 2P.

[0103] For example, if any user performs a predetermined operation using the controller, the state may change to the lower state shown in Figure 8. In the lower state of Figure 8, the second region 156-1 and guide image 164-1 are associated with 2P, and the second region 156-2 and guide image 164-2 are associated with 1P.

[0104] Furthermore, the upper and lower states shown in Figure 8 may be switchable between each other. As one implementation example, at least a part of the user's body detected to determine the completion of the preparation process may be the user's face. That is, the condition for the completion of the preparation process may be that the user's face is included in the second region 156-1 shown in guide image 164-1 and the second region 156-2 shown in guide image 164-2. In this case, any face recognition algorithm may be used to determine whether or not the user's face is included in the second region 156.

[0105] The condition for completing the preparation process may be that the entire user's face is included in the second region 156, or that a part of the user's face is included in the second region 156. In the latter case, for example, the condition for completing the preparation process may be that a predetermined percentage of the detected area of ​​the user's face is included in the second region 156.

[0106] In this case, the user's face detection result may be included in the preparation image 160. Figure 9 is a schematic diagram showing another example of the preparation image 160 displayed in the preparation process of the game device 100 according to this embodiment.

[0107] The part of the user's body detected to determine that the preparation process is complete is not limited to the entire user, the upper body of the user, or a predetermined percentage of the user's entire body or upper body.

[0108] In the preparation image 160 shown in Figure 9, a pointer indicating the position of the user's face is placed on the displayed camera video 162.

[0109] In the example shown in Figure 9(A), user USR2 is located outside the second region 156-2, and user USR2's face is also detected outside the second region 156-2. In this case, an out-of-region pointer 168 is displayed around user USR2's face. The out-of-region pointer 168 indicates that the detected face is not included in the second region 156. Thus, an out-of-region pointer 168 indicating the location of the user's face may be displayed when the user's face is not included in either of the second regions 156.

[0110] The user can see an out-of-bounds pointer 168 indicating the position of their face, which lets them know that their face has been detected and that all they need to do is move their face to the appropriate position.

[0111] On the other hand, user USR1 is located within the second region 156-1, and user USR1's face is also detected within the first region 154-1. In this case, an in-region pointer 169 is displayed around user USR1's face. The in-region pointer 169 has a different display pattern than the out-of-region pointer 168, indicating that the detected face is included in the second region 156. Thus, when a user's face is included in any of the second regions 156, the in-region pointer 169 may be displayed in a different display pattern than when a user's face is not included in that second region 156.

[0112] The user can understand that they just need to maintain the current state because the area pointer 169 is displayed.

[0113] Figure 9(B) shows the state in which user USR2 has moved into the second region 156-2. In this state, the pointer displayed around user USR2's face is updated from the out-of-region pointer 168 to the in-region pointer 169. User USR2 can easily understand that it is in the second region 156 because the in-region pointer 169 is displayed.

[0114] Figure 9 illustrates an example where an out-of-area pointer 168 and an in-area pointer 169 are displayed. At least if the out-of-area pointer 168 is displayed, the user can easily understand which direction to move. In this case, the in-area pointer 169 does not need to be displayed. Instead of displaying the in-area pointer 169, if a face is detected within the second area 156, the display mode of the corresponding guide image 164 may be changed. Note that the in-area pointer 169 may be displayed while the out-of-area pointer 168 is not.

[0115] As one implementation example, a partial video 192 is extracted from the captured video 154 corresponding to each first region 154 contained in the captured video 152. The partial video 192 is displayed, for example, in a post-preparation process. As will be described in detail later, the partial video 192 is displayed when at least one of the following processes is executed: game introduction process, specific game pre-preparation process, or game process. The display position of each partial video 192 may be adjusted based on the results of the preparation process. The extraction position of each partial video 192 may be adjusted based on the results of the preparation process. The partial video 192 may include the entire range of the captured video 152.

[0116] Figure 10 is a diagram illustrating the adjustment process of a partial video of a game device 100 according to this embodiment. Referring to Figure 10(A), the game device 100 is said to have detected the faces of users included in the captured video 152 during the preparation process. For example, face regions 190-1 and 190-2, which represent the ranges of the faces of two users, are detected. For example, as shown in Figure 10(A), the game device 100 may calculate the distance Y1 from the top of the captured video 152 to the top of face region 190-1 and the distance Y2 from the top of the captured video 152 to the top of face region 190-2. Note that the game device 100 may detect not only the faces of users, but also, for example, the upper body or feet.

[0117] Figure 10(B) shows an example of a partial video 192 extracted from the captured video 152. In the partial video 192, the user's background is masked, and the in-game background and objects may be displayed around the user. In the example shown in Figure 10(B), the left side of the center in the horizontal direction of the captured video 152 is extracted as partial video 192-1, and the right side is extracted as partial video 192-2. Each partial video 192 may coincide with each first region 154, or it may be a part of each first region 154. In this embodiment, the size of each partial video 192 and its vertical position in the captured video 152 are the same.

[0118] Figure 10(C) is a schematic diagram showing a game introduction image 180, which is an example of an image displayed when the game introduction process of step S3 shown in Figure 3 is executed. The game introduction process is inserted after the preparation process and before the start of at least one game. In the game introduction process, for example, the content of the game to be started or the content of several candidate games that may be started may be explained. Also, for example, by looking at the game introduction image 180, the user can confirm that the preparation process has been completed correctly and prepare to start the game.

[0119] In the game introduction image 180, partial video 192-1 is displayed shifted downward by a distance y compared to partial video 192-2. Distance y is, for example, the difference between the distance Y1 from the top edge of the captured video 152 to the top edge of the face region 190-1 during the preparation process, and the distance Y2 from the top edge of the captured video 152 to the top edge of the face region 190-2. In the partial video 192-1 shown in Figure 10(C), the portion below the game introduction image 180 is not actually displayed. By adjusting the vertical position of the display of partial video 192 in this way, the user experience is improved.

[0120] The method for calculating distance y is not limited. Distance y may be calculated, for example, by multiplying the difference between distance Y1 and distance Y2 by a predetermined coefficient, or it may be calculated based on the ratio of distance Y1 and distance Y2. Furthermore, not only one segment video 192, but the display position of each segment video 192 may be adjusted. For example, the vertical shift amount y1 of segment video 192-1 may be calculated based on distance Y1, and the vertical shift amount y2 of segment video 192-2 may be calculated based on distance Y2. Also, segment video 192 may be displayed shifted upwards.

[0121] The amount of vertical shift in the display position may vary depending on the process being executed, or on the type of game being played. For example, the method for calculating the shift amount may vary depending on the type of game being played.

[0122] Figure 11 is a schematic diagram showing an example of adjusting the position from which a partial video 192 is extracted from the captured video 152. In the example shown in Figure 10, the vertical position of each user displayed was adjusted by adjusting the display position of the extracted partial video 192, for example, as shown in Figure 10(C). In the example shown in Figure 11, the vertical position from which the partial video 192 is extracted is adjusted, thereby adjusting the vertical position of each user when the partial video 192 is displayed.

[0123] As shown in Figure 11, the partial videos 192-1 and 192-2 are extracted from different distances from the upper edge of the captured video 152. For example, the distance from the upper edge of the captured video 152 to the upper edge of the extraction range of partial video 192-1 is y1, and the distance from the upper edge of the captured video 152 to the upper edge of the extraction range of partial video 192-2 is y2. For example, distance y1 is the value obtained by multiplying distance Y1 by a predetermined coefficient, and distance y2 is the value obtained by multiplying distance Y2 by a predetermined coefficient. Note that the method of calculating y1 and y2 is not limited; for example, the ratio of distance y1 to distance y2 may be calculated based on the ratio of distance Y1 to distance Y2.

[0124] The extracted partial videos 192-1 and 192-2 are displayed during subsequent processing. For example, partial videos 192-1 and 192-2 may be included in the game introduction image 180. In this case, the vertical positions of partial videos 192-1 and 192-2 in the game introduction image 180 may be the same. Even if the vertical positions of partial videos 192-1 and 192-2 are the same, the vertical position of each user displayed is adjusted by the difference in the vertical extraction positions of partial videos 192-1 and 192-2 from the captured video 152.

[0125] Thus, the game device 100 may determine the extraction or display position of each partial video 192 in the image during subsequent processing, based on, for example, the position where the user's face is detected in each of the multiple first regions 154. For example, the vertical position of each user's video in the image displayed after the preparation process is adjusted based on the vertical position of the user in each of the multiple first regions detected during the preparation process.

[0126] [E. Pre-game preparation for specific games] The specific game preparation process in step S5 shown in Figure 3 includes, for example, a process to help each user participating in the game adjust their position in the vertical direction of the imaging range 119, or the distance between each user participating in the game and the camera 118.

[0127] Figure 12 is a diagram illustrating the image 170 used for preparing a specific game, which is displayed during the preparation process for a specific game.

[0128] Figure 12(A) is an example of a specific game pre-preparation image 170. The specific game pre-preparation image 170 will be explained in detail later using Figure 13. In the specific game pre-preparation process, the game device 100 determines whether or not each user's face and body (for example, the upper body from the neck down) are included in the third region 158. In the following explanation, when it is necessary to identify each of multiple third regions 158, they are distinguished by sub-numbers (for example, third regions 158-1, 158-2 as shown in Figure 12).

[0129] The game device 100 terminates the pre-game preparation process for a specific game when the determination result for the third region 158 satisfies predetermined conditions. As will be explained later, a guide image 174 showing the third region 158 may be displayed. The third region 158 is set up to determine, for example, that at least a part of each user's body is included in the first region 154, similar to the second region 156. The position and shape of the third region 158 may be determined or changed according to the shape of the corresponding first region 154 and the number of players participating in the game. The third region 158 corresponding to the first region 154 may be set to the same range as the second region 156 corresponding to the same first region 154, or it may be set to a different range.

[0130] As shown in Figure 12(A), the vertical positions of partial videos 192-1 and 192-2 in the specific game pre-preparation image 170 are adjusted based on the preparation process. In the example shown in Figure 12(A), partial video 192-1 is displayed lower in the vertical direction than partial video 192-2. On the other hand, the vertical positions of the third region 158-1 and the third region 158-2 are the same. In this way, by adjusting the display position of each partial video 192 in the specific game pre-preparation process, the need for each user to stand up, squat down, or move closer to or further away from the camera 118 in order to include their face in each third region 158 is reduced, thereby improving the user experience. Note that during the game, which is executed after the specific game pre-preparation process, each partial video 192 may be displayed at the same vertical position as in the specific game pre-preparation image 170.

[0131] Figure 12(B) is a schematic diagram showing the relationship between the captured video 152 and the first region 154 and the second region 158 when the image 170 for preparing a specific game, shown in Figure 12(A), is displayed. As shown in Figure 12(A), partial video 192-1 is shifted downward relative to partial video 192-2, and the lower portion of partial video 192-1 is not included in the image 170 for preparing a specific game. In other words, the lower portion of the body of user USER1 included in the captured video 152 is not used for the specific game preprocessing. Therefore, as shown in Figure 12(B), in the captured video 152, the first region 154-1 does not include the lower portion. On the other hand, the first region 154-2 includes the entire captured video 152 from the top to the bottom.

[0132] In the example shown in Figure 12, the areas indicated by the first region 154 and the partial video 192 were the same, but the areas indicated by the first region 154 and the partial video 192 may be different. For example, the first region 154 may include a wider range of captured video 152 than the partial video 192. In this case, for example, if there is movement of the user's body in the range of captured video 152 that is not displayed as partial video 192 but is included in the first region 154, the game state may change. Also, if the game device 100 can run multiple games, each of the multiple games may have a different vertical position or vertical size of the first region 154 compared to the other games. The games that the game device 100 can run may include multiple games in which the vertical position or size of the first region 154 differs.

[0133] Figure 13 is a schematic diagram showing an example of a specific game preparation image 170 displayed during the specific game preparation process of the game device 100 according to this embodiment. For the sake of explanation, in the example of the specific game preparation image 170 shown in Figure 13, the vertical position of each of the partial video 192 has not been adjusted. Alternatively, in the example of the specific game preparation image 170 shown in Figure 13, the vertical position of each of the partial video 192 has been adjusted, but because USER2 is bending their knees, for example, USER2 may be positioned lower within the imaging range 119 than during the preparation process.

[0134] Referring to Figure 13(A), the image 170 for preparing for a specific game includes a display camera video 172 based on the captured video 152. The image 170 for preparing for a specific game may also include objects related to the selected game. The display camera video 172 may be a video cropped from all or part of the captured video 152, or it may be a video cropped from the captured video 152 with the outline of a person corresponding to each player.

[0135] The image 170 for preparing for a specific game may be an image in which a guide image 174 indicating the third region 158 is placed on the displayed camera video 172. In the following description, when it is necessary to identify each of the multiple guide images 174, they will be distinguished by adding a sub-number (for example, guide images 174-1, 174-2 as shown in Figure 13).

[0136] For example, guide image 174-1 shows the third region 158-1, and guide image 174-2 shows the third region 158-2. Guide image 174 can be any display mode or shape as long as it shows the third region 158. The variations in the display mode and shape of guide image 174 are the same as those for guide image 164 described above, so the same explanation will not be repeated.

[0137] When multiple first regions 154 are set up side by side, multiple third regions 158 corresponding to each of the multiple first regions 154 are also set up side by side. In a specific game pre-preparation process, the multiple first regions 154 may be set up or changed for the selected game before execution.

[0138] For example, as shown in Figure 13(A), if the face of user USR1 (corresponding to 1P in this example) is included in the third region 158-1, the game device 100 may determine that the pre-game preparation process for the player corresponding to the third region 158-1 has been completed. Once it is determined that the pre-game preparation process for the player corresponding to the third region 158-1 has been completed, as shown in Figure 13(B), the game device 100 may, instead of displaying the guide image 174-1 indicating the third region 158-1, display an object 176 that will be displayed during the subsequent game, associated with the player (1P in this example). Note that even if it is determined that the pre-game preparation process for the player corresponding to the third region 158 has been completed, the display of the guide image 174 may continue.

[0139] Meanwhile, user USR2 (corresponding to 2P in this example) looks at the specific game preparation image 170 shown in Figure 13(A) and, upon realizing that their face is not included in the third region 158-2, moves closer to the camera 118, or stands up if they are sitting. Suppose that user USR2 adjusts their position so that their face is included in the third region 158-2 (see, for example, 12(B)). Once user USR2's face is included in the third region 158-2, the game device 100 may determine that the specific game preparation process for the player corresponding to the third region 158-2 is complete. Once it is determined that the specific game preparation process for the player corresponding to the third region 158-2 is complete, as shown in Figure 13(C), the game device 100 may display an object 176 that will be displayed during the subsequent game, associated with the player, instead of the guide image 174-2 showing the third region 158-2.

[0140] Object 176 is the same object that is displayed during the execution of the selected game. The display of Object 176 allows the user to immediately understand that at least a part of their body has been detected. Furthermore, the user can, for example, anticipate the content of the game that is about to begin. Conversely, a user who does not see Object 176 can easily understand that at least a part of their body has not been detected. Note that Object 176 may be a different object from the one displayed during the game's execution.

[0141] In the preparation process, each user adjusts their position in the horizontal direction of the imaging range 119, so it may be necessary to consider the positioning of other users. In contrast, in the pre-game preparation process, it is assumed that users are already in the appropriate position in the horizontal direction, so each user only needs to adjust their position in the vertical direction of the imaging range 119 or their position in the distance direction of the imaging range 119, and therefore the need to consider other users is considered low. For this reason, it is possible to determine whether the pre-game preparation process for the specific game is complete at each stage of the third region 158. Alternatively, it is also possible to determine whether the pre-game preparation process for the specific game is complete at each stage of the third region 158, regardless of whether the preparation process has been performed or not.

[0142] At least a part of the user's body detected to determine the completion of the specific game pre-preparation process may be the user's face. In this case, as explained with reference to Figure 9, a pointer indicating the position of the user's face may be placed on the display camera video 172 (or the specific game pre-preparation image 170). Similarly, the display of out-of-area pointers 168 and in-area pointers 169 can also be applied to the specific game pre-preparation image 170. Furthermore, the various implementation examples of the preparation process described can also be similarly applied to the specific game pre-preparation image 170.

[0143] Figure 14 is a flowchart showing an example of the processing procedure for the specific game pre-preparation process in step S5 shown in Figure 3. Referring to Figure 14, the game device 100 sets up one or more first regions 154 and one or more third regions 158 depending on the selected game (step S51). Note that the setting process in step S51 may be a process of applying pre-prepared settings depending on the game, or it may be a process of determining the position and range of the first region 154 and the third region 158 each time based on the results of the previous preparation process (for example, distances Y1, Y2, etc.).

[0144] The game device 100 acquires camera video 150 captured by camera 118 (step S52) and identifies the outline of a person included in the acquired camera video 150 (step S53). Then, based on the information of the identified person's outline, the game device 100 generates captured video 152 from the camera video 150 (step S54). The game device 100 displays a pre-game preparation image 170, which includes a display camera video 172 based on the captured video 152 (step S55). Depending on the situation, the pre-game preparation image 170 may also include a guide image 174.

[0145] The game device 100 determines whether at least a part of the user's body is included in each of the third regions 158 for which the specific game pre-preparation process has not been completed (step S56). If at least a part of the user's body is included in the third region 158 (YES in step S56), the game device 100 determines that the specific game pre-preparation process for the third region 158 is complete and displays object 176 instead of guide image 174 (step S57). At this time, the game device 100 displays object 176 at a position corresponding to the detected at least part of the user's body.

[0146] Once all the pre-game preparation processes for the third region 158 are complete (YES in step S58), the process proceeds to step S6 in Figure 3.

[0147] The processes described in steps S52 to S57 above may be repeated, for example, at predetermined intervals (for example, at processing frame intervals of 1 / 60th of a second).

[0148] The specific pre-game preparation process allows the user to be guided to an appropriate vertical position or range depending on the game. This guidance improves the user experience in each game. Furthermore, even when the game device 100 can run multiple games, the need to match the vertical position or vertical size between games is reduced, thereby increasing the freedom of game development.

[0149] By first adjusting the horizontal position through a preparation process, and then adjusting the vertical position as needed through a specific pre-game preparation process, adjustments can be made more smoothly compared to adjusting both horizontally and vertically simultaneously, thus improving the user experience. Furthermore, since individualized videos can be used during the period from the preparation process to the start of the actual game, the user experience is further enhanced.

[0150] As another implementation example, if the state in which at least a part of the user's body is included in all third regions 158 continues for more than two hours, it may be determined that the pre-game preparation process for a particular game is complete. In this case, the second hour may be the same as the first hour used in the preparation process, or it may be shorter than the first hour.

[0151] In preparatory processes that require horizontal position adjustment, a longer duration for the state reduces the possibility of the process progressing unintentionally. Conversely, in specific pre-game preparation processes where horizontal position adjustment is considered less necessary, shortening the duration for decision-making allows the pre-game preparation process to be completed earlier and the game to start, thus improving the user experience.

[0152] [F. Game Processing] The game played by the game device 100 according to this embodiment can be any game that can utilize camera video, such as racing games, card games, board games, FPS (First Person Shooter) games, TPS (Third Person Shooter) games, action games, role-playing games, etc.

[0153] A game that utilizes camera video may be a game run on a single game device 100, or it may be a communication-type game that uses internet communication or local communication.

[0154] In the game processing of step S6 or step S7 shown in Figure 3, the game device 100 can execute one or more games that change the game state of the player corresponding to the first region 154 based on at least a part of the user's body included in the first region 154.

[0155] As described above, when multiple first regions 154 are set, the game device 100 can execute one or more games that change the game state of multiple players corresponding to each of the multiple first regions 154 based on at least a part of the user's body contained in each of the multiple first regions 154.

[0156] A player's game state includes various states and values ​​related to the player participating in the game. Changing a player's game state may include, for example, changing various parameters related to the player (e.g., score or hit points), affecting the object corresponding to the player, and putting the player into a state where they can attack other players. Affecting the object corresponding to the player may include, for example, changing the in-game behavior of other objects, changing the parameters of other objects, changing the position and display of the avatar representing the player, and changing parts of the player's object.

[0157] Based on at least a part of the user's body included in the first region 154 may include, for example, detecting the user's face or upper body and performing some processing based on the detection result, performing some processing based on changes in the part corresponding to the user's body, and performing some processing based on whether or not a part corresponding to the user's body exists.

[0158] The game device 100 may perform game processing based not only on information about at least a part of the user's body contained in the first region 154, but also on other information obtained from the first region 154.

[0159] Figure 15 is a schematic diagram showing an example of game processing performed by a game device 100 according to this embodiment. The game processing includes processing that utilizes camera video.

[0160] Figure 15(A) is an example of a game image 200 of a game executed following the specific game pre-preparation process shown in Figure 14. The game image 200 includes in-game camera videos 202-1 and 202-2 based on captured video 152. The in-game camera video 202 is, for example, a video from a partial video 192 in which all but the user's face are masked. The in-game camera video 202 may differ for each game being executed. In the game shown in Figure 15(A), the game device 100 progresses the game based on the position and orientation of the in-game camera videos 202-1 and 202-2 showing the user's face.

[0161] Figure 15(A) shows an example of a game where the user adjusts the position and posture of their face to catch objects falling from the top to the bottom of game image 200. For example, the falling objects can stack on top of an object placed above the user's face. Each player, corresponding to each user, competes to see who can stack the most objects. The display position and posture of in-game image 200, as well as the state and number of stacked objects, are examples of each player's game state.

[0162] Figure 15(B) shows an example of game images 200 from another game. Game images 200 include in-game camera videos 202-1 and 202-2 based on captured video 152. In the game shown in Figure 15(B), in-game camera videos 202-1 and 202-2 show the body of each user, including the upper body, for example, the whole body.

[0163] Figure 15(B) shows an example of a game where the player moves their arms or other body parts to prevent an object representing a ball falling from the top to the bottom of the game image 200 from falling. The display position and orientation of the in-game image 200, as well as the position, direction of movement, and speed of the ball, are examples of the game state for each player.

[0164] [G. Online Games] The processing according to this embodiment can also be applied to communication-type games that use internet communication or local communication. Below, an example of processing for a game that communicates with another game device 100 will be described.

[0165] For example, by having multiple game devices 100 communicate with each other, a communication-type game in which multiple players can participate is provided. The configuration may involve one player participating via one game device 100, or it may be a configuration in which multiple players can participate via one game device 100.

[0166] As an example, suppose player 1 is participating in a communication-type game from game device 100-1, and player 2 is participating from game device 100-2. In this case, a game image 200 is generated in both game device 100-1 and game device 100-2. Alternatively, the game image 200 may be generated in one game device 100 and then transmitted to the other game device 100. A video feed from an in-game camera 202 is used in communication-type games.

[0167] At this time, game device 100-1 generates, for example, an in-game camera video 202 corresponding to player 1 from the captured video 152. Game device 100-2 similarly generates an in-game camera video 202 corresponding to player 2 from the captured video 152. Then, game devices 100-1 and 100-2 each generate a game image 200 using the in-game camera videos 202 corresponding to players 1 and 2, respectively, based on the in-game camera videos 202 they themselves generated and the in-game camera videos 202 received from the other game device 100. In this way, when running a communication-type game, game control information and in-game camera videos 202 generated from the captured video 152 may be transmitted and received between multiple game devices 100, and a game image 200 may be generated in each game device 100.

[0168] As another example, when running a communication-type game, the captured video 152 may be transmitted and received as is, and the game device 100 that receives the captured video 152 may generate an in-game camera video 202. For example, when a game program 126, which is a communication-type game via the internet, is run together with a chat application 114, the running game program 126 may acquire captured video from another game device 100 via the running chat application 114. Hereinafter, the captured video acquired from another game device 100 will also be referred to as the "received captured video". The received captured video may be included in the communication data 10. The running game program 126 may generate an in-game camera video 202 corresponding to the player of the other game device 100 from the received captured video acquired from the other game device 100. Note that it may not be possible to acquire the received captured video from a game device 100 that does not belong to the same chat group.

[0169] The received captured video corresponds to the captured video 152 based on the camera video 150 captured by the camera 118 acquired by the other game device 100, and may be the same as the captured video 152 acquired by the other game device 100, or it may be the result of some processing.

[0170] The running game program 126 may acquire a received camera video and one or more positional pieces of information (for example, the distance from the top of the captured video 152 shown in Figure 10 to the top of the face region 190) from another game device 100, and use the acquired one or more positional pieces of information to generate an in-game camera video 202 based on the captured video received from the other game device 100.

[0171] In this embodiment, the received captured video is received via the communication path used by the chat application 114, and the location information is received via the communication path used by the game program 126. The game program 126 also transmits at least one or more location information to another game device 100.

[0172] Thus, the game device 100 may display in-game camera video, generated from received captured video, at a position in the game image corresponding to the player of the other game device 100, based on position information received from another game device 100. This can reduce the amount of data transmitted compared to when both the received captured video and the in-game camera video are transmitted and received. Furthermore, since the amount of data for position information may be smaller than that for received captured video, the amount of data flowing through the communication path used by the game program 126 can be reduced.

[0173] Furthermore, even when the chat application 114 is not running and the game program 126 is running, the game device 100 may receive captured video and location information from other game devices 100, generate in-game camera video from the camera video, and use it in the game. [H. Other forms] The example given illustrates the processing when multiple users are within the imaging range 119 of camera 118. However, the same processing may be performed even when only one user is within the imaging range 119, or when only one user within the imaging range 119 is participating in the game. In this case, the captured video 152 may have one first region 154 set, and one second region 156 may be set corresponding to one first region 154. Furthermore, the preparation image 160 displayed during the preparation process may include a display camera video 162 in which one guide image 164 indicating one second region 156 is placed.

[0174] Figure 16 shows an example of a game image 200 when the game shown in Figure 15(A) is played by two users and two non-player objects (hereinafter also referred to as "NPCs"). The two users and the two NPCs are each in a team. In this way, by displaying and competing with the users and NPCs, users can obtain a user experience that makes the NPCs seem as if they are actually alive, and a user experience that makes the user feel as if they have entered the game world themselves.

[0175] As shown in Figure 16, both User USR1 and User USR2 are displayed to one side of the horizontal center. Therefore, during the preparation process or the pre-game preparation process for a particular game, User USR1 and User USR2 may be guided by the guide image 164 to be positioned to one side of the horizontal center of the camera video 150 or the preparation image 160. Alternatively, during the preparation process or the pre-game preparation process for a particular game, User USR1 and User USR2 may be guided by the guide image 164 to be positioned to one side and the other side of the horizontal center of the camera video 150 or the preparation image 160, respectively. In this case, by adjusting the display position of each user's partial video 192 extracted from the captured video 152, both User USR1 and User USR2 may be displayed to one side of the horizontal center, as shown in Figure 16, for example.

[0176] In the above explanation, an example was shown in which the vertical display position of the partial video 192 is adjusted in the game introduction image 180, the specific game preparation image 170, and the game image 200. However, the vertical display position of the partial video 192 may be adjusted in only some of these images. For example, the vertical display position of the partial video 192 may not be adjusted in the game introduction image 180, but may be adjusted in the specific game preparation image 170 and the game image 200. For example, in the game introduction process, if each user is simply displayed as if they were in a virtual space, the user experience may be better if the height differences of each user in the real world and their distance from the camera 118 are also reflected in the game introduction image 180.

[0177] In the above description, an example was given of a process in which a specific game pre-preparation process is executed when a specific game is selected. However, the specific game pre-preparation process may be omitted regardless of which game is selected. In other words, the specific game pre-preparation process does not necessarily have to be implemented. Also, in the above description, an example was given of a process in which a specific game pre-preparation process is executed when a specific game is selected after the preparation process has been executed. However, the preparation process does not necessarily have to be executed. For example, there may be one game to be played by the game device 100, and preparation processing may be executed before that one game is played. Alternatively, there may be multiple games to be played by the game device 100, and preparation processing may be executed before each game is played. In this case, the preparation processing, or the specific game pre-preparation process, may be considered as "preparation processing".

[0178] In the above description, the game device 100 was capable of running one or more games. This may also mean that the game device 100 is capable of running one or more game programs. It may also mean that one game program run by the game device 100 contains multiple games. The multiple games contained in one game program may, for example, each have different rules. For example, the multiple games may be multiple mini-games, in which case the winner and loser between players may be determined in each mini-game, and the final winner may be determined based on the results of the multiple mini-games.

[0179] In-game camera video may also be available in applications other than game programs. For example, in-game camera video can be used in video conferencing applications.

[0180] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0181] 4 Network, 10 Communication Data, 100 Game Device, 102 Processor, 104 Volatile Memory, 106 Network Interface, 108 Camera Interface, 110 Non-Volatile Memory, 112 System Program, 114 Chat Application, 116 Local Communication Unit, 118 Camera, 119 Imaging Range, 120 Display, 122 Microphone, 124 Speaker, 126 Game Program, 150 Camera Video, 152 Captured Video, 154 First Area, 156 Second Area, 158 Third Area, 160 Preparation Image, 162, 172 Display Camera Video, 164, 174 Guide Image, 166, 176 Object, 168 Out-of-Area Pointer, 169 In-Area Pointer, 170 Pre-Preparation Image, 180 Game Introduction Image, 190 Face Area, 192 Partial Video, 200 Game Image, 202 In-game camera video, 300 management servers, 350 relay servers, USR1, USR2 users, Y1, Y2 distance.

Claims

1. A game program that causes a computer to perform game processing, wherein the computer, The system acquires video footage based on the video footage captured by the camera. With respect to a plurality of first regions arranged horizontally within the image of the captured video, one or more games are executed that change the game state of a plurality of players corresponding to each of the plurality of first regions, based on at least a part of the user's body contained in each of the plurality of first regions. Before running one or more of the aforementioned games, perform a preparatory process. The aforementioned preparation process is A preparation image is displayed on the camera video based on the aforementioned captured video, in which multiple guide images representing multiple second regions arranged horizontally and corresponding to each of the multiple first regions are placed. A game program that includes determining whether each of the plurality of second regions contains at least a part of the user's body.

2. The game includes a plurality of games in which at least the vertical position or vertical size of the first region differs, The game program causes the computer to perform a specific pre-game preparation process after the preparation process and before at least one of the one or more games is executed. The aforementioned pre-game preparation process for a specific game is: Displaying a pre-game preparation image for a specific game, wherein a plurality of second guide images are arranged on the display camera video based on the aforementioned captured video, each showing a plurality of horizontally arranged third regions corresponding to the plurality of first regions in at least one game before execution; The game program according to claim 1, further comprising determining that each of the third regions includes at least a part of the user's body.

3. The preparation process includes determining whether the state in which at least a part of the user's body is included in all of the plurality of second regions continues for more than a first hour. The game program according to claim 2, wherein the specific pre-game preparation process includes determining that at least a part of the user's body is included in all of the plurality of third regions, or determining that the state in which at least a part of the user's body is included in all of the plurality of third regions continues for a second time that is shorter than the first time.

4. The game program according to claim 2, wherein the specified game pre-preparation process includes, when it is determined that at least a part of the user's body is included in any of the plurality of third regions, displaying the same object that is displayed during the execution of at least one game prior to execution at a position corresponding to at least a part of the user's body included in the third region.

5. The game program according to claim 1, wherein the preparation process includes determining that the state in which at least a part of the user's body is included in all of the plurality of second regions continues for more than a first hour.

6. The game program according to claim 5, wherein the preparation process includes displaying the guide image in a different display manner than in other cases when at least a part of the user's body is included in all of the plurality of second regions.

7. The game program according to claim 1, wherein the preparation process includes changing the combination of correspondences between the plurality of first regions and the plurality of players in response to user operations.

8. The game includes a plurality of games in which at least the vertical position or vertical size of the first region differs, The game program causes the computer to perform a specific pre-game preparation process after the preparation process and before at least one of the one or more games is executed. The aforementioned pre-game preparation process for a specific game is: Displaying a pre-game preparation image for a specific game, wherein a plurality of second guide images are arranged on the display camera video based on the aforementioned captured video, each showing a plurality of horizontally arranged third regions corresponding to the plurality of first regions in at least one game before execution; This includes determining that each of the three regions includes at least a part of the user's body, The game program according to claim 7, wherein the game program causes the computer to display an image in which a plurality of partial videos, each corresponding to a plurality of first regions extracted from the captured video, are positioned according to the modified combination of correspondences, after the preparation process and before at least one of the one or more games is executed.

9. The aforementioned game program, which the computer, Based on at least a portion of the user's body contained in one first region within the image of the captured video, one or more games are executed that change the game state of the player corresponding to the one first region. The aforementioned preparation process is A preparatory image is displayed on the camera video based on the captured video, in which a guide image indicating a second region corresponding to the first region is placed. The game program according to claim 1, further comprising determining that at least a part of the user's body is included in the one second region.

10. The game program according to claim 1, wherein at least a part of the user's body is the user's face.

11. The game program according to claim 1, wherein the preparation process includes displaying a pointer indicating the position of the user's face when the user's face is not included in any of the plurality of second regions.

12. The game program according to claim 11, wherein the preparation process includes displaying the pointer in a different display manner when the user's face is included in any of the plurality of second regions than when the user's face is not included in the second region.

13. The game program according to claim 1, further comprising determining the vertical position of each user's video in an image displayed after the preparation process, based on the vertical position of the user in each of the plurality of first regions detected in the preparation process.

14. The game program according to claim 1, further comprising determining the vertical position of a portion of a video in an image displayed after the preparation process, based on the vertical position of the user in each of the plurality of first regions detected in the preparation process, the position of a portion of a video corresponding to each of the plurality of first regions.

15. A method performed by a computer, wherein the computer Acquire video footage based on camera video captured by the camera, With respect to a plurality of first regions arranged horizontally within the image of the captured video, one or more games are executed that change the game state of a plurality of players corresponding to each of the plurality of first regions based on at least a part of the user's body contained in each of the plurality of first regions. Before running one or more of the aforementioned games, perform a preparatory process. The aforementioned preparation process is A preparation image is displayed on the camera video based on the aforementioned captured video, in which multiple guide images representing multiple second regions arranged horizontally and corresponding to each of the multiple first regions are placed. A method comprising determining that each of the plurality of second regions includes at least a part of the user's body.

16. A system capable of executing game programs, Acquire video footage based on camera video captured by the camera, With respect to a plurality of first regions arranged horizontally within the image of the captured video, one or more games are executed that change the game state of a plurality of players corresponding to each of the plurality of first regions based on at least a part of the user's body contained in each of the plurality of first regions. Before running one or more of the aforementioned games, perform a preparatory process. The aforementioned preparation process is A preparation image is displayed on the camera video based on the aforementioned captured video, in which multiple guide images representing multiple second regions arranged horizontally and corresponding to each of the multiple first regions are placed. A system that includes determining whether each of the plurality of second regions contains at least a part of the user's body.

Images