Program and information processing system

Abstract

To enhance relationship between a virtual space and an actual space.SOLUTION: A program causes a computer to function as display control means for simultaneously displaying a photographed image of an actual world and a virtual object and variation means which varies the appearance of the virtual object itself to an appearance reflecting the characteristics of a specific object included in the photographed image displayed simultaneously with the virtual object. The appearance of the virtual object after the variation by the variation means is kept even after the photographed image including the specific object is no longer displayed simultaneously with the virtual object.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a program and an information processing system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been known a service that combines a virtual space, such as a game using augmented reality (AR), with a real space (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-161006 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in services that combine virtual space with real space, there is still a problem in that a gap exists between the virtual space and the real space.

[0005] The present invention aims to strengthen the relationship between virtual space and real space. [Means for solving the problem]

[0006] According to one embodiment shown in the present disclosure, The processor, a display control means for simultaneously displaying a captured image of the real world and a virtual object; a control means for varying a specific parameter based on the captured image displayed simultaneously with the virtual object; The specific parameter is at least one of a parameter of the virtual object and a parameter indicating an amount of a predetermined value owned by a user who owns the virtual object. The program is provided. Effect of the Invention

[0007] According to the present invention, it is possible to strengthen the relationship between the virtual space and the real space. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of an information processing system. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the information processing system. [Diagram 3] FIG. 11 is a diagram showing an example of a screen on which a captured image and a virtual object are simultaneously displayed. [Figure 4] 10 is a flowchart illustrating an example of a process related to a variation of a specific parameter based on a captured image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] <System hardware configuration> As shown in FIG. 1, an information processing system 1 of this embodiment includes a plurality of terminal devices 10 and a server 20.

[0011] The terminal device 10 and the server 20 are connected via a network 2. The network 2 may be configured by, for example, the Internet, a mobile communication system (e.g., 3G, 4G, 5G, LTE (Long Term Evolution), etc.), WiFi (Wireless Fidelity), Bluetooth (registered trademark), other communication lines, or a combination of these. In addition, the connection between the terminal device 10 and the server 20 may be a wired connection or a wireless connection.

[0012] The server 20 (in other words, a computer, an information processing device) may be, for example, a general-purpose computer such as a workstation or a personal computer. The server 20 includes a processor 21, a memory 22, a storage 23, a communication IF (interface) 24, and an input / output IF 25. These components included in the server 20 are connected to each other by a communication bus.

[0013] The processor 21 controls the overall operation of the server 20. The processor 21 may include a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc. The processor 21 reads a program from the storage 23 and loads it in the memory 22. The processor 21 executes the loaded program.

[0014] The memory 22 is a main storage device. The memory 22 is composed of storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The memory 22 temporarily stores the programs and various data read by the processor 21 from the storage 23, thereby providing a working area for the processor 21. The memory 22 also temporarily stores various data generated while the processor 21 is operating according to the programs.

[0015] In this embodiment, the program may be a program that realizes a game by the terminal device 10. The program may also be a program that realizes the game through cooperation between the terminal device 10 and the server 20. The game realized through cooperation between the terminal device 10 and the server 20 may be, for example, a game executed on a browser launched in the terminal device 10. The program may also be a program that realizes the game through cooperation between a plurality of terminal devices 10. The various data include, for example, data related to the game, such as user information and game information, and instructions and notifications transmitted and received between the terminal device 10 and the server 20.

[0016] The storage 23 is an auxiliary storage device. The storage 23 is configured with a storage device such as a flash memory or a hard disk drive (HDD), etc. The storage 23 stores various data related to the game.

[0017] The communication IF 24 controls transmission and reception of various data between the server 20 and the terminal device 10 etc. via the network.

[0018] The input / output IF 25 is an interface through which the server 20 receives input of data and also an interface through which the server 20 outputs data. The input / output IF 25 may include, for example, an input unit that is an information input device such as a mouse or a keyboard, and a display unit that is a device that displays and outputs an image.

[0019] The terminal device 10 (in other words, a computer, an information processing device) may be, for example, a smartphone, a feature phone, a PDA (Personal Digital Assistant), a tablet computer, a personal computer, a wearable terminal, or a game device. The terminal device 10 may be a mobile terminal. The terminal device 10 may be a portable terminal that a user uses when playing a game.

[0020] The terminal device 10 includes a processor 11, a memory 12, a storage 13, a communication IF 14, an input / output IF 15, an input unit 17, a display unit 18, and an image capture unit 19. These components included in the terminal device 10 are connected to each other via a communication bus.

[0021] The processor 11 controls the overall operation of the terminal device 10. The processor 11 may include a CPU, an MPU, a GPU, etc. The processor 11 reads a program from the storage 13 and loads it in the memory 12. The processor 11 executes the loaded program.

[0022] The memory 12 is a main storage device. The memory 12 is composed of storage devices such as a ROM and a RAM. The memory 12 temporarily stores the programs and various data that the processor 11 reads from the storage 13, thereby providing a working area for the processor 11. The memory 12 also temporarily stores various data that the processor 11 generates while it is operating according to the programs.

[0023] The storage 13 is an auxiliary storage device. The storage 13 is configured with a storage device such as a flash memory or a HDD. The storage 13 stores various data related to the game.

[0024] The communication IF 14 controls transmission and reception of various data between the terminal device 10 and the server 20 etc. via the network.

[0025] The input / output IF 15 is an interface through which the terminal device 10 receives input of data and also an interface through which the terminal device 10 outputs data. The input / output IF 15 may input and output data via, for example, a Universal Serial Bus (USB) or the like. The input / output IF 15 may include an input unit 17, a display unit 18, or the like.

[0026] The input unit 17 accepts input from a user. The input unit 17 may be, for example, a pointing device such as a touch pad. The display unit 18 displays an image. The display unit 18 may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The terminal device 10 includes, for example, a touch screen 16 which is an electronic component combining the input unit 17 and the display unit 18.

[0027] The input unit 17 has a function of detecting a position input on the input surface by a user operation (for example, a touch operation, a tap operation, a slide operation, a swipe operation, a flick operation, etc.) and transmitting information indicating the detected position as an input signal. The touch panel as the input unit 17 may be of a capacitive type or a resistive film type, or may be of another type.

[0028] The input unit 17 may be, for example, a keyboard, various physical buttons, various sensors (for example, an acceleration sensor, an angular velocity sensor, a motion sensor, a gaze sensor, a bioelectric potential sensor, a fingerprint sensor, a breath sensor, a pressure sensor, an image sensor, or the like), an operation stick, a camera, a microphone, etc. The display unit 18 may be, for example, a projector, etc.

[0029] The image capturing unit 19 is a camera that captures an image of the real world around the terminal device 10. The image capturing unit 19 may be included in the input unit 17.

[0030] <System Functional Configuration> 2 is a block diagram showing the functional configuration of the server 20 and the terminal device 10. The server 20 in this embodiment has, for example, a function of providing each of the terminal devices 10 with various data and programs required to realize a game, a function of collecting and managing data related to the game from each of the terminal devices 10, and a function of performing synchronization processing between the multiple terminal devices 10.

[0031] In this embodiment, the server 20 identifies each user and the terminal device 10 using a user account that is registered in advance for each game. The method of registering the account is not particularly limited. For example, the terminal device 10 or another device such as a personal computer may transmit information required for registering a user's account to the server 20 based on a user's operation, and the server 20 may create and store an account for each user based on the received information.

[0032] 2, the server 20 functions as a control unit 210 and a storage unit 220 through cooperation of a processor 21, a memory 22, a storage 23, a communication IF (interface) 24, an input / output IF 25, etc. The storage unit 220 stores various data used by the control unit 210. The various data include, for example, a game program, game information, and user information.

[0033] The game program is a program for implementing the game. The game information and user information are data that the control unit 210 refers to when executing the game program.

[0034] In this embodiment, the game program may include a game program executed on the terminal device 10 side, in addition to a game program executed on the server 20 side. Furthermore, the storage unit 220 may store a game program executed on the server 20 side and a game program executed on the terminal device 10 side.

[0035] The game information is information common between accounts. The game information includes, for example, information for defining various virtual spaces. The virtual space is, for example, a space in which various objects (in other words, virtual objects) such as character objects owned by a user are arranged. In other words, game media (character objects, etc.) used in the game are arranged in the virtual space. The game information includes, for example, various setting information related to objects common between accounts, such as background objects such as buildings, trees, and stones arranged in the virtual space and the arrangement positions, sizes, colors, and shapes of non-player character (NPC) objects. The game information also includes, for example, setting values ​​of various parameters of NPCs. Hereinafter, the object of a character arranged in a virtual space may be simply referred to as a "character".

[0036] User information is information managed for each game account. User information includes, for example, information on characters owned by the user, information on owned assets, and information indicating the progress of the game. Examples of owned assets include currency and items in a virtual space. In other words, owned assets can be said to be value owned by a user in a virtual space (in other words, value associated with a user). An example of such value (in other words, value in a virtual space) is electronic currency. Examples of electronic currency include in-game currency that can be used in a game and cryptocurrency.

[0037] The control unit 210 controls various processes related to the game by executing a game program stored in the storage unit 220. The control unit 210 has, for example, a transmission / reception unit 211, a server processing unit 212, a data management unit 213, and a synchronization processing unit 214.

[0038] The transmitting / receiving unit 211 transmits or receives various data. For example, the transmitting / receiving unit 211 receives requests to transmit various data and programs, requests for synchronization processing to support the multiplay function, data to be subjected to synchronization processing, and the like from each terminal device 10, and passes them to the server processing unit 212. In addition, the transmitting / receiving unit 211 transmits various data and programs including instructions for synchronization to each terminal device 10 according to the control of the server processing unit 212.

[0039] In this embodiment, the multiplay function is a function that synchronizes game processing by multiple accounts. When multiple accounts logged in to the information processing system 1 participate in the same game, the server 20 and the terminal device 10 of the information processing system 1 execute various processes to support the multiplay function.

[0040] The server processing unit 212 provides a game to the terminal device 10 by executing arithmetic processing described in a game program in response to a request from the terminal device 10. For example, when the server processing unit 212 receives a request for synchronization processing to support the multiplay function or data to be the subject of synchronization processing from the terminal device 10 via the transmission / reception unit 211, the server processing unit 212 executes synchronization processing to support the multiplay function. In addition, the server processing unit 212 issues a command to the transmission / reception unit 211 to transmit game information or user information. Furthermore, the server processing unit 212 issues a command to the data management unit 213 to add, update, or delete records of the game information or user information.

[0041] The data management unit 213 manages various data stored in the storage unit 220 in accordance with instructions from the server processing unit 212. For example, the data management unit 213 reads game information or user information in accordance with instructions from the server processing unit 212, and transmits it to the terminal device 10 via the transmission / reception unit 211. In addition, the data management unit 213 adds, updates, or deletes records of the game information or user information in accordance with instructions from the server processing unit 212.

[0042] The synchronization processing unit 214 executes synchronization processing to support the multi-play function of the game according to an instruction from the server processing unit 212. For example, when the server 20 transmits information to a plurality of terminal devices 10, the synchronization processing unit 214 transmits information to each terminal device 10 at the same time, thereby synchronizing the game proceeding among the terminal devices 10. Specifically, the synchronization processing unit 214 transmits operation information received from the terminal device 10 corresponding to each account within a predetermined period (for example, one frame) to each terminal device 10 at the same time for each predetermined period. The operation information is information related to an operation input to the terminal device 10. The timing of synchronization and information to be synchronized may be received from the server processing unit 212 at any time. By executing the synchronization processing, it becomes possible to simultaneously reflect an event in the game caused by an operation input in one terminal device 10 in the other terminal devices 10.

[0043] The terminal device 10 in this embodiment has, for example, a function as an input device that accepts input operations by a user, and a function as an output device that outputs images and sounds of a game.

[0044] The terminal device 10 functions as a control unit 110 and a storage unit 120 through cooperation of the processor 11, memory 12, storage 13, communication IF 14, input / output IF 15, etc. The storage unit 120 stores various data used by the control unit 110. The various data include, for example, a game program, game information, and user information. The game program is a program for implementing a game on the terminal device 10 side. The game information and user information are data referenced when the control unit 110 executes the game program. The game information and user information stored in the storage unit 120 may include information similar to the game information and user information stored in the storage unit 220.

[0045] The control unit 110 executes a game program stored in the memory unit 120, thereby controlling various processes related to the game executed on the terminal device 10. The control unit 110 has, for example, an operation reception unit 111, a transmission / reception unit 112, a game progression unit 113, a display control unit 114, and a detection unit 115.

[0046] The operation reception unit 111 receives an operation (hereinafter, also referred to as an "input operation") input by the user via the input unit 17. Specifically, when an input operation is performed on the input unit 17, the operation reception unit 111 detects the coordinates of the input position and the type of the input operation. Examples of the types of input operations include various operations using fingers, such as a touch operation, a tap operation, a slide operation, a swipe operation, a flick operation, a pinch-in operation, and a pinch-out operation. The input operation is not limited to an operation of physically contacting the input unit 17 (for example, the touch screen 16), but may also include a non-contact operation. Note that an operation of ending a previously performed input operation, such as a touch-off operation of ending contact with the touch screen 16, can also be considered as one aspect of the input operation.

[0047] Here, the operation reception unit 111 can also receive an input operation performed using an operation device connected via the input / output IF 15 in the same manner as an input operation to the input unit 17 .

[0048] The transmission / reception unit 112 transmits or receives various data. For example, the transmission / reception unit 112 transmits game information or user information, or a synchronization request for supporting the multiplay function to the server 20. The transmission / reception unit 112 receives various data, programs, data for synchronization for supporting the multiplay function, and the like from the server 20. The data for synchronization includes, for example, synchronization instruction data for instructing each of the terminal devices 10 participating in the multiplay to synchronize. The synchronization instruction data includes, for example, data to be synchronized, the type of the data, data for specifying the time of synchronization, and the like.

[0049] The transmitting / receiving unit 112 transmits operation information related to the input operation accepted by the operation accepting unit 111 to the server 20. The transmitting / receiving unit 112 receives, from the server 20, operation information related to an operation input by another user in another terminal device 10.

[0050] The game progression unit 113 executes various processes related to the progression of the game. A specific example will be explained below.

[0051] The game progression unit 113 defines a virtual space based on information for defining the virtual space, which is included in the game information. The game progression unit 113 places objects in the virtual space based on setting information for the objects, which is included in the game information. The game progression unit 113 controls the objects placed in the virtual space. Specifically, the game progression unit 113 changes the position, orientation, shape, color, etc. of the objects in the virtual space, and causes the objects to perform predetermined actions.

[0052] The game progression unit 113 defines a virtual camera for specifying an area of ​​the virtual space to be presented to the user. The game progression unit 113 places the virtual camera in the virtual space by defining the position and orientation of the virtual camera in the virtual space. The game progression unit 113 instructs the display control unit 114 to generate an image that depicts the field of view defined by the virtual camera and the objects placed in this field of view.

[0053] The position and orientation of the virtual camera can be appropriately determined for each virtual space. For example, the game progression unit 113 uses the position and orientation of a specific object as a reference and positions the virtual camera so that the specific object is located in the center of the field of view in a specific orientation. At that time, the game progression unit 113 adjusts the position and orientation of the virtual camera using the direction, distance, and angle relative to the specific object. The specific object may be, for example, a dynamic object or a static object such as a building, tree, stone, etc. An example of a dynamic object is a character object.

[0054] The game progression unit 113 interprets the user's instructions based on the coordinates of the input position and the type of input operation detected by the operation reception unit 111. The game progression unit 113 executes various determination processes related to the progress of the game based on the interpreted instructions, etc. The game progression unit 113 progresses the game while controlling objects, a virtual camera, etc. based on the results of the determination processes, etc. The game progression unit 113 updates, adds, or deletes game information and user information according to the progress of the game.

[0055] The display control unit 114 causes the display unit 18 to display an image. A specific example will be described below.

[0056] The display control unit 114 generates an image that depicts the area of ​​the field of view of the virtual camera defined by the game progression unit 113 in the virtual space and objects present in that area, and displays it on the display unit 18. The display control unit 114 can superimpose and draw objects related to a UI (User Interface) required for various operations of the game, such as icons, buttons, and menus showing various parameters, on the image to be displayed on the display unit 18.

[0057] The functions of the terminal device 10 and the server 20 shown in FIG. 2 are merely an example. Each of the terminal device 10 and the server 20 may have at least a part of the functions of the other device. Each of the devices such as the terminal device 10 and the server 20 may not be realized by an integrated device, and may be realized by, for example, a plurality of devices connected via a network or the like. The information processing system 1 may be configured by, for example, only the terminal device 10 or the server 20. In other words, the information processing system 1 may not be realized by a plurality of devices connected via a network.

[0058] <Processing according to this embodiment> Next, the processing according to this embodiment will be described. In this embodiment, the processor 11 of the terminal device 10 or the processor 21 of the server 20 executes the game program stored in the information processing system 1 to perform each processing described later. However, at least a part of the processing described later and performed by the processor 11 may be executed by a processor other than the processor 11. Also, at least a part of the processing described later and performed by the processor 21 may be executed by a processor other than the processor 21. In other words, the computer that executes the game program in this embodiment may be either the terminal device 10 or the server 20, or may be realized by a combination of multiple devices.

[0059] In the game of this embodiment, the user is able to train a character 30 owned by the user (in other words, a character corresponding to the user). The character 30 may be a character capable of competing against a specified opponent. Here, the "opponent" may be a character owned by another user, or an NPC (in other words, an enemy character controlled by a computer), etc. The character 30 may also be something like a pet. In other words, the character 30 may be something that is not trained for competing, but is instead something that is kept as a pet.

[0060] Moreover, "development" can also be said to mean changing the parameters of the character 30. The change in the parameters of the character 30 will be described later.

[0061] In the game of this embodiment, the display control unit 114 performs control to simultaneously display a photographed image of the real world photographed by the photographing unit 19 and a virtual object (for example, a character 30) on the display unit 18, as shown in FIG. 3. Specifically, the display control unit 114 displays a superimposed image (in other words, an augmented reality image (AR image)) in which the character 30 is superimposed on the photographed image on the display unit 18, thereby simultaneously displaying the photographed image and the character 30. In other words, the display control unit 114 simultaneously displays a virtual object and a real-world object (hereinafter, referred to as a "real object") 32 reflected in the photographed image. In FIG. 3, the displayed objects other than the character 30 (specifically, a banana, a plate, and a desk) are real objects. In addition, the display control unit 114 may simultaneously display the photographed image and the character 30 by displaying the character 30 side by side on the display unit 18 without superimposing the character 30 on the photographed image (for example, by displaying the character and the photographed image separately in each area divided into left and right). That is, "at the same time" means that the character 30 and the captured image are displayed on the same screen, and that at least a part of the original image captured by the camera serving as the image capturing unit 19 is shown (in other words, a part of the captured image may be hidden by the character 30, etc.). That is, "displaying a superimposed image" in the following description can be appropriately interpreted as displaying the captured image and the virtual object simultaneously.

[0062] In addition, in the game of this embodiment, the photographing unit 19 photographs the photographed image and displays the superimposed image in real time. Specifically, an image of the part over which the camera of the terminal device 10 is held is displayed on the display unit 18 as a photographed image, and the character 30 is displayed on the display unit 18 superimposed on the photographed image. That is, in this embodiment, "photographing (in other words, acquiring a photographed image)" is not limited to acquiring an image by pressing a predetermined button or the like to release the shutter, but may also include holding the camera over the real object 32. In addition, the photographed image may be a video or a still image.

[0063] In this embodiment, a virtual object (e.g., character 30) displayed simultaneously with the photographed image can move on the screen. In other words, the game progression unit 113 moves the character 30 or moves each part of the character 30's body in a virtual space in which the photographed image is displayed superimposed. In other words, the game progression unit 113 is capable of moving the character 30 on the photographed image. The movement of the character 30 may be performed based on a user's input operation or may be performed independently of a user's input operation.

[0064] The game progression unit 113 acquires a captured image and varies a specific parameter based on the captured image. This will be described in detail below. In this embodiment, unless otherwise specified, "variation" of a parameter basically means a variation in a direction advantageous to the user (in other words, an increase or rise in the parameter), but is not limited to this.

[0065] The specific parameter may be, for example, at least one of a parameter of a virtual object (e.g., the character 30) displayed simultaneously with the captured image and a parameter indicating a predetermined amount of value owned by a user who owns the virtual object (hereinafter, also referred to as a "value parameter"). In other words, the specific parameter may be a parameter related to an in-game object such as the character 30. The parameter of the character 30 may be, for example, a parameter related to the ability (e.g., strength, etc.) of the character. Specifically, the parameter of the character 30 may be a parameter related to the level of the character 30 (e.g., the level itself or experience points for increasing the level, etc.), or may be a parameter indicating the character's intelligence, physical strength, attack power, defense power, speed, etc. In addition, the parameter of the character may be, for example, a parameter indicating the intimacy between the character and a specific person (e.g., the user who owns the character) (hereinafter, simply referred to as "intimacy"). In other words, the parameter of the character may be the status of the character (in other words, a status parameter). Additionally, the value parameter may be, for example, a parameter indicating the amount of electronic currency owned by the user, or a parameter related to a specified point (e.g., a specified item or electronic currency in the virtual space, or points exchangeable for a specified object in the real world, etc.).

[0066] The game progression unit 113 analyzes the captured image. Specifically, the game progression unit 113 identifies the real object 32 included in the captured image (in other words, the type of the real object 32 shown in the captured image). For example, when a banana is shown in the captured image (see FIG. 3), the game progression unit 113 identifies that the captured image includes a banana. Furthermore, when a book is shown in the captured image, the game progression unit 113 identifies that the captured image includes a book. Furthermore, when money is shown in the captured image, the game progression unit 113 identifies that the captured image includes money. Note that the granularity of the analysis may be appropriately determined according to the content of the game, and may be identified at the level of, for example, bananas, apples, and grapes, or may be identified at the level of fruits or food. Furthermore, the real object 32 may be identified by a classification different from the classification of the real object 32 in the real world. Furthermore, the type of the identified real object (in other words, the real object related to the variation of the specific parameter) is not particularly limited, and may be, for example, not limited to food and books, but may be living things, landmarks of each land, etc.

[0067] Note that, regarding a method of analyzing an image and identifying a real object, various methods known in the art have been proposed, and various known methods can be used. Furthermore, the image analysis may be performed by a server or the like external to the information processing system 1.

[0068] The game progression unit 113 varies the specific parameter based on the analysis result of the captured image. Specifically, the game progression unit 113 varies the specific parameter when the analysis of the captured image indicates that a specific real object 32 is included in the captured image. In other words, the game progression unit 113 varies the specific parameter based on the specific real object 32 and the character 30 as a virtual object being displayed together.

[0069] For example, when the game progression unit 113 determines, as a result of analyzing a photographed image, that the photographed image contains a banana as a specific real object 32, it increases a specific parameter (for example, intimacy as a parameter of the character 30). Also, when the game progression unit 113 determines, as a result of analyzing a photographed image, that the photographed image contains a book as a specific real object, it increases a specific parameter (for example, intelligence as a parameter of the character 30). Also, when the game progression unit 113 determines, as a result of analyzing a photographed image, that the photographed image contains money, it increases (in other words, increases) a specific parameter (for example, a value parameter, specifically, the amount of in-game currency).

[0070] Furthermore, the game progression unit 113 varies the specific parameter based on the position where the photographed image displayed simultaneously with the virtual object (e.g., character 30) was photographed (hereinafter also referred to as the "photographing position"). Note that the photographing position may be determined based on position information acquired by a GPS sensor or the like serving as the input unit 17 when the photograph is taken, or may be determined by image analysis from the object shown in the photographed image. Various known methods can be used to identify the photographing position.

[0071] For example, when a banana is included in a photographed image, the game progression unit 113 increases a specific parameter (for example, the intimacy level of the character 30), but controls the variation of the specific parameter differently between when the photographed position is a first position and when the photographed position is a second position. Note that "differentiating the control of the variation" may mean, for example, increasing the increase in a specific parameter in one case compared to the other case. Also, "differentiating the control of the variation" may mean, for example, increasing a parameter that is not increased in one case in the other case. Also, in the following description, "advantageous to the user" in terms of parameter variation includes a case where the increase in the parameter is large, a case where a parameter that is not increased in one case is increased, and the like. In other words, in terms of parameter variation, no parameter variation may be performed on the unfavorable side.

[0072] Specifically, when the photographed location of a photographed image including a banana is "Philippines", the game progression unit 113 increases the extent of increase in a predetermined parameter (e.g., intelligence) of the character 30 compared to when the photographed location is "Tokyo". Furthermore, when the photographed location of a photographed image including a banana is "Philippines", the game progression unit 113 increases a parameter (e.g., value parameter) that is not increased when the photographed location is "Tokyo".

[0073] Furthermore, the game progression unit 113 may vary the specific parameter based on a correspondence relationship between at least two of the shooting position, the real object 32, and the virtual object. Note that such a correspondence relationship may be registered in advance in a predetermined storage unit (for example, the storage unit 120 or the storage unit 220).

[0074] For example, the game progression unit 113 may vary a specific parameter based on the correspondence between the character 30 and the shooting position. For example, when the shooting position is a location corresponding to the "otaku attribute" (e.g., a real-world location preferred by otakus) in relation to the variation of a specific parameter based on a photographed image displayed simultaneously with a character 30 to which an attribute "otaku attribute" is set as a predetermined attribute, the game progression unit 113 performs a variation that is more advantageous to the user than when the shooting position is not a location corresponding to the "otaku attribute". Also, for example, when the shooting position is near a sea or a river as a location corresponding to the "water attribute" (in other words, likes water), the game progression unit 113 performs a variation that is more advantageous to the user than when the shooting position is away from the sea or a river as a location not corresponding to the "water attribute".

[0075] Furthermore, for example, the game progression unit 113 may vary a specific parameter based on the correspondence between the real object 32 and the shooting position. For example, when a captured image displayed simultaneously with the character 30 includes a banana, the game progression unit 113 performs a variation of the specific parameter that is more advantageous to the user when the shooting position is "Philippines" as a place corresponding to bananas, compared to when the shooting position is "Tokyo" as a place not corresponding to bananas. Similarly, when a captured image displayed simultaneously with the character 30 includes an apple, the game progression unit 113 performs a variation of the specific parameter that is more advantageous to the user when the shooting position is "Aomori" as a place corresponding to apples, compared to when the shooting position is "Tokyo" as a place not corresponding to apples. In other words, when a specific real object 32 (e.g., banana) is included in a captured image captured at a specific position (e.g., Philippines), the game progression unit 113 may perform a specific process (e.g., a process that increases the increase in the parameter) that is advantageous to the user with respect to the variation of the specific parameter.

[0076] Furthermore, for example, the game progression unit 113 may vary a specific parameter based on the correspondence between the character 30 and the real object 32. For example, when the game progression unit 113 varies a specific parameter based on the inclusion of an apple in a captured image displayed simultaneously with the character 30, if the character 30 is a character corresponding to apples (for example, a character with an attribute "likes apples" set), the game progression unit 113 may vary the specific parameter in a way that is more advantageous to the user than if the character 30 is a character not corresponding to apples (for example, a character without an attribute "likes apples" set).

[0077] Furthermore, for example, the game progression unit 113 may vary a specific parameter based on the correspondence between the character 30, the real object 32, and the shooting position. For example, in the game of this embodiment, a first character to which the attribute "likes apples from Aomori" (or "likes apples") is set and a second character to which the attribute "likes apples from Aomori" (or "likes apples") is not set (in other words, a character corresponding to apples from Aomori and a character not corresponding to apples from Aomori). Then, when the shooting position is "Aomori" and the character 30 is the first character, the game progression unit 113 may vary the specific parameter based on the inclusion of an apple in the photographed image displayed simultaneously with the character 30 in a manner that is more advantageous to the user than when the character 30 is the second character or when the character 30 is the first character but the shooting position is not "Aomori".

[0078] Furthermore, the game progression unit 113 may vary the specific parameter based on the time when the photographed image displayed simultaneously with the virtual object (e.g., character 30) was captured (hereinafter, also referred to as "captured time"). In other words, the game progression unit 113 may vary the control related to the variation of the specific parameter depending on the photographed time. For example, in the game of this embodiment, there may be a first character to which an attribute of "nocturnal" is set and a second character to which an attribute of "diurnal" is set (in other words, a character corresponding to night and a character not corresponding to night). Then, the game progression unit 113 may vary the specific parameter based on the photographed image displayed simultaneously with the first character in a manner that is more advantageous to the user when the photographed time is night than when the photographed time is day. Specifically, for example, if the captured image is taken at night, the game progression unit 113 may treat the first character as being active and vary a specific parameter (for example, vary a specific parameter by eating a real object 32 included in the captured image), and if the captured image is taken during the day, the game progression unit 113 may not treat the first character as being inactive and vary a specific parameter (for example, vary a specific parameter by eating a real object 32 included in the captured image).

[0079] Furthermore, the game progression unit 113 may vary the specific parameter based on a predetermined reference position related to the user who owns the virtual object (for example, the character 30) and the shooting position. Here, the reference position may be, for example, the user's residence. That is, in the game of this embodiment, the user may be able to set the position of his / her own home (the position of the home on the map). Then, when a photographed image displayed simultaneously with the character 30 is photographed at a first position away from the position set as the home of the user who owns the character 30, the game progression unit 113 may vary the specific parameter in a way that is advantageous to the user, compared to when the photographed image is photographed at a second position closer to the home than the first position. In other words, when varying the specific parameter based on the photographed image, the game progression unit 113 may vary according to the moving distance of the user in the real world. Note that the "moving distance" here is not limited to the distance on the map from the reference position, but may be, for example, the total distance traveled by the user within a predetermined period, or the distance traveled from the previous photographing to the next photographing.

[0080] Note that the points on the map that the user can set as the residence may be limited to a range corresponding to the user's range of movement in the real world. Specifically, for example, the points on the map that the user can set as the user's home may be limited to points corresponding to places the user has been to in the real world, the place where the user is currently located in the real world, the place where the user has stayed in the real world for a predetermined time or more (for example, continuously or cumulatively), or the place where the user visits in the real world a predetermined number of times or more or with a predetermined frequency. In other words, the points that the user can set as the residence may be limited to places where the game progression unit 113 can infer that the first user actually lives there.

[0081] Furthermore, the reference position does not have to be the user's place of residence. Furthermore, the reference position does not have to be set by the user. Specifically, the game progression unit 113 may automatically set, as the reference position, a location in the real world where the user has stayed for a predetermined time or more (for example, continuously or cumulatively), or a location in the real world that the user has visited a predetermined number of times or more or with a predetermined frequency.

[0082] The game progression unit 113 may also limit the number of times that a variation of a specific parameter based on a captured image can be executed.

[0083] Specifically, for example, the game progression unit 113 may limit the number of times that a specific parameter can be changed based on a captured image including the same real object 32 within a predetermined period. More specifically, for example, the game progression unit 113 may limit the number of times that a specific parameter can be changed based on the same real object 32 (for example, a banana, etc.) to N times per day (N is a natural number equal to or greater than "1"). For example, when the game progression unit 113 has changed a specific parameter based on a banana being included in a real image displayed simultaneously with the character 30 N times within a predetermined period, it may prohibit a change of a specific parameter based on a banana being included in a real image displayed simultaneously with the character 30 until a certain period has elapsed. Note that the predetermined period may be, for example, a period starting from a predetermined time (for example, midnight every day, etc.) or a period starting from the time when a captured image including a specific real object 32 is captured (in other words, a period determined based on the user's behavior), etc.

[0084] Furthermore, the game progression unit 113 may prohibit a variation of a specific parameter based on the inclusion of a specific real object 32 in a captured image acquired at a first position based on the fact that the captured image acquired at a first position has already been varied, and in a state in which a variation of a specific parameter based on the inclusion of the specific real object 32 in a captured image acquired at a first position is prohibited, may vary a specific parameter based on the inclusion of the specific real object 32 in a captured image acquired at a second position. For example, when the game progression unit 113 varies a specific parameter based on the inclusion of a banana in a captured image that is displayed simultaneously with the character 30 and that was captured at a first position, N times within a predetermined period, the game progression unit 113 may prohibit a variation of a specific parameter based on the inclusion of a banana in the captured image for the captured image captured at the first position until a certain period has elapsed, while varying a specific parameter based on the inclusion of a banana in the captured image for the captured image captured at the second position even during the certain period. The second position may be a position away from the first position by a predetermined distance or more.

[0085] In addition, when a specific condition is met (for example, a specific parameter is changed N times within a specific period based on the fact that a specific real object 32 is included in a real image displayed simultaneously with the character 30), the specific parameter may be changed by using a specific item in a state in which the change of the specific parameter based on the captured image is restricted (in other words, prohibited). The specific item may be, for example, an item (in other words, a paid item) that the user can obtain by consuming his / her assets (for example, electronic currency). The specific item may be, for example, a virtual object (for example, food, book, etc.) that can be used (in other words, given) to the character. The specific item may also be consumed by use. The specific item is used, for example, based on an input operation by the user to select an item to be used for the character 30. When the specific item is used, the display control unit 114 simultaneously displays the specific item and the character 30 on the display unit 18, and displays a display in which the character 30 takes an action on the specific item (for example, an action of eating the specific item, etc.). When the specific item is used, the game progression unit 113 varies the specific parameters, such as increasing the parameters of the character 30. Note that the screen on which the specific item and the character 30 are displayed may be an augmented reality image with a photographed image as the background. In other words, in a state in which an image in which the character 30 is superimposed on a photographed image is displayed on the display unit 18 (for example, the state exemplified in FIG. 3), it may be possible to vary the specific parameters by using the specific item.

[0086] Furthermore, the game progression unit 113 may impose a penalty (e.g., a predetermined penalty related to a specific parameter) on the user when the same real object 32 is photographed multiple times from different shooting positions. In this embodiment, the "penalty" may be, for example, a penalty that reduces the specific parameter (in other words, changes it in a direction unfavorable to the user), or may be a penalty that prevents an increase in the specific parameter based on photographing the real object 32 (in other words, displaying the real object 32 and the character 30 simultaneously) for a predetermined period of time. Specifically, the game progression unit 113 may impose a penalty on the user and reduce the specific parameter when a photographed image including the specific real object 32 is photographed at a first position and displayed simultaneously with the character 30, and after a variation of the specific parameter is performed based on this, a photographed image including the specific real object 32 is photographed at a second position and displayed simultaneously with the character 30. In other words, the game progression unit 113 may impose a penalty when the user tries to increase a specific parameter using the same real object 32 from a different shooting position. According to this configuration, in a configuration in which the variation of a specific parameter using the same real object 32 at the same location is restricted, a penalty can be imposed on a user who attempts to circumvent the restriction and vary a specific parameter.

[0087] Furthermore, in a case where a captured image displayed simultaneously with a virtual object (e.g., character 30) is captured multiple times within a predetermined period, if the user (terminal device 10) moves for each capture and the type of real object 32 included in the captured image is different for each capture, the game progression unit 113 may perform a variation of the specific parameter that is more advantageous to the user than when the user does not move for each capture and the multiple captures are performed (and the type of real object 32 included in the captured image is different for each capture) and when the type of real object 32 included in the captured image is the same for the multiple captures (and the user moves for each capture and the multiple captures are performed). In addition, in a case where a captured image displayed simultaneously with a virtual object (e.g., character 30) is captured and the user (terminal device 10) moves for each capture and the type of real object 32 included in the captured image is changed for each capture, the game progression unit 113 may perform a variation of the specific parameter that is more advantageous to the user than when the predetermined number of captures exceed the predetermined period. Specifically, in a case where an image including an apple is taken, then the user moves and an image including a banana is taken, then the user moves and an image including grapes is taken, and the process from taking the image of the apple to taking the image of the grapes is completed within a predetermined period of time, the game progression unit 113 may make a change in a specific parameter that is more advantageous to the user than a case where an image including an apple, an image including a banana, and an image including grapes are taken in sequence without the user moving at all, or a case where an image including an apple is taken, then the user moves and an image including the apple is taken, then the user moves and an image including the apple is taken, or a case where an image including an apple is taken, then the user moves and an image including a banana is taken, then the user moves and an image including the grapes is taken, but the process from taking the image of the apple to taking the image of the grapes is not completed within the predetermined period of time.

[0088] Furthermore, the control unit 110 may have a detection unit 115 capable of detecting whether or not the real object 32 shown in the captured image is an object that actually exists at that location. Specifically, the control unit 110 may have a detection unit 115 capable of detecting whether or not the real object 32 shown in the captured image has a three-dimensional shape. For example, the detection unit 115 may determine whether or not the real object 32 shown in the captured image has a three-dimensional shape based on information from a depth camera serving as the input unit 17. Furthermore, for example, the detection unit 115 may determine whether or not the real object 32 photographed from multiple directions by moving the terminal device 10 or the real object 32 during the photographing of the captured image has a three-dimensional shape (in other words, by instructing the user to photograph the real object 32 from multiple directions and based on the photographed images acquired in accordance with the instruction). Then, when it is determined that the real object 32 included in the photographed image has a three-dimensional shape, the game progression unit 113 may execute (in other words, permit) the variation of the specific parameter based on the photographed image, and when it is determined that the real object 32 included in the photographed image does not have a three-dimensional shape, the game progression unit 113 may not execute (in other words, not permit) the variation of the specific parameter based on the photographed image (specifically, the variation in a direction advantageous to the user). That is, when the real object 32 shown in the photographed image is, for example, an image displayed on a predetermined display or an image printed on paper, the game progression unit 113 may not execute the variation of the specific parameter based on the photographed image (specifically, the variation in a direction advantageous to the user). Furthermore, when it is determined that the real object 32 included in the photographed image does not have a three-dimensional shape, the game progression unit 113 may impose a penalty on the user. That is, for example, when a user attempts to increase a specific parameter by cheating, such as by taking an image of a banana displayed on a display, the game progression unit 113 may detect the cheating and impose a penalty, such as decreasing the specific parameter.

[0089] Furthermore, the game progression unit 113 may determine the value in the real world of the real object 32 included in the captured image, and vary a specific parameter according to the value. For example, the game progression unit 113 may control the increase in the specific parameter (for example, the increase in the value parameter) to be larger as the value of the real object included in the captured image is higher. The determination of the value of the real object 32 may be performed using AI (Artificial Intelligence). The AI ​​may be either a machine learning type or a non-machine learning type (rule-based type). For example, a correspondence between the type of the real object 32 and the value in the real world of each real object 32 (for example, the market price in the real world) may be stored in a predetermined storage unit (for example, the storage unit 220), and the game progression unit 113 may determine the value in the real world from the type of the real object 32 identified by analyzing the captured image. Alternatively, machine learning may be performed based on images of various real objects 32 and information indicating the real-world values ​​of the real objects 32, and the obtained model may be used to determine the real-world value of the real object 32 included in the photographed image (in other words, by providing the photographed image as an input to the model). Furthermore, the game progression unit 113 may determine the real-world value of the real object 32 included in the photographed image by analyzing the photographed image to identify the type of real object and searching for its value on the Internet (in other words, by searching for the value of the real object 32 included in the photographed image on the Internet based on the photographed image).

[0090] Furthermore, when a positional relationship between a specific real object 32 included in a photographed image and a virtual object (e.g., character 30) displayed superimposed on the photographed image becomes a predetermined positional relationship, the game progression unit 113 may execute a variation of a specific parameter based on the photographed image and the virtual object being displayed simultaneously. Specifically, the game progression unit 113 may not vary the specific parameter when the specific real object 32 and the character 30 are displayed overlapping by a predetermined amount or more, and may vary the specific parameter when the overlap amount is less than the predetermined amount (e.g., when there is no overlap at all). In other words, the variation of the specific parameter may require that the specific real object 32 is photographed so as not to overlap with the character 30. Separately or in addition to this, the game progression unit 113 may vary the specific parameter based on at least a partial overlap between the specific real object 32 and the character 30. Specifically, the game progression unit 113 may identify the type of the real object 32 overlapping with the character 30, and vary the specific parameter based on the identification result. According to this configuration, it is possible to limit the objects appearing in the captured image, the types of which are identified by image analysis, thereby reducing the processing load associated with the image analysis.

[0091] Furthermore, the game progression unit 113 may store a history of shooting of the captured images displayed simultaneously with the virtual object (hereinafter, also referred to as the "shooting history") in a predetermined storage unit (for example, the storage unit 120). For example, a history of the locations where the captured images were shot may be stored as the shooting history. For example, the captured captured images themselves may be stored as the shooting history. For example, the types of the real objects 32 included in the captured captured images may be stored as the shooting history. For example, the contents of the executed variation of the specific parameters (specifically, which parameters have increased, how many parameters have increased, which character 30's parameters have increased, etc.) may be stored as the shooting history in association with each captured image, the location where the captured image was shot, or the type of the real object 32 included in the captured image. Note that the captured images may be stored here in the form of superimposed images.

[0092] Furthermore, the game progression unit 113 may display the shooting history on the display unit 18 based on a predetermined operation by the user for displaying the shooting history on the display unit 18. Specifically, for a plurality of past shootings, at least one (or all) of the following may be displayed on the display unit 18 in a list or the like: the shot image (for example, the shot image in a state of superimposed image), the location where the shot image was shot, the type of real object included in the shot image, and the contents of the executed change of a specific parameter. According to such a configuration, for example, it is possible to check from the shooting history what kind of real-world thing should be given to a virtual object owned by the user such as the character 30 to obtain an effect (for example, an effect related to a change in a parameter, whether the character 30 will be pleased, etc.), and the user can be given the fun of searching for such things, etc. According to such a configuration, for example, it is possible to leave a record of the places the user has visited with the virtual object such as the character 30 in the form of a shooting history, and the user can be given the fun of traveling with the character 30, etc.

[0093] As described above, in this embodiment, the game progression unit 113 varies the specific parameter when a specific real object 32 is included in the captured image, but the specific real object 32 may also be a person.

[0094] Furthermore, in a configuration in which a specific parameter is varied when a captured image including a person as a specific real object 32 is displayed simultaneously with a character 30 as a virtual object, the game progression unit 113 may vary the specific parameter based on the facial expression of the person included in the captured image. For example, when the person has a smiling expression, the game progression unit 113 may vary the specific parameter in a way that is more advantageous to the user than when the person has an angry expression.

[0095] Furthermore, in a configuration in which a specific parameter is varied when a photographed image including a person as a specific real object 32 is displayed simultaneously with a character 30 as a virtual object, if the person included in the photographed image is a specific person, the game progression unit 113 may vary a parameter of the character 30 related to the specific person (for example, the intimacy between the character 30 and the specific person). The specific person may be, for example, a person registered in the game as the owner of the character 30. Furthermore, the specific person may be, for example, a person appearing in an image (for example, a photo) stored in a predetermined image storage area (for example, a so-called photo folder) in the storage unit 120 of the terminal device 10 (in other words, a person who is inferred from the image to be the owner of the terminal device 10 or a related person of the terminal device 10, etc.).

[0096] The game progression unit 113 may perform a process of varying a specific parameter based on a captured image displayed simultaneously with a virtual object, based on the weather conditions at the time the captured image is captured. Specifically, a weather information acquisition unit (not shown) included in the control unit 110 may acquire weather information indicating the weather conditions at the time the captured image is captured, and the game progression unit 113 may vary the specific parameter based on the weather information. The weather information acquisition unit may acquire weather information indicating the weather conditions of the area where the image was captured, based on, for example, location information indicating the user's location. Specifically, for example, the weather information acquisition unit may access a website that publishes weather information or a predetermined server that stores weather information, and acquire weather information for the area corresponding to the location information. The weather information acquisition unit may acquire weather information indicating the weather conditions (for example, weather, etc.) at the time the captured image was captured by performing image analysis on the captured image. The weather information acquisition unit may acquire weather information from a predetermined sensor (for example, a temperature sensor) included in the terminal device 10. The weather information may be, for example, information on the weather, such as "sunny," "rain," "cloudy," or "snow," or may be information on the temperature. When a captured image displayed simultaneously with the character 30 includes a specific real object 32 (e.g., juice) and the captured image is captured in a first weather condition with a relatively low temperature, the game progression unit 113 may vary the specific parameter in a way that is more advantageous to the user than when the captured image is captured in a second weather condition with a higher temperature than the first weather condition. With this configuration, it is possible to express an appearance in which the character 30 is exhausted due to the heat, for example. Note that compatibility with weather conditions may be set for the character 30.For example, for a first character 30 (e.g., a character 30 that prefers cold climates), a specific parameter may be changed in a way that is more favorable to the user when the image is captured in a first weather condition (e.g., a relatively low temperature condition) than when the image is captured in a second weather condition (e.g., a relatively high temperature condition), and for a second character 30 (e.g., a character 30 that prefers hot climates), a specific parameter may be changed in a way that is more favorable to the user when the image is captured in the second weather condition than when the image is captured in the first weather condition.

[0097] Furthermore, the game progression unit 113 may perform a process of varying a specific parameter based on a captured image displayed simultaneously with a virtual object, based on a situation related to ambient sounds when the captured image is captured. Specifically, the game progression unit 113 may vary a specific parameter based on a sound input from a microphone serving as the input unit 17. For example, when a captured image is captured in a first situation (in other words, a quiet situation) in which ambient sounds are relatively low, the game progression unit 113 may vary the specific parameter in a manner more advantageous to the user than when the captured image is captured in a second situation (in other words, a noisy situation) in which ambient sounds are louder than the first situation. Furthermore, when a captured image is captured in a first situation in which the favorite music of a character 30 displayed simultaneously with the captured image is playing, the game progression unit 113 may vary the specific parameter in a manner more advantageous to the user than when the captured image is captured in a second situation in which the favorite music of the character 30 is not playing. Such preferred music may be preset for each character 30 by the game creator or the like, or may be set by the user who owns the character 30, or the character 30 may memorize as its preferred music music that is often played while the game is being played (e.g., while taking pictures, etc.) (in other words, it may be set according to the user's play), etc.

[0098] In this embodiment, the shooting of the captured image and the simultaneous display of the captured image and the virtual object are performed in real time, and the game progression unit 113 varies the specific parameter based on the analysis result of the captured image. However, the shooting of the captured image and the simultaneous display of the captured image and the virtual object do not have to be performed in real time. For example, the game progression unit 113 may simultaneously display the selected image and the virtual object based on the input operation of the user who selects an image to be applied to a virtual object (for example, the character 30) from a plurality of images captured in the past, and may vary the specific parameter based on the analysis result of the selected image. In other words, the "captured image" in this embodiment may be an image captured in the past. In such a configuration, the captured image captured in the past may have a freshness (in other words, a use-by date). Specifically, for example, when an image containing an apple is selected, the more recently it was captured (the more recently it was captured), the more advantageous the user may be to the variation of the specific parameter. Furthermore, when a photographed image including an apple is selected after a predetermined period of time has passed since it was photographed (in other words, after the expiration date), the specific parameter may not be changed or a penalty may be given to the user (in other words, the character 30 may become ill). Furthermore, such an expiration date may be different for each real object 32 included in the photographed image (for example, whether it is an apple, a fish, meat, etc.).

[0099] Note that, when increasing a specific parameter based on the inclusion of a specific real object 32 in a captured image displayed simultaneously with a virtual object, the game progression unit 113 may cause the virtual object to execute a predetermined action (in other words, a predetermined effect) with respect to the specific real object. For example, the game progression unit 113 causes the character 30 as the virtual object to execute an action of eating food as the specific real object. In other words, the game progression unit 113 instructs the display control unit 114 to display, on the display unit 18, an effect in which the character 30 eats the food. Also, for example, the game progression unit 113 causes the character 30 as the virtual object to execute an action of reading a book as the specific real object. In other words, the game progression unit 113 instructs the display control unit 114 to display, on the display unit 18, an effect in which the character 30 reads the book. Also, the game progression unit 113 may cause the character 30 to speak a predetermined line, such as "It's a feast." Specifically, the game progression unit 113 may instruct the display control unit 114 to display the specified line on the display unit 18, or may output the specified line as audio from a speaker (in other words, a sound output unit) of the terminal device 10, etc.

[0100] Furthermore, the game progression unit 113 may cause the virtual object to execute a specific effect when a captured image displayed simultaneously with the virtual object is acquired at a position having a predetermined relationship with the virtual object, and may not cause the virtual object to execute a specific effect when the captured image is not acquired at a position having a predetermined relationship with the virtual object. Specifically, for example, for a character 30 to which an attribute "otaku attribute" is set, when a specific parameter is changed based on a captured image displayed simultaneously with the character 30, if the shooting position is a location corresponding to the "otaku attribute", the game progression unit 113 may cause the character 30 to execute a specific action that is not executed when the shooting position is not a location corresponding to the "otaku attribute". Furthermore, for example, for a character 30 to which a correspondence relationship with "Aomori" is set, when a specific parameter is changed based on a captured image displayed simultaneously with the character 30, if the shooting position is "Aomori", the game progression unit 113 may cause the character 30 to execute a specific action that is not executed when the shooting position is a location other than "Aomori". Furthermore, the game progression unit 113 may cause the virtual object to execute a specific performance when a captured image displayed simultaneously with the virtual object was acquired at a position having a predetermined relationship with the virtual object and includes a specific real object (for example, a real object having a predetermined relationship with the virtual object). Specifically, for example, for a character 30 set as "I like apples from Aomori," when a specific parameter is varied based on a captured image displayed simultaneously with the character 30, if the captured position is "Aomori" and the captured image includes apples, the game progression unit 113 may cause the character 30 to execute a specific action that is not executed when the captured position is a place other than "Aomori" or when the captured image does not include apples.

[0101] Further, as a conventional map application, an application that allows a user to post a review for each location on a map (for example, a facility, etc.) is known. Specifically, an application that allows a user to post word-of-mouth, evaluation, etc. for a restaurant, etc. associated with each point on a map is known. The game progression unit 113 may vary the specific parameter based on information associated with such a location on a map that corresponds to a shooting position where a photographed image displayed simultaneously with the character 30 was taken. For example, the game progression unit 113 may vary the specific parameter according to an attribute set for a location on a map that corresponds to a shooting position where a photographed image displayed simultaneously with the character 30 was taken. Specifically, when the attribute of a location (for example, a facility) corresponding to the shooting position is "restaurant", the game progression unit 113 may control the variation of the specific parameter differently from when the attribute of a location corresponding to the shooting position is "school". Also, for example, the game progression unit 113 may vary the specific parameter according to reviews (for example, the number of reviews, the content of the reviews, etc.) posted for a location on a map that corresponds to a shooting position where a photographed image displayed simultaneously with the character 30 was taken. Specifically, the game progression unit 113 may control the variation of a specific parameter differently when the number of reviews is large compared to when the number of reviews is small. Furthermore, the game progression unit 113 may control the variation of a specific parameter differently when the rating based on the reviews is high compared to when the rating based on the reviews is low.

[0102] In addition, in a configuration in which the specific parameter is varied based on information associated with a location on a map corresponding to the shooting location where the photographed image displayed simultaneously with the character 30 was taken, the game progression unit 113 may vary the specific parameter without analyzing the photographed image. For example, when the photographed image displayed simultaneously with the character 30 is taken, the game progression unit 113 may determine a location on the map corresponding to the shooting location based on position information acquired by a GPS sensor or the like as the input unit 17, and vary the specific parameter based on information associated with the determined location (e.g., facility attributes, reviews, etc.) (in other words, regardless of the real object 32 shown in the photographed image). In addition, for example, the game progression unit 113 may vary the specific parameter based on a user posting a photographed image (e.g., a superimposed image) displayed simultaneously with the character 30 as a review associated with a specific location on the map. In other words, the variation of the specific parameter may be triggered by the review. In addition, the shooting location and the location on the map corresponding to the shooting location may be determined based on an input operation by the user, such as posting a review. In other words, the game progression unit 113 may determine that the location where the captured image was posted as a review is the shooting location of that captured image.

[0103] Furthermore, the game progression unit 113 may set the above-mentioned correspondence between the character 30 and the real object 32, the above-mentioned correspondence between the character 30 and the shooting position, etc., based on a user posting a photographed image (for example, a superimposed image) displayed simultaneously with the character 30 as a review associated with a specific location on the map. For example, the game progression unit 113 may set the character 30 as a "yakitori lover" (in other words, set a correspondence between the character 30 and the real object 32) or set the character 30 as a "yakitori restaurant lover" (in other words, set a correspondence between the character 30 and the shooting position) based on a user posting a photographed image (for example, a superimposed image) displayed simultaneously with the character 30 as a review of a yakitori restaurant as a specific location on the map. Then, the game progression unit 113 may make a variation that is more advantageous to the user when the character 30 is a character set as a "yakitori lover" in terms of variation of a specific parameter based on the inclusion of yakitori in a photographed image displayed simultaneously with the character 30, compared to a case where the character 30 is a character not set as a "yakitori lover".

[0104] Note that when processing related to variation of a specific parameter is performed based on the attributes and reviews of facilities on a map in this way, the map may be a map within the application that provides the game of this embodiment, or may be a map of another application (e.g., a map application) or a website, etc. In other words, the application that provides the game of this embodiment may be linked with another application or website using known technology, and the game progression unit 113 may acquire information about facilities set on the map and information about reviews, and perform the various processes described above.

[0105] Furthermore, the game progression unit 113 may change the appearance of a virtual object (e.g., character 30) displayed simultaneously with the captured image based on the captured image. Then, the display control unit 114 may cause the display unit 18 to display a display in which the appearance of the virtual object changes based on the captured image based on an instruction from the game progression unit 113. Specifically, for example, the game progression unit 113 may change the appearance (e.g., shape, pattern, or color) of the character 30 as the virtual object to one that reflects the characteristics of a specific real object 32 included in the captured image. For example, when the captured image includes a zebra as the real object 32, the game progression unit 113 may change the appearance of the character 30 displayed simultaneously with the zebra to a black and white striped pattern. Furthermore, for example, when the captured image includes a giraffe as the real object 32, the game progression unit 113 may change the appearance of the character 30 displayed simultaneously with the giraffe to one with an elongated neck or horns. In other words, the game progression unit 113 may change (in other words, inherit) the appearance of the character 30 to imitate (inherit) the specific real object 32 included in the captured image. The changed appearance may be maintained even after the specific real object 32 disappears from the captured image. With this configuration, it is possible to save an encounter with the specific real object 32 as a memory (in the form of the appearance of a virtual object). Note that in this configuration, the game progression unit 113 may vary a specific parameter (e.g., a parameter of the character 30) and change the appearance of the virtual object based on a captured image displayed simultaneously with the virtual object (for example, when the captured image includes a specific real object 32), or may change the appearance of the virtual object without varying the specific parameter. In other words, a configuration for changing the appearance of a virtual object based on a captured image may be realized without a configuration for changing a specific parameter based on a captured image.

[0106] (Processing related to variation of specific parameters based on captured images) An example of a process for varying a specific parameter based on a captured image will be described with reference to the flowchart shown in FIG.

[0107] First, the game progression unit 113 acquires a captured image of the real world (Step S101).

[0108] Next, the game progression unit 113 analyzes the captured image (Step S102). Specifically, the game progression unit 113 identifies the real object 32 included in the captured image.

[0109] Next, the game progression unit 113 determines whether or not the captured image includes a specific real object 32 related to an increase in a specific parameter (Step S103).

[0110] If a specific real object 32 related to an increase in the specific parameter is included in the captured image (YES in step S103), the game progression unit 113 increases the specific parameter (step S104). Specifically, the game progression unit 113 determines the type and amount of increase of the parameter to be increased according to the type of the specific real object 32, and increases the specific parameter based on the determination. Note that, in addition to or instead of the type of the specific real object 32, the game progression unit 113 may determine the type and amount of increase of the parameter to be increased (or whether or not to increase the specific parameter) based on at least one of the shooting position, shooting time, a correspondence between at least two of the shooting position, the real object 32, and the virtual object, the value of the real object 32 in the real world, the weather conditions at the time of shooting, and the conditions related to the surrounding sounds at the time of shooting, and increase the specific parameter based on the determination.

[0111] On the other hand, if a specific real object 32 related to an increase in the specific parameter is not included in the captured image (NO in step S103), the game progression unit 113 does not increase the specific parameter.

[0112] The game progression unit 113 instructs the display control unit 114 to display on the display unit 18 a screen on which the captured image and the character 30 as a virtual object are simultaneously displayed, and the display control unit 114 displays the screen on the display unit 18 based on the instruction, but the timing of the instruction (in other words, the timing at which the display control unit 114 displays the screen on the display unit 18) may be any timing after step S1.

[0113] The present invention is not limited to the above-described embodiment, and can be modified in various ways without departing from the gist of the present invention. The present invention can also be applied to services other than games. Within the scope of the present invention, the components can be freely combined, any components can be modified, or any components can be omitted. The process flow described in this specification is merely an example, and the order and configuration of each process may be different. Some processes shown in the flowchart may not exist. In other words, the process flow and specific determination processes may be different from those exemplified in this specification.

[0114] <Additional Notes> The matters described in the above embodiment can also be described as follows.

[0115] (Appendix 1) The processor, A display control means (e.g., a display control unit 114) for simultaneously displaying a captured image of the real world and a virtual object; a control unit (e.g., a game progression unit 113) that varies a specific parameter based on the captured image that is displayed simultaneously with the virtual object, The specific parameter is at least one of a parameter of the virtual object and a parameter indicating an amount of a predetermined value owned by a user who owns the virtual object. program. According to this configuration, it is possible to change the parameters of a virtual object based on an image captured of the real world, and to change the amount of value owned by the user related to the virtual object, thereby strengthening the connection between the virtual space and the real world.

[0116] (Appendix 2) The control means The specific parameter is varied based on a position at which the photographed image, which is displayed simultaneously with the virtual object, is acquired. 2. The program described in Appendix 1. According to such a configuration, it is possible to associate a real-world position where a photographed image was acquired with a variation in a specific parameter, thereby strengthening the connection between the virtual space and the real world.

[0117] (Appendix 3) The control means The control related to the variation of the specific parameter is varied depending on whether the photographed image displayed simultaneously with the virtual object is acquired at a position having a predetermined relationship with the virtual object. 2. The program described in Appendix 2. According to this configuration, a correspondence can be established between the real-world position where the captured image is acquired and the virtual object, thereby strengthening the connection between the virtual space and the real world.

[0118] (Appendix 4) The control means The specific parameter is varied based on a predetermined reference position of a user who owns the virtual object and a position at which the photographed image, which is displayed simultaneously with the virtual object, was acquired. 2. The program described in Appendix 1. With this configuration, a specific parameter can be varied based on the relative relationship between a reference position for the user and the position where the captured image is obtained, and the variation in the specific parameter can be associated with a real-world location that has a predetermined relationship with the user, thereby strengthening the connection between the virtual space and the real world.

[0119] (Appendix 5) The control means When the photographed image displayed simultaneously with the virtual object is acquired at a position having a predetermined relationship with the virtual object, the virtual object is caused to execute a specific effect, and when the photographed image is not acquired at a position having a predetermined relationship with the virtual object, the virtual object is not caused to execute the specific effect. 5. The program according to any one of appendices 1 to 4. With this configuration, a correspondence can be created between the real-world position where the captured image is obtained and the virtual object, and this correspondence can be reflected in the presentation, giving the user a strong sense of a strong connection between the virtual space and the real world.

[0120] (Appendix 6) The control means causes the virtual object to execute the specific performance when the photographed image displayed simultaneously with the virtual object is acquired at a position having a predetermined relationship with the virtual object and includes a specific object. 6. The program described in Appendix 5. With this configuration, a correspondence can be created between the real-world position where the captured image is acquired, the virtual object, and the real-world object, and this correspondence can be reflected in the presentation, giving the user a strong sense of a strong connection between the virtual space and the real world.

[0121] (Appendix 7) The processor further comprises: The function of the game progression unit 113 is to limit the number of times that the specific parameter can be varied based on the captured image. 5. The program according to any one of appendices 1 to 4. According to this configuration, it is possible to prevent the specific parameter from being changed too easily due to unlimited variation of the specific parameter based on the captured image.

[0122] (Appendix 8) The limiting means is Limiting the number of times that the specific parameter variation based on the captured images including the same object can be performed within a predetermined period of time. 7. The program described in Appendix 7. With this configuration, it is possible to prevent a specific parameter from suddenly changing within a short period of time.

[0123] (Appendix 9) The limiting means is prohibiting a variation of the specific parameter based on the specific object being included in the photographed image acquired at a first position, based on the fact that a variation of the specific parameter based on the specific object being included in the photographed image acquired at the first position having been executed; The control means In a state in which the variation of the specific parameter based on the specific object being included in the photographed image acquired at the first position is prohibited, the variation of the specific parameter based on the specific object being included in the photographed image acquired at the second position may be executed. 7. The program described in Appendix 7. With this configuration, it is possible to prevent specific parameters from being easily changed by photographing the same object in the same place multiple times, while widening the scope of enjoyment for the user.

[0124] (Appendix 10) The processor further comprises: The virtual object is caused to function as a change unit (e.g., a game progression unit 113) that changes the appearance of the virtual object to one that reflects the characteristics of a specific object included in the captured image that is displayed simultaneously with the virtual object. 5. The program according to any one of appendices 1 to 4. With this configuration, it becomes possible to reflect the characteristics of a real-world object in a virtual object, thereby strengthening the connection between the virtual space and the real world.

[0125] (Appendix 11) a display control means for simultaneously displaying a captured image of the real world and a virtual object; and a control means for varying a specific parameter based on the captured image displayed simultaneously with the virtual object, The specific parameter is at least one of a parameter of the virtual object and a parameter indicating an amount of a predetermined value owned by a user who owns the virtual object. Information processing system. According to this configuration, the same effects as those of the program described in Supplementary Note 1 can be achieved. [Explanation of symbols]

[0126] 1 Information processing system, 10 Terminal device, 11 Processor, 12 Memory, 13 Storage, 17 Input unit, 18 Display unit, 20 Server, 21 Processor, 22 Memory, 23 Storage, 110 Control unit, 111 Operation reception unit, 112 Transmission / reception unit, 113 Game progression unit, 114 Display control unit, 115 Detection unit, 120 Storage unit, 210 Control unit, 211 Transmission / reception unit, 212 Server processing unit, 213 Data management unit, 214 Synchronization processing unit, 220 Storage unit

Claims

1. Computer, a display control means for simultaneously displaying a captured image of the real world and a virtual object; a change means for changing the appearance of the virtual object itself to one that reflects features of a specific object included in the captured image that is displayed simultaneously with the virtual object; The appearance of the virtual object after the change by the change means is maintained even after the captured image including the specific object is no longer displayed simultaneously with the virtual object. program.

2. a display control means for simultaneously displaying a captured image of the real world and a virtual object; A change means for changing the appearance of the virtual object itself to one that reflects a feature of a specific object included in the captured image that is displayed simultaneously with the virtual object, The appearance of the virtual object after the change by the change means is maintained even after the captured image including the specific object is no longer displayed simultaneously with the virtual object. Information processing system.

Images