Programs and Information Processing Systems

Abstract

To enhance relationship between a virtual space and an actual space.SOLUTION: A program causes a computer to function as display control means for simultaneously displaying a photographed image of an actual world and a virtual object and variation means which varies the appearance of the virtual object itself to an appearance reflecting the characteristics of a specific object included in the photographed image displayed simultaneously with the virtual object. The appearance of the virtual object after the variation by the variation means is kept even after the photographed image including the specific object is no longer displayed simultaneously with the virtual object.SELECTED DRAWING: Figure 3

Description

Technical Field

[0002]

[0001] The present invention relates to a program and an information processing system.

Background Art

[0002] Conventionally, services that fuse a virtual space such as a game using augmented reality (AR) with the real space have been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in services that fuse a virtual space with the real space, there has been a problem that there is still a gap between the virtual space and the real space.

[0005] An object of the present invention is to strengthen the relationship between a virtual space and the real space.

Means for Solving the Problems

[0006] According to one embodiment shown in the present disclosure, a processor is caused to function as display control means for simultaneously displaying a captured image of the real world and a virtual object, and control means for varying a specific parameter based on the captured image simultaneously displayed with the virtual object, wherein the specific parameter is at least one of a parameter of the virtual object and a parameter indicating an amount of a predetermined value owned by a user who owns the virtual object. A program is provided. [Effects of the Invention]

[0007] According to the present invention, the relationship between virtual space and real space can be strengthened. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram illustrating the schematic configuration of an information processing system. [Figure 2] This is a block diagram showing the functional configuration of an information processing system. [Figure 3] This figure shows an example of a screen where a captured image and a virtual object are displayed simultaneously. [Figure 4] This flowchart shows an example of processing related to changes in specific parameters based on captured images. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings.

[0010] <System Hardware Configuration> As shown in Figure 1, the information processing system 1 of this embodiment includes a plurality of terminal devices 10 and a server 20.

[0011] The terminal device 10 and the server 20 are connected via network 2. Network 2 may consist of, for example, the internet, a mobile communication system (e.g., 3G, 4G, 5G, LTE (Long Term Evolution), etc.), WiFi (Wireless Fidelity), Bluetooth (registered trademark), other communication lines, or a combination thereof. Furthermore, the connection between the terminal device 10 and the server 20 may be wired or wireless.

[0012] The server 20 (in other words, a computer or information processing device) may be a general-purpose computer such as a workstation or personal computer. The server 20 comprises a processor 21, memory 22, storage 23, a communication interface 24, and an input / output interface 25. These components of the server 20 are connected to each other by a communication bus.

[0013] The processor 21 controls the overall operation of the server 20. The processor 21 may include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit), etc. The processor 21 reads programs from the storage 23 and loads them into memory 22. The processor 21 executes the loaded programs.

[0014] Memory 22 is the main memory. Memory 22 is composed of storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). Memory 22 provides the processor 21 with a workspace by temporarily storing programs and various data read by the processor 21 from storage 23. Memory 22 also temporarily stores various data generated by the processor 21 while it is operating according to the program.

[0015] In addition, in this embodiment, the program may be a program for realizing a game by the terminal device 10. Further, the program may be a program for realizing the game by the cooperation of the terminal device 10 and the server 20. As an example, the game realized by the cooperation of the terminal device 10 and the server 20 may be a game executed on a browser started on the terminal device 10. Further, the program may be a program for realizing the game by the cooperation of a plurality of terminal devices 10. In addition, various data includes, for example, data related to the game such as user information and game information, and instructions and notifications transmitted and received between the terminal device 10 and the server 20.

[0016] The storage 23 is an auxiliary storage device. The storage 23 is constituted by, for example, a storage device such as a flash memory or an HDD (Hard Disk Drive). Various data related to the game is stored in the storage 23.

[0017] The communication IF 24 controls the transmission and reception of various data via a network between the server 20 and the terminal device 10 or the like.

[0018] The input / output IF 25 is an interface for the server 20 to receive data input and an interface for the server 20 to output data. The input / output IF 25 may include, for example, an input unit which is an information input device such as a mouse and a keyboard, and a display unit which is a device for displaying and outputting an image.

[0019] The terminal device 10 (in other words, a computer, an information processing device) may be, for example, a smartphone, a feature phone, a PDA (Personal Digital Assistant), a tablet computer, a personal computer, a wearable terminal, or a game device. The terminal device 10 may be a mobile terminal. The terminal device 10 may be a portable terminal when the user executes a game.

[0020] The terminal device 10 includes a processor 11, a memory 12, a storage 13, a communication IF 14, an input / output IF 15, an input unit 17, a display unit 18, and a photographing unit 19. These components included in the terminal device 10 are connected to each other by a communication bus.

[0021] The processor 11 controls the operations of the entire terminal device 10. The processor 11 may include a CPU, an MPU, a GPU, etc. The processor 11 reads a program from the storage 13 and expands it in the memory 12. The processor 11 executes the expanded program.

[0022] The memory 12 is a main storage device. The memory 12 is composed of storage devices such as a ROM and a RAM, for example. The memory 12 provides a working area for the processor 11 by temporarily storing the program and various data read by the processor 11 from the storage 13. The memory 12 also temporarily stores various data generated while the processor 11 is operating according to the program.

[0023] The storage 13 is an auxiliary storage device. The storage 13 is composed of storage devices such as a flash memory or an HDD, for example. Various data related to the game are stored in the storage 13.

[0024] The communication IF 14 controls the transmission and reception of various data via a network between the terminal device 10 and the server 20, etc.

[0025] The input / output IF 15 is an interface for the terminal device 10 to receive data input and an interface for the terminal device 10 to output data. The input / output IF 15 may perform data input and output via, for example, USB (Universal Serial Bus), etc. The input / output IF 15 may include the input unit 17 or the display unit 18, etc.

[0026] The input unit 17 accepts input from the user. The input unit 17 may be, for example, a pointing device such as a touchpad. The display unit 18 displays an image. The display unit 18 may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The terminal device 10 includes, for example, a touchscreen 16 which is an electronic component combining the input unit 17 and the display unit 18.

[0027] The input unit 17 has the function of detecting the position input to the input surface by user operation (e.g., touch operation, tap operation, slide operation, swipe operation, and flick operation, etc.) and transmitting information indicating the detected position as an input signal. The touch panel as the input unit 17 can employ a capacitive type or a resistive type, but other types may also be used.

[0028] The input unit 17 may be, for example, a keyboard, various physical buttons, various sensors (e.g., an acceleration sensor, angular velocity sensor, motion sensor, gaze sensor, biopotential sensor, fingerprint sensor, breath sensor, pressure sensor, or image sensor), an operating stick, a camera, or a microphone. The display unit 18 may be, for example, a projector.

[0029] The imaging unit 19 is a camera that captures images of the real world surrounding the terminal device 10. The imaging unit 19 may be included in the input unit 17.

[0030] <System Functional Configuration> Figure 2 is a block diagram showing the functional configuration of the server 20 and the terminal device 10. In this embodiment, the server 20 has functions such as providing various data and programs necessary to realize the game to each terminal device 10, collecting and managing game-related data from each terminal device 10, and performing synchronization processing between multiple terminal devices 10.

[0031] In this embodiment, the server 20 identifies each user and terminal device 10 using user accounts that are registered in advance for each game. The method of registering accounts is not particularly limited. For example, the terminal device 10 or another device such as a personal computer may transmit information necessary for user account registration to the server 20 based on user operations, and the server 20 may create and store each user's account based on the received information.

[0032] As shown in Figure 2, the server 20 functions as a control unit 210 and a storage unit 220 through the cooperation of the processor 21, memory 22, storage 23, communication interface 24, and input / output interface 25. The storage unit 220 stores various types of data used by the control unit 210. These types of data include, for example, game programs, game information, and user information.

[0033] The game program is a program that enables the game to be played. Game information and user information are data that the control unit 210 refers to when executing the game program.

[0034] In this embodiment, the game program may include not only the game program executed on the server 20 but also the game program executed on the terminal device 10. Furthermore, the storage unit 220 may store both the game program executed on the server 20 and the game program executed on the terminal device 10.

[0035] Game information is shared across accounts. This includes, for example, information defining various virtual spaces. A virtual space is a space where various objects (in other words, virtual objects), such as character objects owned by the user, are placed. In other words, the virtual space contains the game medium used in the game (such as character objects). Game information includes, for example, setting information about objects shared across accounts, such as the placement, size, color, and shape of background objects like buildings, trees, and stones, as well as non-player character (NPC) objects placed in the virtual space. Furthermore, game information includes, for example, the setting values ​​of various NPC parameters. In the following, the character objects placed in the virtual space may simply be referred to as "characters."

[0036] User information is information managed for each game account. User information includes, for example, information about the characters owned by the user, information about owned assets, and information indicating the progress of the game. Examples of owned assets include virtual currency and items. In other words, owned assets can be said to be the value owned by the user in the virtual world (in other words, the value associated with the user). An example of such value (in other words, value within the virtual world) is electronic currency. Examples of electronic currency include in-game currency usable within the game and cryptocurrencies.

[0037] The control unit 210 controls various processes related to the game by executing the game program stored in the memory unit 220. The control unit 210 includes, for example, a transmitting / receiving unit 211, a server processing unit 212, a data management unit 213, and a synchronization processing unit 214.

[0038] The transmitting / receiving unit 211 transmits or receives various types of data. For example, the transmitting / receiving unit 211 receives requests to transmit various types of data and programs, requests for synchronization processing to support the multiplayer function, and data to be synchronized from each terminal device 10, and passes them to the server processing unit 212. The transmitting / receiving unit 211 also transmits various types of data and programs, including instructions for synchronization, to each terminal device 10 in accordance with the control of the server processing unit 212.

[0039] In this embodiment, the multiplayer function is a function that synchronizes the game processing of multiple accounts. The server 20 and terminal device 10 of the information processing system 1 execute various processes to support the multiplayer function when multiple accounts logged into the information processing system 1 participate in the same game.

[0040] The server processing unit 212 provides the game to the terminal device 10 by executing calculations described in the game program in response to requests from the terminal device 10. For example, when the server processing unit 212 receives a request for synchronization processing to support the multiplayer function or data to be synchronized from the terminal device 10 via the transmission / reception unit 211, it executes the synchronization processing to support the multiplayer function. The server processing unit 212 also instructs the transmission / reception unit 211 to send game information or user information. Furthermore, the server processing unit 212 instructs the data management unit 213 to add, update, or delete records of game information or user information.

[0041] The data management unit 213 manages various data stored in the storage unit 220 according to instructions from the server processing unit 212. For example, the data management unit 213 reads game information or user information according to instructions from the server processing unit 212 and transmits it to the terminal device 10 via the transmission / reception unit 211. The data management unit 213 also adds, updates, or deletes records of game information or user information according to instructions from the server processing unit 212.

[0042] The synchronization processing unit 214 performs synchronization processing to support the game's multiplayer function in accordance with instructions from the server processing unit 212. For example, when the server 20 sends information to multiple terminal devices 10, the synchronization processing unit 214 synchronizes the game progressing among the terminal devices 10 by simultaneously sending the information to each terminal device 10. Specifically, the synchronization processing unit 214 simultaneously sends operation information received from each terminal device 10 corresponding to each account within a predetermined period (e.g., one frame) to each terminal device 10 at predetermined intervals. The operation information is information about operations input to the terminal device 10. The timing of synchronization and the information to be synchronized may be received from the server processing unit 212 as needed. By performing synchronization processing, it becomes possible to simultaneously reflect in-game events caused by operations input on one terminal device 10 to other terminal devices 10.

[0043] In this embodiment, the terminal device 10 has, for example, functions as an input device that accepts user input operations, and functions as an output device that outputs game images and sounds.

[0044] The terminal device 10 functions as a control unit 110 and a storage unit 120 through the cooperation of the processor 11, memory 12, storage 13, communication IF 14, and input / output IF 15, etc. The storage unit 120 stores various data used by the control unit 110. These various data include, for example, game programs, game information, and user information. The game program is a program for implementing the game on the terminal device 10 side. Game information and user information are data that the control unit 110 refers to when executing the game program. The game information and user information stored in the storage unit 120 may include the same information as the game information and user information stored in the storage unit 220.

[0045] The control unit 110 controls various processes related to the game executed in the terminal device 10 by executing the game program stored in the memory unit 120. The control unit 110 includes, for example, an operation reception unit 111, a transmission / reception unit 112, a game progress unit 113, a display control unit 114, and a detection unit 115.

[0046] The operation reception unit 111 receives operations (hereinafter also referred to as "input operations") input by the user via the input unit 17. Specifically, when an input operation is performed on the input unit 17, the operation reception unit 111 detects the coordinates of the input position and the type of input operation. Examples of input operations include various operations performed with fingers, such as touch operations, tap operations, slide operations, swipe operations, flick operations, pinch-in operations, and pinch-out operations. Input operations are not limited to operations that physically contact the input unit 17 (for example, the touchscreen 16), but can also include non-contact operations. Operations that terminate an input operation that was previously performed, such as a touch-off operation that ends contact with the touchscreen 16, can also be considered a form of input operation.

[0047] Here, the operation reception unit 111 can accept input operations performed using operating devices connected via the input / output IF 15, in the same way as input operations performed on the input unit 17.

[0048] The transmitting / receiving unit 112 transmits or receives various types of data. The transmitting / receiving unit 112 transmits, for example, game information or user information, or synchronization requests for multiplayer functionality to the server 20. The transmitting / receiving unit 112 receives various types of data, programs, and synchronization data for multiplayer functionality from the server 20. The synchronization data includes, for example, synchronization instruction data to instruct each terminal device 10 participating in multiplayer to synchronize. The synchronization instruction data includes, for example, data to specify the data to be synchronized, the type of data, and the timing of the synchronization.

[0049] The transmitting / receiving unit 112 transmits operation information related to input operations received by the operation reception unit 111 to the server 20. The transmitting / receiving unit 112 also receives operation information from the server 20 related to operations entered by other users on other terminal devices 10.

[0050] The game progress unit 113 executes various processes related to the progress of the game. A specific example is given below.

[0051] The game progress unit 113 defines the virtual space based on the information for defining the virtual space included in the game information. The game progress unit 113 places objects in the virtual space based on the object setting information included in the game information. The game progress unit 113 controls the objects placed in the virtual space. Specifically, the game progress unit 113 changes the position, orientation, shape, color, etc., of objects in the virtual space, and makes objects perform predetermined actions.

[0052] The game progress unit 113 defines a virtual camera for specifying the area to be presented to the user within the virtual space. The game progress unit 113 positions the virtual camera within the virtual space by defining its position and orientation. The game progress unit 113 instructs the display control unit 114 to generate an image that renders the field of view defined by the virtual camera and the objects placed within this field of view.

[0053] The position and orientation of the virtual camera can be determined as appropriate for each virtual space. For example, the game progress unit 113 uses the position and orientation of a specific object as a reference and positions the virtual camera so that the specific object is centered in the field of view with a specific orientation. In doing so, the game progress unit 113 adjusts the position and orientation of the virtual camera using the direction, distance, and angle relative to the specific object. The specific object may be a dynamic object, or a static object such as a building, tree, or stone. An example of a dynamic object is a character object.

[0054] The game progress unit 113 interprets the user's instructions based on the coordinates of the input position and the type of input operation detected by the operation reception unit 111. Based on the interpreted instructions, the game progress unit 113 executes various judgment processes related to the progress of the game. Based on the results of the judgment processes, the game progress unit 113 controls objects, virtual cameras, etc., to advance the game. The game progress unit 113 updates, adds, or deletes game information and user information according to the progress of the game.

[0055] The display control unit 114 causes the display unit 18 to display an image. A specific example will be given below.

[0056] The display control unit 114 generates an image that depicts the field of view of the virtual camera defined by the game progress unit 113 within the virtual space, along with the objects present in that field, and displays it on the display unit 18. The display control unit 114 can also superimpose objects related to the UI (User Interface) necessary for various game operations, such as icons, buttons, and menus showing various parameters, onto the image displayed on the display unit 18.

[0057] Note that the functions of the terminal device 10 and server 20 shown in Figure 2 are merely examples. Each of the terminal device 10 and server 20 may also possess at least some of the functions of other devices. Furthermore, each of the terminal device 10 and server 20, etc., does not have to be implemented by a single device, but may be implemented by multiple devices connected via a network, for example. Also, the information processing system 1 may consist of only the terminal device 10 or the server 20, for example. In other words, the information processing system 1 does not have to be implemented by multiple devices connected via a network.

[0058] <Processing according to this embodiment> Next, the processing according to this embodiment will be described. In this embodiment, the processor 11 of the terminal device 10 or the processor 21 of the server 20 will be described as performing the processing described later by executing the game program stored in the information processing system 1. However, at least a portion of the processing described later that is performed by the processor 11 may be executed by a processor other than the processor 11. Also, at least a portion of the processing described later that is performed by the processor 21 may be executed by a processor other than the processor 21. In other words, in this embodiment, the computer that executes the game program may be either the terminal device 10 or the server 20, or it may be realized by a combination of multiple devices.

[0059] In the game of this embodiment, users can raise their own character 30 (in other words, the character corresponding to the user). Character 30 may be a character that can battle against a predetermined opponent. Here, "opponent" may be a character owned by another user, or it may be an NPC (in other words, an enemy character controlled by the computer), etc. Also, character 30 may be something like a pet. In other words, character 30 may not be raised for the purpose of battling, but rather something to be cherished, etc.

[0060] Furthermore, "training" can be described as changing the parameters of the 30 characters. The changes to the 30 characters' parameters will be explained later.

[0061] In the game of this embodiment, the display control unit 114 controls the simultaneous display of a real-world image captured by the camera unit 19 and a virtual object (for example, a character 30) on the display unit 18, as shown in Figure 3. Specifically, the display control unit 114 displays the captured image and the character 30 simultaneously by displaying an overlaid image (in other words, an augmented reality image (AR image)) on the display unit 18, which is created by superimposing the character 30 onto the captured image. In other words, the display control unit 114 simultaneously displays the virtual object and the real-world object (hereinafter referred to as "real object") 32 that appears in the captured image. Note that in Figure 3, the displayed objects other than the character 30 (specifically, the banana, plate, and table) are real objects. The display control unit 114 may also simultaneously display the captured image and the character 30 by displaying them side by side on the display unit 18 without superimposing the character 30 onto the captured image (for example, by displaying the character and the captured image separately in each of the left and right divided areas). In other words, "simultaneously" means that the character 30 and the captured image are displayed on the same screen, and that at least a portion of the original image acquired by the camera as the shooting unit 19 is visible (in other words, a portion of the captured image may be hidden by the character 30, etc.). That is, "display of superimposed images" in the following explanation can be appropriately reinterpreted as displaying the captured image and the virtual object simultaneously.

[0062] Furthermore, in the game of this embodiment, the capture of images by the shooting unit 19 and the display of superimposed images are performed in real time. Specifically, the image of the area pointed at by the camera of the terminal device 10 is displayed on the display unit 18 as a captured image, and the character 30 is displayed on the display unit 18 superimposed on the captured image. In other words, in this embodiment, "capture (in other words, acquisition of a captured image)" is not limited to acquiring an image by pressing a predetermined button or the like to take a picture, but may also include pointing the camera at a real object 32, etc. Also, the captured image may be a video or a still image.

[0063] In this embodiment, the virtual object (for example, character 30) displayed simultaneously with the captured image can move on the screen. In other words, the game progress unit 113 can move character 30 or move different parts of character 30's body within the virtual space where the captured image is superimposed. In other words, the game progress unit 113 can move character 30 on the captured image. The movement of character 30 may be performed based on user input or independently of user input.

[0064] The game progression unit 113 acquires captured images and changes specific parameters based on these images. This will be explained in detail below. In this embodiment, unless otherwise specified, "change" of parameters generally means a change in a direction advantageous to the user (in other words, an increase in the parameter), but is not limited to this.

[0065] The specific parameter may be, for example, at least one of the following: the parameter of a virtual object (e.g., character 30) displayed simultaneously with the captured image, or a parameter indicating the amount of a predetermined value owned by the user who owns the virtual object (hereinafter also referred to as the "value parameter"). In other words, the specific parameter can also be said to be a parameter related to an in-game object such as character 30. The parameter of character 30 may, for example, be a parameter related to the character's abilities (e.g., strength). Specifically, the parameter of character 30 may be a parameter related to the character's level (e.g., the level itself or the experience points required to increase the level), or it may be a parameter indicating the character's intelligence, stamina, attack power, defense power, or speed. Furthermore, the character's parameter may, for example, be a parameter indicating the level of intimacy between the character and a specific person (e.g., the user who owns the character) (hereinafter simply referred to as "intimacy"). In other words, the character's parameter can also be called the character's status (in other words, status parameter). Furthermore, value parameters may include, for example, parameters indicating the amount of electronic currency owned by a user, or parameters relating to predetermined points (for example, points that can be exchanged for predetermined items or electronic currency in the virtual space, or predetermined objects in the real world).

[0066] The game progress unit 113 analyzes the captured image. Specifically, the game progress unit 113 identifies the real-world objects 32 contained in the captured image (in other words, the types of real-world objects 32 that appear in the captured image). For example, if a banana is visible in the captured image (see Figure 3), the game progress unit 113 identifies that a banana is present in the captured image. Similarly, if a book is visible in the captured image, the game progress unit 113 identifies that a book is present in the captured image. Furthermore, if money is visible in the captured image, the game progress unit 113 identifies that money is present in the captured image. The granularity of the analysis can be determined appropriately according to the content of the game; for example, it may be identified at the level of bananas, apples, grapes, or at the level of fruit or food. Also, the real-world objects 32 may be identified using a classification different from that of real-world objects 32 in the real world. Moreover, the types of real-world objects identified (in other words, real-world objects related to the fluctuation of specific parameters) are not particularly limited; for example, they may be not limited to food or books, but may also include living things or landmarks of various locations.

[0067] Regarding methods for analyzing images to identify real-world objects, various methods have been proposed in the past, and various publicly known methods can be used. Furthermore, image analysis may be performed on an external server or other device outside of the information processing system 1.

[0068] The game progress unit 113 changes specific parameters based on the analysis results of the captured image. Specifically, the game progress unit 113 changes specific parameters when the analysis results of the captured image show that a specific real-world object 32 is included in the captured image. In other words, the game progress unit 113 changes specific parameters based on the fact that a specific real-world object 32 and a character 30 as a virtual object are displayed together.

[0069] For example, if the game progress unit 113 determines, as a result of analyzing the captured image, that the captured image contains a banana as a specific real-world object 32, it will increase a specific parameter (for example, intimacy as a parameter of character 30). Also, for example, if the game progress unit 113 determines, as a result of analyzing the captured image, that the captured image contains a book as a specific real-world object, it will increase a specific parameter (for example, intelligence as a parameter of character 30). Also, for example, if the game progress unit 113 determines, as a result of analyzing the captured image, that the captured image contains money, it will increase a specific parameter (for example, a value parameter, specifically the amount of in-game currency).

[0070] Furthermore, the game progression unit 113 varies specific parameters based on the location where the captured image, which is displayed simultaneously with the virtual object (e.g., character 30), was taken (hereinafter also referred to as the "capture location"). The capture location may be determined, for example, based on location information acquired by a GPS sensor or the like as the input unit 17 when the image is taken, or it may be determined by image analysis from the objects shown in the captured image. Various known methods can be used to determine the capture location.

[0071] For example, the game progression unit 113 increases a specific parameter (e.g., the intimacy level of character 30) when a banana is included in the captured image, but it controls the change in the specific parameter differently depending on whether the image is taken at a first position or a second position. "Controlling the change differently" means, for example, that the increase in a predetermined parameter in one case is larger than in the other. It also means, for example, that a parameter that is not increased in one case is increased in the other. Furthermore, in the following explanation, "favorable to the user" in relation to parameter changes includes cases where the increase in the parameter is large, or where a parameter that is not increased in one case is increased. In other words, for the disadvantageous side of parameter changes, no parameter change may occur at all.

[0072] Specifically, the game progression unit 113 increases the rate at which a predetermined parameter (e.g., intelligence) of character 30 increases when the shooting location of a photograph containing a banana is "Philippines" compared to when the shooting location is "Tokyo". In addition, the game progression unit 113 increases a parameter (e.g., value parameter) that is not increased when the shooting location is "Tokyo" when the shooting location of a photograph containing a banana is "Philippines".

[0073] Furthermore, the game progression unit 113 may vary certain parameters based on the correspondence between the shooting position and at least two of the real-world object 32 and the virtual object. Such correspondences may be pre-registered in a predetermined memory unit (for example, memory unit 120 or memory unit 220).

[0074] For example, the game progress unit 113 may change specific parameters based on the correspondence between the character 30 and the shooting location. For example, when the game progress unit 113 changes specific parameters based on a shooting image displayed simultaneously with a character 30 that has the attribute "otaku attribute" set as a predetermined attribute, if the shooting location is a place corresponding to the "otaku attribute" (for example, a real-world place favored by otaku), the changes will be more favorable to the user than when the shooting location is not a place corresponding to the "otaku attribute". Also, for example, when the game progress unit 113 changes specific parameters based on a shooting image displayed simultaneously with a character 30 that has the attribute "water attribute" set as a predetermined attribute, if the shooting location is near the sea or a river, which are places corresponding to the "water attribute", the changes will be more favorable to the user than when the shooting location is far from the sea or a river, which are places not corresponding to the "water attribute".

[0075] Furthermore, for example, the game progress unit 113 may change specific parameters based on the correspondence between the real-world object 32 and the shooting location. For example, regarding the change of specific parameters based on the inclusion of a banana in the captured image displayed simultaneously with the character 30, the game progress unit 113 will make a change to the specific parameter that is more favorable to the user when the shooting location is "Philippines," which corresponds to the banana, compared to the case where the shooting location is "Tokyo," which does not correspond to the banana. Similarly, regarding the change of specific parameters based on the inclusion of an apple in the captured image displayed simultaneously with the character 30, the game progress unit 113 will make a change to the specific parameter that is more favorable to the user when the shooting location is "Aomori," which corresponds to the apple, compared to the case where the shooting location is "Tokyo," which does not correspond to the apple. In other words, if a specific real-world object 32 (for example, a banana) is included in a captured image taken at a specific location (for example, the Philippines), the game progress unit 113 may perform a specific process that is more favorable to the user regarding the change of the specific parameter (for example, a process that increases the magnitude of the parameter's increase).

[0076] Furthermore, for example, the game progression unit 113 may change certain parameters based on the correspondence between the character 30 and the real-world object 32. For example, regarding the change of certain parameters based on whether an apple is included in the captured image displayed simultaneously with the character 30, the game progression unit 113 may make a change that is more advantageous to the user if the character 30 is a character that corresponds to an apple (for example, a character with the attribute "apple lover" set), compared to if the character 30 is a character that does not correspond to an apple (for example, a character without the attribute "apple lover" set).

[0077] Furthermore, for example, the game progression unit 113 may change certain parameters based on the correspondence between the character 30, the real-world object 32, and the shooting location. For example, the game of this embodiment may have a first character with the attribute "likes Aomori apples" (or "likes apples") and a second character without the attribute "likes Aomori apples" (or "likes apples") (in other words, a character that corresponds to Aomori apples and a character that does not). The game progression unit 113 may then change certain parameters based on whether an apple is included in the captured image displayed simultaneously with the character 30 in a way that is more advantageous to the user when the shooting location is "Aomori" and the character 30 is the first character, compared to when the character 30 is the second character or when it is the first character but the shooting location is not "Aomori".

[0078] Furthermore, the game progress unit 113 may change certain parameters based on the time when the captured image displayed simultaneously with the virtual object (for example, character 30) was taken (hereinafter also referred to as "shooting time"). In other words, the game progress unit 113 may control the changes in certain parameters differently depending on the shooting time. For example, the game of this embodiment may have a first character with the attribute of "nocturnal" and a second character with the attribute of "diurnal" (in other words, a character that corresponds to night and a character that does not correspond to night). The game progress unit 113 may then change certain parameters based on the captured image displayed simultaneously with the first character in a way that is more advantageous to the user when the shooting time is at night compared to when the shooting time is during the day. Specifically, for example, the game progress unit 113 may, when the shooting time is nighttime, perform changes to specific parameters (for example, changes to specific parameters due to eating real-world objects 32 included in the captured image) as if the first character is active, and when the shooting time is daytime, not perform changes to specific parameters (for example, changes to specific parameters due to eating real-world objects 32 included in the captured image) as if the first character is inactive.

[0079] Furthermore, the game progress unit 113 may vary certain parameters based on a predetermined reference position and a shooting position relating to the user who owns the virtual object (e.g., character 30). Here, the reference position may be, for example, the user's place of residence. That is, in the game of this embodiment, the user may be able to set the location of their own home (the location of their home on a map). The game progress unit 113 may then make changes to the certain parameters that are more favorable to the user when the captured image displayed simultaneously with the character 30 is taken at a first location far from the location set as the home of the user who owns the character 30, compared to when the image is taken at a second location closer to the home than the first location. In other words, when the game progress unit 113 varies certain parameters based on the captured image, it may make changes that correspond to the user's travel distance in the real world. Here, "travel distance" is not limited to the distance on the map from the reference position, but may also be, for example, the total distance traveled by the user within a predetermined period, or the distance traveled from the previous shooting to the next shooting.

[0080] Furthermore, the locations on the map that a user can set as their place of residence may be limited to areas corresponding to the user's range of activity in the real world. Specifically, for example, the locations on the map that a user can set as their home may be limited to locations that correspond to places the user has been to in the real world, places the user is currently at in the real world, places the user has stayed at for a predetermined amount of time or more (e.g., continuously or cumulatively) in the real world, or places the user visits a predetermined number of times or with a predetermined frequency in the real world. In other words, the locations where a user can set as their place of residence may be limited to places that the game progress unit 113 can infer are where the first user actually lives.

[0081] Furthermore, the reference location does not have to be the user's place of residence. Also, the reference location does not have to be set by the user. Specifically, the game progress unit 113 may automatically set the reference location to a place in the real world where the user has stayed for a predetermined amount of time or more (for example, continuously or cumulatively), or a place in the real world that the user visits a predetermined number of times or at a predetermined frequency.

[0082] Furthermore, the game progression unit 113 may limit the number of times it can perform changes to specific parameters based on captured images.

[0083] Specifically, for example, the game progress unit 113 may limit the number of times it can change a specific parameter based on a captured image containing the same real-world object 32 within a predetermined period. More specifically, for example, the game progress unit 113 may limit the number of times it can change a specific parameter based on the same real-world object 32 (e.g., a banana) to N times per day (where N is a natural number greater than or equal to "1"). For example, if the game progress unit 113 has changed a specific parameter N times within a predetermined period based on the inclusion of a banana in a real-world image displayed simultaneously with the character 30, it may prohibit further changes to the specific parameter based on the inclusion of a banana in a real-world image displayed simultaneously with the character 30 until a certain period has elapsed. Here, the predetermined period may be, for example, a period starting from a predetermined time (e.g., 0:00 every day), or a period starting from the time the captured image containing the specific real-world object 32 was taken (in other words, a period determined based on the user's actions).

[0084] Furthermore, the game progress unit 113 may, based on the fact that a change in a specific parameter based on the inclusion of a specific real-world object 32 in a captured image acquired at the first position has already been performed, prohibit the change in a specific parameter based on the inclusion of the specific real-world object 32 in a captured image acquired at the first position. In a state where the change in a specific parameter based on the inclusion of the specific real-world object 32 in a captured image acquired at the first position is prohibited, the game progress unit 113 may perform a change in a specific parameter based on the inclusion of the specific real-world object 32 in a captured image acquired at the second position. For example, if the game progress unit 113 has performed a change in a specific parameter based on the inclusion of a banana in a captured image taken at the first position that is displayed simultaneously with the character 30 N times within a predetermined period, the game progress unit 113 may prohibit the change in a specific parameter based on the inclusion of a banana in a captured image taken at the first position until a certain period has elapsed, while still performing a change in a specific parameter based on the inclusion of a banana in a captured image taken at the second position during that certain period. The second position may be a position located at or beyond a predetermined distance from the first position.

[0085] Furthermore, when a predetermined condition is met (for example, when a specific parameter is changed N times within a predetermined period based on the inclusion of a specific real object 32 in a real image displayed simultaneously with the character 30), and the change of a specific parameter based on the captured image is restricted (in other words, prohibited), the change of a specific parameter by using a specific item may be permitted. The specific item may be, for example, an item that a user can obtain by spending their own assets (for example, electronic currency) (in other words, a paid item). The specific item may also be, for example, a virtual object that can be used (in other words, given) to a character (for example, food or a book). The specific item may also be consumed upon use. The specific item is used, for example, based on the user's input operation to select an item to use on the character 30. When the specific item is used, the display control unit 114 displays the specific item and the character 30 simultaneously on the display unit 18, and also displays a display indicating that the character 30 is performing an action on the specific item (for example, an action of eating the specific item). When the specific item is used, the game progression unit 113 modifies the specific parameters, such as increasing the parameters of the character 30. The screen on which the specific item and character 30 are displayed may be an augmented reality image with a captured image as the background. That is, it may be possible to use the specific item to modify the specific parameters when an image in which character 30 is superimposed on a captured image is displayed on the display unit 18 (for example, the state illustrated in Figure 3).

[0086] Furthermore, the game progress unit 113 may impose a penalty on the user (for example, a predetermined penalty related to a specific parameter) if the same real-world object 32 is photographed multiple times from different shooting positions. In this embodiment, the "penalty" may be, for example, one that decreases a specific parameter (in other words, changes it in a direction unfavorable to the user), or one that prevents the increase of a specific parameter based on photographing the real-world object 32 (in other words, displaying the real-world object 32 and the character 30 simultaneously) for a predetermined period of time. Specifically, the game progress unit 113 may impose a penalty on the user and decrease the specific parameter if, after a photograph containing a specific real-world object 32 is taken at a first position and displayed simultaneously with the character 30, and a change in a specific parameter is performed based on this, the same photograph containing the real-world object 32 is taken at a second position and displayed simultaneously with the character 30. In other words, the game progress unit 113 may impose a penalty if the user attempts to increase a specific parameter using the same real-world object 32 by changing the shooting position. With this configuration, in a configuration where the variation of a specific parameter using the same real object 32 at the same location is restricted, it is possible to impose a penalty on a user who attempts to circumvent this restriction and vary the specific parameter.

[0087] Furthermore, when multiple images are taken of a virtual object (e.g., character 30) simultaneously with other images within a predetermined period, the game progress unit 113 may, in cases where the user (terminal device 10) moves with each shot and the type of real-world object 32 included in the captured image differs with each shot, make fluctuations in specific parameters that are more favorable to the user compared to cases where the user does not move with each shot and the multiple shots are taken (and the type of real-world object 32 included in the captured image differs with each shot), and cases where the type of real-world object 32 included in the captured image is the same for all multiple shots (and the user moves with each shot and the multiple shots are taken). Also, when multiple images are taken of a virtual object (e.g., character 30) simultaneously with other images, the game progress unit 113 may, in cases where a predetermined number of shots are taken within a predetermined period with the user (terminal device 10) moving with each shot and the type of real-world object 32 included in the captured image differs with each shot, make fluctuations in specific parameters that are more favorable to the user compared to cases where the predetermined number of shots are taken beyond the predetermined period. Specifically, the game progress unit 113 may, in the event that an image containing an apple is taken, then the user moves and an image containing a banana is taken, then the user moves and an image containing grapes is taken, and the events from the apple to the grapes are taken within a predetermined period, make changes to a specific parameter that are more favorable to the user than in the case where the user does not move at all and images containing an apple, an apple, and grapes are taken sequentially, or an image containing an apple is taken, then the user moves and an image containing an apple is taken, then the user moves and an image containing an apple is taken, or an image containing an apple is taken, then the user moves and an image containing a banana is taken, then the user moves and an image containing grapes is taken, but the events from the apple to the grapes are not taken within a predetermined period.

[0088] Furthermore, the control unit 110 may have a detection unit 115 capable of detecting whether or not the real object 32 captured in the captured image is an object that actually exists in that location. Specifically, the control unit 110 may have a detection unit 115 capable of detecting whether or not the real object 32 captured in the captured image has a three-dimensional shape. For example, the detection unit 115 may determine whether or not the real object 32 captured in the captured image has a three-dimensional shape based on information from the depth camera, which acts as the input unit 17. Alternatively, for example, the detection unit 115 may determine whether or not the real object 32 has a three-dimensional shape based on images of the real object 32 captured from multiple directions by moving the terminal device 10 or the real object 32 during the capture of the image (in other words, the user is instructed to capture the real object 32 from multiple directions, and the detection unit 115 determines whether or not the real object 32 has a three-dimensional shape based on the captured images obtained in accordance with that instruction). Furthermore, the game progress unit 113 may, if it determines that the real-world object 32 included in the captured image has a three-dimensional shape, execute (in other words, permit) a change in a specific parameter based on the captured image, and if it determines that the real-world object 32 included in the captured image does not have a three-dimensional shape, it may not execute (in other words, dispermit) a change in a specific parameter based on the captured image (specifically, a change in a direction favorable to the user). That is, if the real-world object 32 in the captured image is, for example, an image displayed on a predetermined display or an image printed on paper, the game progress unit 113 may not execute a change in a specific parameter based on the captured image (specifically, a change in a direction favorable to the user). In addition, the game progress unit 113 may impose a penalty on the user if it determines that the real-world object 32 included in the captured image does not have a three-dimensional shape. That is, for example, if the user tries to increase a specific parameter by taking a picture of an image of a banana displayed on a display, the game progress unit 113 may detect the fraud and impose a penalty such as decreasing the specific parameter.

[0089] Furthermore, the game progress unit 113 may determine the real-world value of the real-world objects 32 included in the captured image and adjust specific parameters according to that value. For example, the game progress unit 113 may control the system so that the higher the value of the real-world objects included in the captured image, the greater the increase in a specific parameter (e.g., the increase in the value parameter). The determination of the value of the real-world objects 32 may be performed using AI (Artificial Intelligence). The AI ​​may be either machine learning type or non-machine learning type (rule-based type). For example, the correspondence between the types of real-world objects 32 and their real-world values ​​(e.g., market prices in the real world) may be stored in a predetermined memory unit (e.g., memory unit 220), and the game progress unit 113 may analyze the captured image and determine the real-world value of the identified real-world objects 32. Alternatively, machine learning may be performed based on images of various real-world objects 32 and information indicating the real-world value of those real-world objects 32, and the resulting model may be used (in other words, by providing the captured images as input to the model) to determine the real-world value of the real-world objects 32 included in the captured images. Furthermore, the game progression unit 113 may analyze the captured images to identify the types of real-world objects and determine the real-world value of the real-world objects 32 included in the captured images by searching for their values ​​on the internet (in other words, by searching for the values ​​of the real-world objects 32 included in the captured images on the internet based on the captured images).

[0090] Furthermore, the game progress unit 113 may perform a change in specific parameters based on the simultaneous display of a captured image and a virtual object (e.g., a character 30) when the positional relationship between a specific real-world object 32 included in the captured image and a virtual object (e.g., a character 30) superimposed on the captured image reaches a predetermined positional relationship. Specifically, the game progress unit 113 may refrain from changing the specific parameters if the specific real-world object 32 and the character 30 overlap by a predetermined amount or more, and perform a change in the specific parameters when the overlap is less than a predetermined amount (e.g., when there is no overlap at all). In other words, the change in specific parameters may require that the specific real-world object 32 be photographed so as not to overlap with the character 30. Separately from, or in addition to, the game progress unit 113 may perform a change in specific parameters based on the fact that the specific real-world object 32 and the character 30 overlap by at least a portion. Specifically, the game progress unit 113 may identify the type of real-world object 32 that overlaps with the character 30 and perform a change in specific parameters based on the result of the identification. With this configuration, it is possible to limit the objects in the captured image whose type can be identified through image analysis, thereby reducing the processing burden related to image analysis.

[0091] Furthermore, the game progress unit 113 may store a history of the capture of images displayed simultaneously with virtual objects (hereinafter also referred to as "capture history") in a predetermined storage unit (for example, storage unit 120). For example, the history of the location where the captured image was taken may be stored as the capture history. Also, for example, the captured image itself may be stored as the capture history. Also, for example, the types of real-world objects 32 included in the captured image may be stored as the capture history. Also, for example, the details of the changes in specific parameters that were performed (specifically, which parameters increased, how many parameters increased, which character 30's parameters increased, etc.) may be stored as the capture history in association with each captured image, the location where the captured image was taken, or the types of real-world objects 32 included in the captured image. Note that the storage of captured images may be performed in the state of superimposed images.

[0092] Furthermore, the game progress unit 113 may display the shooting history on the display unit 18 based on a predetermined operation by the user to display the shooting history on the display unit 18. Specifically, for multiple past shots, at least one (or all) of the following may be displayed on the display unit 18 in a list or the like: the captured image (for example, the captured image in the state of the superimposed image), the location where the captured image was taken, the type of real-world object included in the captured image, and the details of the changes in specific parameters that were performed. With such a configuration, for example, it becomes possible to investigate from the shooting history what kinds of real-world items have an effect on the virtual object owned by the user, such as character 30 (for example, the effect on parameter changes, or whether character 30 is pleased, etc.), and the user can be given the enjoyment of searching for such things. Also, with such a configuration, for example, it becomes possible to leave a record in the form of a shooting history of the places the user has visited with the virtual object such as character 30, and the user can be given the enjoyment of traveling together with character 30.

[0093] In this embodiment, the game progression unit 113 changes a specific parameter when the captured image includes a specific real-world object 32, but the specific real-world object 32 may be a person.

[0094] Furthermore, in a configuration in which the game progress unit 113 changes specific parameters when a captured image containing a person as a specific real-world object 32 is displayed simultaneously with a character 30 as a virtual object, the game progress unit 113 may change the specific parameters based on the facial expression of the person included in the captured image. For example, if the person has a smiling expression, the game progress unit 113 may change the specific parameters in a way that is more favorable to the user compared to if the person has an angry expression.

[0095] Furthermore, in a configuration in which the game progress unit 113 changes specific parameters when a captured image containing a person as a specific real object 32 is displayed simultaneously with a character 30 as a virtual object, if the person included in the captured image is a specific person, the game progress unit 113 may change a parameter of the character 30 that is related to that specific person (for example, the intimacy level between the character 30 and the specific person). The specific person may be, for example, a person registered in the game as the owner of the character 30. Alternatively, the specific person may be, for example, a person pictured in an image (e.g., a photograph) stored in a predetermined image storage area (e.g., a so-called photo folder) in the memory unit 120 of the terminal device 10 (in other words, the owner of the terminal device 10 or a person inferred from the image to be related to the owner of the terminal device 10).

[0096] Furthermore, the game progress unit 113 may perform a process to change specific parameters based on a captured image displayed simultaneously with a virtual object, based on the weather conditions at the time the image was taken. Specifically, a weather information acquisition unit (not shown) in the control unit 110 may acquire weather information indicating the weather conditions at the time the image was taken, and the game progress unit 113 may change specific parameters based on this weather information. The weather information acquisition unit may, for example, acquire weather information indicating the weather conditions of the area where the image was taken, based on location information indicating the user's location. Specifically, for example, the weather information acquisition unit may access a website that publishes weather information or a predetermined server that stores weather information, and acquire weather information for the area corresponding to the location information. The weather information acquisition unit may also acquire weather information indicating the weather conditions (e.g., weather) at the time the image was taken by performing image analysis on the captured image. The weather information acquisition unit may also acquire weather information from a predetermined sensor (e.g., temperature sensor) provided by the terminal device 10. The weather information may be information about the weather, such as "sunny," "rainy," "cloudy," or "snowy," or it may be information about temperature. The game progression unit 113 may, when a captured image displayed simultaneously with the character 30 includes a specific real-world object 32 (e.g., juice), and when the captured image is taken under a first weather condition with relatively low temperatures, make changes to certain parameters that are more favorable to the user compared to when the captured image is taken under a second weather condition with higher temperatures than the first weather condition. With such a configuration, it is possible to express that the character 30 is exhausted due to the heat, etc. The character 30 may also have a compatibility setting with different weather conditions.For example, for a first character 30 (e.g., a character 30 that prefers cold climates), if the captured image is taken under first weather conditions (e.g., relatively low temperatures), the fluctuations of specific parameters may be more favorable to the user than if the captured image is taken under second weather conditions (e.g., relatively high temperatures). Similarly, for a second character 30 (e.g., a character 30 that prefers hot climates), if the captured image is taken under second weather conditions, the fluctuations of specific parameters may be more favorable to the user than if the captured image is taken under first weather conditions.

[0097] Furthermore, the game progress unit 113 may perform a process of changing specific parameters based on a captured image displayed simultaneously with a virtual object, based on the ambient sound conditions at the time the image is captured. Specifically, the game progress unit 113 may change specific parameters based on the sound input from the microphone, which acts as the input unit 17. For example, if the captured image is taken in a first situation where ambient sound is relatively low (in other words, a quiet situation), the game progress unit 113 may change the specific parameters in a way that is more favorable to the user compared to a second situation where ambient sound is louder than in the first situation (in other words, a noisy situation). Also, if the captured image is taken in a first situation where music preferred by the character 30, which is displayed simultaneously with the image, is playing, the game progress unit 113 may change the specific parameters in a way that is more favorable to the user compared to a second situation where music preferred by the character 30 is not playing. Such preferred music may be pre-set for each character 30 by the game's creators, or it may be set by the user who owns the character 30. Alternatively, the character 30 may remember music that is frequently played during gameplay (for example, during recording) as their preferred music (in other words, it may be set according to the user's gameplay).

[0098] In this embodiment, the capture of the image and the simultaneous display of the captured image and the virtual object are performed in real time, and the game progress unit 113 adjusts specific parameters based on the analysis results of the captured image. However, the capture of the image and the simultaneous display of the captured image and the virtual object do not necessarily have to be performed in real time. For example, the game progress unit 113 may simultaneously display the selected image and the virtual object based on the user's input operation to select an image to apply to the virtual object (e.g., character 30) from among several images taken in the past, and adjust specific parameters based on the analysis results of the selected image. In other words, the "captured image" in this embodiment may be an image taken in the past. In such a configuration, the captured image taken in the past may have a freshness (in other words, an expiration date). Specifically, for example, for captured images containing apples, the more recent the capture date (the less time has passed since the image was taken), the more favorable the adjustment of specific parameters may be to the user when selected. Furthermore, with respect to images containing apples, if a certain period of time has elapsed since the image was taken (in other words, if the expiration date has passed), then when the image is selected, the change in a specific parameter may not occur, or the user may be penalized (in other words, character 30 may become ill, etc.). In addition, such expiration dates may differ for each real object 32 included in the image (for example, whether it is an apple, fish, meat, etc.).

[0099] Furthermore, when the game progress unit 113 increases a specific parameter based on the fact that a specific real-world object 32 is included in the captured image displayed simultaneously with the virtual object, it may cause the virtual object to perform a predetermined action (in other words, a predetermined animation) for that specific real-world object. For example, the game progress unit 113 may cause the character 30, as the virtual object, to perform the action of eating food, which is the specific real-world object. In other words, the game progress unit 113 instructs the display control unit 114 to display the animation of the character 30 eating the food on the display unit 18. Also, for example, the game progress unit 113 may cause the character 30, as the virtual object, to perform the action of reading a book, which is the specific real-world object. In other words, the game progress unit 113 instructs the display control unit 114 to display the animation of the character 30 reading the book on the display unit 18. The game progress unit 113 may also cause the character 30 to say a predetermined line such as "It's a feast." Specifically, the game progression unit 113 may instruct the display control unit 114 to display the predetermined lines on the display unit 18, or it may output the predetermined lines as sound from the speaker (in other words, the sound output unit) of the terminal device 10.

[0100] Furthermore, the game progress unit 113 may cause the virtual object to perform a specific animation if the captured image displayed simultaneously with the virtual object was taken at a location having a predetermined relationship with the virtual object, and may not cause the virtual object to perform a specific animation if the captured image was not taken at a location having a predetermined relationship with the virtual object. Specifically, for example, when changing a specific parameter based on a captured image displayed simultaneously with a character 30 that has the attribute "otaku attribute," the game progress unit 113 may cause the character 30 to perform a predetermined action that would not be performed if the captured location did not correspond to the "otaku attribute," if the captured location is a place corresponding to the "otaku attribute." Also, for example, when changing a specific parameter based on a captured image displayed simultaneously with a character 30 that has the attribute "Aomori," the game progress unit 113 may cause the character 30 to perform a predetermined action that would not be performed if the captured location is a place other than "Aomori," if the captured location is "Aomori." Furthermore, the game progression unit 113 may cause the virtual object to perform a specific action if the captured image displayed simultaneously with the virtual object was taken at a location having a predetermined relationship with the virtual object and includes a specific real-world object (for example, a real-world object having a predetermined relationship with the virtual object). Specifically, for example, when changing a specific parameter based on a captured image displayed simultaneously with character 30, if the shooting location is "Aomori" and the captured image includes apples, the game progression unit 113 may cause character 30 to perform a predetermined action that would not be performed if the shooting location was anywhere other than "Aomori" or if the captured image did not include apples.

[0101] Furthermore, conventional map applications are known that allow users to post reviews for various locations on a map (e.g., facilities). Specifically, applications are known that allow users to post reviews and ratings for restaurants and other establishments associated with various points on a map. The game progress unit 113 may change specific parameters based on information associated with such locations on the map that correspond to the shooting location where the captured image displayed simultaneously with the character 30 was taken. For example, the game progress unit 113 may change specific parameters according to the attributes set for the location on the map that corresponds to the shooting location where the captured image displayed simultaneously with the character 30 was taken. Specifically, the game progress unit 113 may perform different control over the change of specific parameters when the attribute of the location corresponding to the shooting location (e.g., facility) is "restaurant" compared to when the attribute of the location corresponding to the shooting location is "school". Also, for example, the game progress unit 113 may change specific parameters according to the reviews posted (e.g., the number of reviews, the content of the reviews, etc.) for the location on the map that corresponds to the shooting location where the captured image displayed simultaneously with the character 30 was taken. Specifically, the game progress unit 113 may perform different control over the fluctuations of certain parameters when there are many reviews compared to when there are few reviews. Furthermore, the game progress unit 113 may perform different control over the fluctuations of certain parameters when the review ratings are high compared to when the review ratings are low.

[0102] Furthermore, in a configuration where specific parameters are changed based on information associated with a map location corresponding to the shooting location of a captured image displayed simultaneously with the character 30, the game progress unit 113 may change the specific parameters without analyzing the captured image. For example, when a captured image displayed simultaneously with the character 30 is taken, the game progress unit 113 may determine the map location corresponding to the shooting location based on location information acquired by a GPS sensor or the like as an input unit 17, and change the specific parameters based on information associated with the determined location (e.g., facility attributes or reviews) (in other words, regardless of the real-world objects 32 shown in the captured image). Alternatively, for example, the game progress unit 113 may change specific parameters based on the fact that a user has posted a captured image displayed simultaneously with the character 30 (e.g., an overlaid image) as a review associated with a specific location on the map. In other words, the change in specific parameters may be triggered by the review. The shooting location and the map location corresponding to the shooting location may also be determined based on user input operations such as posting a review. In other words, the game progress unit 113 may determine the location where the captured image was posted as a review as the location where the image was taken.

[0103] Furthermore, the game progress unit 113 may set the aforementioned correspondence between character 30 and real-world objects 32, and the aforementioned correspondence between character 30 and the shooting location, etc., based on the fact that the user has posted a captured image (e.g., an overlaid image) displayed simultaneously with character 30 as a review associated with a specific location on the map. For example, the game progress unit 113 may set character 30 as a "yakitori lover" (in other words, set the correspondence between character 30 and real-world objects 32) or set character 30 as a "yakitori restaurant lover" (in other words, set the correspondence between character 30 and the shooting location), based on the fact that the user has posted a captured image (e.g., an overlaid image) displayed simultaneously with character 30 as a review of a yakitori restaurant as a specific location on the map. Then, regarding the changes in certain parameters based on the fact that yakitori is included in the captured image displayed simultaneously with character 30, the game progress unit 113 may make changes that are more favorable to the user when character 30 is set as a "yakitori lover" compared to when character 30 is not set as a "yakitori lover".

[0104] Furthermore, when processing related to changes in specific parameters based on the attributes and reviews of facilities on the map, the map may be a map within the application that provides the game of this embodiment, or it may be a map from another application (for example, a map application) or a website. In other words, the application that provides the game of this embodiment may be linked with other applications or websites using known technology, and the game progress unit 113 may acquire information about facilities and reviews set on the map and perform the various processing described above.

[0105] Furthermore, the game progress unit 113 may change the appearance of a virtual object (e.g., character 30) that is displayed simultaneously with the captured image, based on the captured image. The display control unit 114 may then, based on instructions from the game progress unit 113, display on the display unit 18 a display indicating that the appearance of the virtual object has changed based on the captured image. Specifically, for example, the game progress unit 113 may change the appearance of the character 30 as a virtual object (e.g., shape, pattern, or color) to reflect the characteristics of a specific real-world object 32 included in the captured image. For example, if the captured image includes a zebra as a real-world object 32, the game progress unit 113 may change the appearance of the character 30 displayed simultaneously with the zebra to have black and white stripes. Also, for example, if the captured image includes a giraffe as a real-world object 32, the game progress unit 113 may change the appearance of the character 30 displayed simultaneously with the giraffe to have an elongated neck or horns. In other words, the game progression unit 113 may change (in other words, inherit) the appearance of the character 30 to resemble a specific real-world object 32 included in the captured image. Furthermore, the changed appearance may be maintained even after the specific real-world object 32 disappears from the captured image. With such a configuration, it becomes possible to save the encounter with the specific real-world object 32 as a memory (in the form of the appearance of the virtual object). In such a configuration, the game progression unit 113 may change specific parameters (for example, the parameters of the character 30) and the appearance of the virtual object based on the captured image displayed simultaneously with the virtual object (for example, when the captured image includes a specific real-world object 32), or it may change the appearance of the virtual object without changing the specific parameters. In other words, a configuration that changes the appearance of a virtual object based on a captured image may be implemented without a configuration that changes specific parameters based on a captured image.

[0106] (Processing related to changes in specific parameters based on captured images) An example of processing related to changes in specific parameters based on captured images will be explained with reference to the flowchart shown in Figure 4.

[0107] First, the game execution unit 113 acquires a captured image of the real world (step S101).

[0108] Next, the game progress unit 113 analyzes the captured image (step S102). Specifically, the game progress unit 113 identifies the real-world objects 32 included in the captured image.

[0109] Next, the game progress unit 113 determines whether the captured image contains a specific real-world object 32 related to the increase of a specific parameter (step S103).

[0110] If a specific real-world object 32 related to the increase of a specific parameter is included in the captured image (YES in step S103), the game progress unit 113 increases the specific parameter (step S104). Specifically, the game progress unit 113 determines the type and amount of increase of the parameter to be increased according to the type of the specific real-world object 32, and increases the specific parameter based on this determination. In addition to or instead of the type of the specific real-world object 32, the game progress unit 113 may also determine the type and amount of increase of the parameter to be increased (or whether or not to increase the specific parameter) based on at least one of the following: the shooting location, the shooting time, the correspondence between the shooting location and at least two of the real-world object 32 and virtual objects, the real-world value of the real-world object 32, the weather conditions at the time of shooting, and the ambient sound conditions at the time of shooting, and increases the specific parameter based on this determination.

[0111] On the other hand, if a specific real-world object 32 related to the increase of a specific parameter is not included in the captured image (NO in step S103), the game progression unit 113 does not increase the specific parameter.

[0112] The game progress unit 113 instructs the display control unit 114 to display a screen on the display unit 18 in which the captured image and the character 30 as a virtual object are displayed simultaneously. The display control unit 114 then displays the screen on the display unit 18 based on this instruction. However, the timing of this instruction (in other words, the timing at which the display control unit 114 displays the screen on the display unit 18) may be at any point after step S1.

[0113] Furthermore, the present invention is not limited to the embodiments described above, and can be implemented in various modified forms without departing from its essence. The present invention is also applicable to services other than games. Within the scope of the invention, the components can be freely combined, any component can be modified, or any component can be omitted. Also, the processing flow described herein is merely an example, and the order and configuration of each process may differ. In addition, some processes shown in the flowchart may not exist. In other words, the processing flow and specific decision processes may differ from those exemplified herein.

[0114] <Note> The matters described in the above embodiments may also be described as follows:

[0115] (Note 1) The processor, A display control means (for example, a display control unit 114) that simultaneously displays captured images of the real world and virtual objects, Based on the captured image displayed simultaneously with the virtual object, a control means (for example, a game progress unit 113) is provided to vary specific parameters, and this unit functions as such. The aforementioned specific parameter is at least one of the parameters of the virtual object and a parameter indicating a predetermined amount of value owned by the user who owns the virtual object. program. With this configuration, it becomes possible to change the parameters of virtual objects based on images taken of the real world, or to change the amount of value owned by the user related to the virtual objects. Therefore, the connection between the virtual space and the real world can be strengthened.

[0116] (Note 2) The control means is The specific parameter is varied based on the position where the captured image, which is displayed simultaneously with the virtual object, was acquired. The program described in Appendix 1. With this configuration, it is possible to correlate the real-world location where the captured image was acquired with the fluctuations of specific parameters, thereby strengthening the connection between the virtual space and the real world.

[0117] (Note 3) The control means is Depending on whether the captured image displayed simultaneously with the virtual object was acquired at a position having a predetermined relationship with the virtual object, the control related to the variation of the specific parameter is varied. The program described in Appendix 2. This configuration allows for a correspondence between the real-world location where the captured image is acquired and the virtual object, thereby strengthening the connection between the virtual space and the real world.

[0118] (Note 4) The control means is The specific parameter is varied based on a predetermined reference position relating to the user who owns the virtual object and the position from which the captured image, which is displayed simultaneously with the virtual object, was acquired. The program described in Appendix 1. With this configuration, specific parameters can be varied based on the relative relationship between the user's reference position and the position where the captured image is acquired. This allows for the association of the user with real-world locations that have a predetermined relationship with the user, thereby strengthening the connection between the virtual space and the real world.

[0119] (Note 5) The control means is When the captured image displayed simultaneously with the virtual object is acquired at a position having a predetermined relationship with the virtual object, the virtual object is made to perform a specific animation; when the image is not acquired at a position having a predetermined relationship with the virtual object, the virtual object is not made to perform the specific animation. The program described in one of the appendices 1-4. With this configuration, a correspondence can be established between the real-world location where the captured image is taken and the virtual object, and this correspondence can be reflected in the visual presentation, allowing users to strongly feel that the virtual space and the real world are closely connected.

[0120] (Note 6) The control means causes the virtual object to perform the specific performance when the captured image, which is displayed simultaneously with the virtual object, is taken at a position having a predetermined relationship with the virtual object and includes a specific object. The program described in Appendix 5. With this configuration, a correspondence can be established between the real-world location where the captured image is taken and between the virtual object and the real-world object. This correspondence can then be reflected in the visual presentation, allowing users to strongly feel that the virtual space and the real world are closely connected.

[0121] (Note 7) The processor, further, This is intended to function as a limiting means (e.g., game progression unit 113) that limits the number of times the variation of the specific parameter can be performed based on the captured image. The program described in one of the appendices 1-4. With this configuration, it is possible to prevent the fluctuation of specific parameters from becoming too easy, even though the fluctuation of specific parameters based on the captured image can be made without limit.

[0122] (Note 8) The aforementioned restricting means is Limit the number of times within a predetermined period the variation of the specific parameter based on the captured image containing the same object can be performed. The program described in Appendix 7. This configuration makes it possible to prevent specific parameters from fluctuating rapidly in a short period of time.

[0123] (Note 9) The aforementioned restricting means is Based on the fact that the variation of the specific parameter has been performed based on the inclusion of a specific object in the captured image acquired at the first position, the variation of the specific parameter based on the inclusion of a specific object in the captured image acquired at the first position is prohibited. The control means is In a state where the variation of the specific parameter based on the inclusion of the specific object in the captured image acquired at the first position is prohibited, the variation of the specific parameter based on the inclusion of the specific object in the captured image acquired at the second position may be performed. The program described in Appendix 7. This configuration prevents specific parameters from being easily altered by repeatedly photographing the same object in the same location, while also broadening the range of user enjoyment.

[0124] (Note 10) The processor, further, The appearance of the virtual object is to be changed to reflect the characteristics of a specific object included in the captured image that is displayed simultaneously with the virtual object (for example, the game progression unit 113). The program described in one of the appendices 1-4. This configuration makes it possible to reflect the characteristics of real-world objects in virtual objects, thereby strengthening the connection between the virtual space and the real world.

[0125] (Note 11) A display control means that simultaneously displays captured images of the real world and virtual objects, The system includes control means for varying specific parameters based on the captured image displayed simultaneously with the virtual object, The aforementioned specific parameter is at least one of the parameters of the virtual object and a parameter indicating a predetermined amount of value owned by the user who owns the virtual object. Information processing system. With this configuration, the same effects and benefits as the program described in Appendix 1 can be achieved. [Explanation of symbols]

[0126] 1 Information processing system, 10 Terminal device, 11 Processor, 12 Memory, 13 Storage, 17 Input unit, 18 Display unit, 20 Server, 21 Processor, 22 Memory, 23 Storage, 110 Control unit, 111 Operation reception unit, 112 Transmit / receive unit, 113 Game progress unit, 114 Display control unit, 115 Detection unit, 120 Memory unit, 210 Control unit, 211 Transmit / receive unit, 212 Server processing unit, 213 Data management unit, 214 Synchronization processing unit, 220 Memory unit

Claims

1. Computers, A display control means that simultaneously displays captured images of the real world and virtual objects, The means for changing the appearance of the virtual object itself to reflect the characteristics of a specific object included in the captured image that is displayed simultaneously with the virtual object, and for changing the parameters of the virtual object based on the captured image, The changing means is capable of changing the appearance of the virtual object itself to reflect the characteristics of the first specific object included in the first captured image displayed simultaneously with the virtual object, and changing it to reflect the characteristics of the second specific object included in the second captured image displayed simultaneously with the virtual object. The appearance of the virtual object after being altered by the alteration means is maintained even after the captured image containing the specific object is no longer displayed simultaneously with the virtual object. The aforementioned specific object is a living being other than a person, food, a book, or a landmark. program.

2. A display control means that simultaneously displays captured images of the real world and virtual objects, The system includes a means for changing the appearance of the virtual object itself to reflect the characteristics of a specific object included in the captured image that is displayed simultaneously with the virtual object, and for changing the parameters of the virtual object based on the captured image. The changing means is capable of changing the appearance of the virtual object itself to reflect the characteristics of the first specific object included in the first captured image displayed simultaneously with the virtual object, and changing it to reflect the characteristics of the second specific object included in the second captured image displayed simultaneously with the virtual object. The appearance of the virtual object after being altered by the alteration means is maintained even after the captured image containing the specific object is no longer displayed simultaneously with the virtual object. The aforementioned specific object is a living being other than a person, food, a book, or a landmark. Information processing system.

Images