Game program, recording medium, and game processing method

Abstract

To share real-world experiences with game characters.SOLUTION: A game program causes an information processing device 3 including a display unit 13 which can be worn on a player's head and displays a virtual image superimposed on a real space image and a line-of-sight direction detection unit 15 for detecting a player's line-of-sight direction and configured to transmit and receive signals to function as: a line-of-sight direction setting processing unit 127 for setting the line-of-sight direction of a virtual game character having eyes based on the detected player's line-of-sight direction; and a second character display processing unit 129 for displaying the image of the game character on the display unit 13 so that the game character changes the line-of-sight direction based on the set line-of-sight direction of the game character.SELECTED DRAWING: Figure 23

Description

[Technical Field]

[0001] The present invention relates to a game program, a recording medium, and a game processing method. [Background technology]

[0002] Conventionally, there are known game systems that perform game processing that reflects the relationship between a player's line of sight and a character. For example, Patent Document 1 describes a game system that calculates line of sight relationship information that represents the relationship between the player's line of sight and a character that is the target of the player's game play, based on the line of sight information, and performs game processing by changing game parameters based on the line of sight relationship information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-231443 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, the player and the character communicate in a virtual three-dimensional space, but it is not possible to share real-world experiences with the character.

[0005] The present invention has been made in consideration of such problems, and aims to provide a game program, a recording medium, and a game processing method that enable a player to share experiences in the real world with a game character. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the game program of the present invention causes a display unit that is configured to be wearable on a player's head and that displays a virtual image superimposed on an image of real space, a gaze direction detection unit that detects the direction of the player's gaze, and an information processing device configured to send and receive signals to function as a gaze direction setting processing unit that sets the gaze direction of a virtual game character having eyes based on the detected gaze direction of the player, and a character display processing unit that displays an image of the game character on the display unit so that the game character changes its gaze direction based on the set gaze direction of the game character.

[0007] In order to achieve the above object, a recording medium of the present invention is a recording medium that stores the above game program and is readable by an information processing device.

[0008] In order to achieve the above-mentioned object, the game processing method of the present invention is a game processing method executed by a display unit configured to be wearable on a player's head and that displays a virtual image superimposed on an image of real space, a gaze direction detection unit that detects the direction of the player's gaze, and an information processing device configured to send and receive signals, and includes the steps of: setting the gaze direction of a virtual game character having eyes based on the detected gaze direction of the player; and displaying an image of the game character on the display unit so that the game character changes its gaze direction based on the set gaze direction of the game character. [Effects of the Invention]

[0009] According to the game program etc. of the present invention, it is possible to share experiences in the real world with game characters. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a system configuration diagram showing an example of the overall configuration of a game system common to all embodiments. [Figure 2]FIG. 1 is a block diagram illustrating an example of a schematic configuration of a head-mounted display. [Figure 3] FIG. 10 is a diagram showing a specific example of a manual operation by a player. [Figure 4] 10 is a flowchart illustrating an example of the flow of a game executed by a control unit of a head-mounted display and an information processing device. [Figure 5] FIG. 10 is a diagram illustrating an example of a character selection screen. [Figure 6] FIG. 10 is a diagram illustrating an example of a screen for adjusting the position of a virtual object. [Figure 7] FIG. 10 is a diagram illustrating an example of a virtual object orientation adjustment screen. [Figure 8] FIG. 10 is a diagram illustrating an example of a display screen of a game character. [Figure 9] FIG. 10 is a diagram showing an example of a game screen when a game character is touched. [Figure 10] FIG. 10 is a diagram illustrating an example of a game screen in a fan mode. [Figure 11] FIG. 10 is a diagram illustrating an example of a game screen in a spray mode. [Figure 12] 1 is a block diagram illustrating an example of a functional configuration of an information processing device according to a first embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual object of a chair with a backrest. [Figure 14] FIG. 10 is a diagram illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a balance ball virtual object. [Figure 15] 10A and 10B are diagrams illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual object of a seat cushion. [Figure 16] 10A and 10B are diagrams illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual pillow object. [Figure 17] FIG. 10 is a diagram illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual object, which is a glass with a straw. [Figure 18] FIG. 10 is a diagram illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual object on a table. [Figure 19] FIG. 10 is a diagram illustrating an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual wall object. [Figure 20] 4 is a flowchart illustrating an example of a processing procedure executed by a CPU of the information processing device according to the first embodiment. [Figure 21] FIG. 10 is a block diagram illustrating an example of a functional configuration of an information processing device according to a modified example of the first embodiment. [Figure 22] FIG. 10 is a diagram illustrating an example of a real space in which a virtual object is registered. [Figure 23] FIG. 10 is a block diagram illustrating an example of a functional configuration of an information processing device according to a second embodiment. [Figure 24] FIG. 10 is an explanatory diagram showing an example of a case where the direction of the game character's line of sight is changed to follow the direction of the player's line of sight when the player and game character face each other. [Figure 25] FIG. 10 is an explanatory diagram showing an example of changing the line of sight of the game character to follow the line of sight of the player when the player and game character are side by side. [Figure 26] 10 is a flowchart illustrating an example of a processing procedure executed by a CPU of an information processing device according to a second embodiment when setting and changing the line of sight of a game character based on the line of sight of a player. [Figure 27] FIG. 10 is a block diagram illustrating an example of a functional configuration of an information processing device according to a modified example of the second embodiment. [Figure 28] FIG. 10 is an explanatory diagram showing an example of changes in the direction of the line of sight of the game character and the player when a time difference is set in a case where the player and game character are side by side. [Figure 29]FIG. 10 is an explanatory diagram showing an example of the change in the direction of the head and eyes of a game character and the direction of the player's gaze when the game character's gaze direction is set based on the direction of the player's head when the player and game character are side-by-side. [Figure 30] FIG. 10 is a block diagram illustrating an example of the configuration of a head-mounted display and the functional configuration of an information processing device according to a third embodiment. [Figure 31] FIG. 11 is a diagram illustrating an example of a game screen in the third embodiment. [Figure 32] FIG. 11 is a diagram illustrating another example of a game screen in the third embodiment. [Figure 33] A flowchart showing an example of a processing procedure executed by the CPU of an information processing device according to the third embodiment when changing the deformation pattern and behavior of a flexible object associated with a game character according to the detected wind conditions. [Figure 34] FIG. 10 is a block diagram illustrating an example of the configuration of a head-mounted display and the functional configuration of an information processing device according to a fourth embodiment. [Figure 35] FIG. 13 is a diagram illustrating an example of a game screen in the fourth embodiment. [Figure 36] FIG. 13 is a diagram illustrating another example of a game screen in the fourth embodiment. [Figure 37] A flowchart showing an example of a processing procedure executed by the CPU of an information processing device related to the fourth embodiment when changing the wetness or sweating of a game character's skin or an object associated with the game character in accordance with detected humidity and temperature. [Figure 38] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, first to fourth embodiments of the present invention will be described with reference to the drawings.

[0012] <1. Overall structure of the game system> First, an example of the overall configuration of a game system 1 common to all embodiments will be described using Fig. 1. As shown in Fig. 1, the game system 1 has an information processing device 3, a game controller 5, and a head-mounted display 7. Each of the game controller 5 and the head-mounted display 7 is connected to the information processing device 3 via wire or wirelessly so as to be communicable (capable of transmitting and receiving signals).

[0013] The information processing device 3 is, for example, a stationary game console. However, it is not limited to this and may be, for example, a portable game console that is integrated with an input unit, a display unit, etc. In addition to game consoles, it may also be, for example, a computer manufactured and sold, such as a server computer, a desktop computer, a notebook computer, a tablet computer, etc., or a telephone manufactured and sold, such as a smartphone, a mobile phone, a phablet, etc.

[0014] The information processing device 3 may be mounted on the head-mounted display 7, so that the information processing device 3 and the head-mounted display 7 are integrated. Furthermore, if the head-mounted display 7 has the same functions as the information processing device 3, the information processing device 3 may be omitted.

[0015] A player performs various operation inputs using a game controller 5. In the example shown in Fig. 1, the game controller 5 has, for example, a cross key 9, a plurality of buttons 10, a joystick 11, a touchpad 12, and the like.

[0016] The head-mounted display 7 is a display device that can be worn on the user's head or face and realizes so-called MR (Mixed Reality). The head-mounted display 7 has a transparent display unit 13, and displays a virtual image related to a game generated by the information processing device 3, superimposed on an image in real space.

[0017] <2. Overview of the head-mounted display> Next, an example of the schematic configuration of the head mounted display 7 will be described with reference to Figures 2 and 3. As shown in Figure 2, the head mounted display 7 has a display unit 13, a gaze direction detection unit 15, a position detection unit 17, an audio input unit 19, an audio output unit 21, a manual operation input unit 23, an information acquisition unit 25, and a control unit 27.

[0018] The display unit 13 is configured with, for example, a transmissive (see-through) liquid crystal display or organic EL display, and displays a virtual image related to the game generated by the information processing device 3, for example, as a holographic video, superimposed on an image of real space that can be seen through the display unit 13. The virtual image may be either a two-dimensional image or a three-dimensional image, and may be either a still image or a moving image. Note that the display unit 13 may be non-transmissive, and may display a virtual image generated by the information processing device 3 superimposed on an image of real space captured by a camera, for example.

[0019] The gaze direction detection unit 15 detects the direction of the player's gaze. In this specification, "the player's gaze direction" refers to a direction that combines the direction of the player's eyes and head. For example, if the player turns his head to the right from the reference direction and then turns his eyes to the right, the gaze direction is the sum of the angle of the head turned to the right and the angle of the eyes turned to the right from the reference direction. The gaze direction detection unit 15 has an eye direction detection unit 29 and a head direction detection unit 31.

[0020] The eye direction detection unit 29 detects the direction of the player's eyes. The eye direction is detected, for example, as a relative direction (angle) with respect to the forward direction of the head. The method for detecting the eye direction is not particularly limited, and various detection methods can be adopted. For example, the eye direction detection unit 29 may be configured with an infrared LED and an infrared camera. In this case, an image of the player's eyes illuminated by the infrared LED is captured with an infrared camera. Based on the captured image, the control unit 27 may calculate the direction of the player's eyes based on the position of the pupil relative to the position of the corneal reflex, with the reference point being the position on the cornea of ​​the reflected light (corneal reflex) generated by the illumination of the infrared LED and the moving point being the pupil. Alternatively, the eye direction detection unit 29 may be configured with a visible light camera. Based on an image of the player's eyes captured with the visible light camera, the control unit 27 may calculate the direction of the eye based on the position of the iris relative to the inner corner of the eye, with the reference point being the inner corner of the eye and the moving point being the iris (so-called black eye).

[0021] The head direction detection unit 31 detects the direction of the player's head (direction of the face). The direction of the head is detected as a direction (vector) in a stationary coordinate system in real space recognized by, for example, a space recognition process described below. The method for detecting the head direction is not particularly limited, and various detection methods can be adopted. For example, the head-mounted display 7 may be provided with an acceleration sensor, a gyro sensor, etc., and the control unit 27 may calculate the direction of the player's head based on the detection results of these sensors.

[0022] The position detection unit 17 detects the position of the player's head. The method for detecting the head position is not particularly limited, and various detection methods can be adopted. For example, a method may be used in which cameras and depth sensors are provided at multiple locations around the head-mounted display 7, the depth sensors are used to recognize the space around the player (real space), and the control unit 27 recognizes the position of the player's head in the surrounding space based on the detection results from the multiple cameras. Another example may be a method in which a camera is installed outside the head-mounted display 7 and a marker such as a light-emitting unit is attached to the head-mounted display 7, and the position of the player's head is detected by the external camera.

[0023] The voice input unit 19 is composed of, for example, a microphone, and inputs the voice of the player and other external sounds. The input voice of the player is recognized as words by, for example, voice recognition processing by the control unit 27, and processing is performed based on the recognized voice. Also, the input external sound is recognized by, for example, voice recognition processing by the control unit 27, and processing is performed based on the recognized type of sound.

[0024] The audio output unit 21 is configured with, for example, a speaker, and outputs audio to the player's ears, such as the voice of a character, sound effects, background music, etc.

[0025] The hand operation input unit 23 is composed of, for example, a camera, which detects the player's hand movements and inputs them as hand operations. The control unit 27 executes processing based on the input hand operations. This allows the player to perform various operations with hand movements. Specifically, as shown in FIG. 3, for example, a "tap" operation in which the tip of the thumb and the tip of the index finger of the right hand touch once, a "double tap" operation in which they touch twice, and a "rotation" operation in which the index finger of the right hand is extended upward and rotated are possible. In addition to these, a wide variety of other hand operations may also be possible.

[0026] The information acquisition unit 25 connects to the Internet, for example, via wireless communication, and acquires various information from information distribution sites that distribute information, databases, etc. For example, it is possible to acquire information related to date and time, weather, news, entertainment (concert and movie schedules, etc.), shopping (bargain sales, etc.), etc. The information acquisition unit 25 may also acquire information about the player's current location based on signals from GPS satellites, for example.

[0027] The control unit 27 executes various processes based on the detection signals of the various sensors. The various processes include, for example, image display process, line-of-sight direction detection process, line-of-sight adjustment process, position detection process, space recognition process, voice recognition process, voice output process, manual input process, information acquisition process, etc. In addition, a wide variety of other processes may be executed.

[0028] The information processing device 3 generates or changes a virtual image to be displayed on the display unit 13 based on the processing results of the gaze direction detection unit 15, position detection unit 17, voice input unit 19, manual operation input unit 23, information acquisition unit 25, etc. of the head-mounted display 7, thereby expressing MR (mixed reality).

[0029] <3. Game Overview> Next, an example of the outline of the game according to this embodiment, that is, the game provided by executing the game program and game processing method of the present invention by the information processing device 3, will be described with reference to FIGS.

[0030] The game according to this embodiment allows the player to communicate with a game character that appears to exist in real space by overlaying an image of the virtual game character on an image of real space. The position, posture, movement, behavior, etc. of the game character change in response to various operational inputs by the player (eye movement and head movement, hand operation, voice, operational input via the game controller 5, etc.). The game character may be any virtual character or object with eyes, and its type is not particularly limited. For example, it may be a human male character, a female character, a non-human animal character, a virtual living creature character other than a human or animal, a non-living robot, etc.

[0031] FIG. 4 shows an example of the flow of a game executed by the control unit 27 of the head-mounted display 7 and the information processing device 3.

[0032] In step S5, the control unit 27 determines whether or not it is the first time the player has worn the head mounted display 7. For example, a message confirming whether or not it is the first time the player has worn the head mounted display 7 may be displayed on the display unit 13, and the determination may be made based on an input (YES or NO) by the player. If it is the first time (step S5: YES), the process proceeds to the next step S10. On the other hand, if it is not the first time (step S5: NO), the process proceeds to step S15, which will be described later.

[0033] In step S10, control unit 27 executes a line-of-sight adjustment process. The line-of-sight adjustment method is not particularly limited. For example, markers may be displayed at five locations near the center and four corners of the player's field of view on display unit 13, and line-of-sight information may be detected while the player is gazing at the markers.

[0034] In step S15, the control unit 27 displays a home menu screen on the display unit 13.

[0035] In step S20, the control unit 27 determines whether or not to start the game. For example, this determination may be made based on whether or not the game has been started by the player on the home menu screen. The control unit 27 waits in this step S20 until the game is started (step S20: NO), and if the game is to be started (step S20: YES), the control unit 27 proceeds to the next step S25.

[0036] In step S25, the control unit 27 executes a space recognition process to recognize the real space around the player using, for example, a depth sensor or a camera, thereby generating a stationary coordinate system (a coordinate system corresponding to the real space) that does not change even if the position of the head-mounted display 7 changes as the player moves around.

[0037] In step S30, the information processing device 3 displays a virtual object on the display unit 13. The virtual object is displayed, for example, in the center of the field of view of the display unit 13, and in the depth direction, at a position near the collision point between the direction in which the display unit 13 is facing (the direction of the player's head) and the real space recognized in step S25. When the player changes the direction of his or her head, the display position of the virtual object in the field of view does not change from the center, and only the position in the depth direction changes according to the real space. Note that the position in the depth direction may also be fixed and displayed at a preset position. Also, the player may be able to adjust the position in the up / down / left / right directions or the position in the depth direction in the field of view. The type of virtual object is not particularly limited. For example, it may be an object with a simple shape such as a cube, a rectangular parallelepiped, or a sphere, or it may be a virtual object imitating various real-world objects such as a chair or a table. Also, the game character itself may be displayed.

[0038] In step S35, the information processing device 3 determines whether or not a game character has been selected by the player. The information processing device 3 waits in step S35 until a game character is selected (step S35: NO), and if a game character has been selected (step S35: YES), the information processing device 3 proceeds to the next step S40.

[0039] FIG. 5 shows an example of a character selection screen. In the example shown in FIG. 5, rectangular parallelepiped virtual objects 33A to 33C are displayed superimposed on an image 35 of the player's room, which is real space. In this example, the virtual objects 33A to 33C are displayed, for example, near a sofa. The virtual objects 33A, 33B, and 33C correspond to game characters A, B, and C, respectively, and each object has a different color. The player can switch between the virtual objects 33A, 33B, and 33C in order as shown in FIG. 5 by, for example, performing the tap operation described above or a switching operation using the game controller 5. Note that the virtual objects may also be switched by calling out the character names. The player can select a game character by, for example, performing the double-tap operation described above or a confirmation operation using the game controller 5 when a virtual object corresponding to a desired game character is displayed.

[0040] 4, in step S40, the information processing device 3 determines whether or not the position and orientation of the virtual object have been determined by the player. The information processing device 3 waits in step S40 until the position and orientation of the virtual object have been determined (step S40: NO), and if the position and orientation of the virtual object have been determined (step S40: YES), the information processing device 3 proceeds to the next step S45.

[0041] Fig. 6 shows an example of a virtual object position adjustment screen. As shown in Fig. 6, the player can adjust the position of virtual object 33A, which is displayed in the center of the field of view of display unit 13, in the vertical and horizontal directions relative to real space by, for example, changing the direction of his or her head. Since the display position of virtual object 33A corresponds to the display position of game character A, the player adjusts the position of virtual object 33A so that game character A is placed in a desired position.

[0042] An example of a virtual object orientation adjustment screen is shown in Fig. 7. As shown in Fig. 7, the player can adjust the orientation of virtual object 33A by rotating it around the vertical axis, for example, by performing the rotation operation using the hand movement described above or by performing a rotation operation using game controller 5. Since the orientation of virtual object 33A corresponds to the orientation of game character A, the player adjusts the orientation of virtual object 33A so that game character A faces a desired direction.

[0043] After completing the adjustment of the position and orientation of the virtual object relative to the real space, the player can determine the position and orientation of the virtual object by, for example, performing the aforementioned double tap operation or a determination operation using the game controller 5.

[0044] 4, in step S45, the information processing device 3 displays a game character so that the game character is positioned in accordance with the position and orientation of the virtual object. An example of a game character display screen is shown in Fig. 8. In this example, a female game character 37, game character A, is displayed standing in front of a sofa, for example.

[0045] In step S50, the information processing device 3 executes a communication process that enables communication between the player and the game character. The player can communicate with the game character in various ways. For example, the player can move around the displayed game character by moving around in real space, looking at it from various directions, and talking to it.

[0046] The player can also touch, for example, a game character. FIG. 9 shows an example of a game screen when touching a game character. As shown in FIG. 9, a targeting line 41 is projected from the player's left hand 39. When the targeting line 41 is projected onto a touchable portion of the game character 37, the color of the targeting line 41 changes, allowing the player to identify the touchable portion. The player can touch the game character 37 by, for example, fisting their left hand when the color has changed. At this time, the game character 37 may take an action corresponding to being touched, such as expressing joy, blushing in embarrassment, becoming upset, or saying a predetermined line. In FIG. 9, the targeting line 41 is projected onto, for example, the right hand of the game character 37. In this state, the player can touch the right hand of the game character 37 (fist their right hand) by fisting their left hand. Note that an image of real space is omitted from FIG. 9 (the same applies to FIGS. 10 and 11, which will be described later).

[0047] Although not shown, the player can stroke the head of the game character 37 by, for example, tilting the left hand 39 so that it is approximately parallel to the floor and moving it left and right while shining the aiming line 41 on the head of the game character 37. At this time, the game character 37 may perform a behavior in response to being stroked on the head, such as showing joy, acting cute, or saying something like "ehehe."

[0048] Furthermore, if the player waves their right hand towards the game character 37, the game character 37 may wave back while saying, for example, "Bye-bye." If the player makes a peace sign with their right hand towards the game character 37, the game character 37 may return the peace sign while saying, for example, a predetermined phrase such as "Peace!"

[0049] Furthermore, if the player utters, for example, "fan" with their right hand held out in front, the right hand switches to fan mode. FIG. 10 shows an example of a game screen in fan mode. As shown in FIG. 10, when the player waves (fans) their right hand 43 toward the game character 37 while in fan mode, a virtual wind is generated in a direction corresponding to the direction of the fan with the right hand 43, causing flexible objects attached to the game character 37, such as hair and clothing, to flutter in the wind. In FIG. 10, for example, the hair 45 of the game character 37 is fluttering in a direction corresponding to the virtual wind. At this time, the game character 37 may take an action corresponding to the wind, such as bracing themselves or yelling "Whoa!" An image of a fan may be displayed in addition to or instead of the right hand 43.

[0050] Furthermore, if the player utters, for example, "spray bottle" while holding out their right hand, the right hand switches to spray bottle mode. FIG. 11 shows an example of a game screen in spray bottle mode. As shown in FIG. 11, when the player is in spray bottle mode, for example, if the player bends the second joint of the index finger 47 of their right hand while extending it upward, spray 49 from the spray bottle is fired toward the game character 37. Note that the spray 49 may or may not be displayed. As a result, the skin of the game character 37 and associated objects (e.g., hair, clothing, etc.) become wet. Because spray 49 is fired each time the index finger 47 is bent, repeated spraying gradually progresses the wet state until the game character 37 becomes completely soaked. In FIG. 11, for example, the face 51 and clothing 53 of the game character 37 are wet. At this time, the game character 37 may take an action corresponding to being wet, such as getting defensive, yelling "Eek!", or getting angry.

[0051] The player can also call out to the game character 37. For example, if the player calls the game character 37 by name, the game character 37 may respond. For example, if the player makes a compliment such as "cute" or "pretty," the game character 37 may become happy or blush with embarrassment. For example, if the player says "Say cheese," the game character 37 may strike a pose for a photo, such as making a peace sign or smiling. In addition, in response to various calls from the player, the game character 37 may show various expressions and gestures, such as laughing, looking troubled, or angry, and may speak lines.

[0052] Returning to Fig. 4, in step S55, it is determined whether or not to end the game. If the game is not to be ended (step S55: NO), the process returns to step S50. On the other hand, if the game is to be ended (step S55: YES), the process ends this flowchart.

[0053] Although the above description has been given of the case where the game character 37 is displayed in a standing position, it is also possible to display the game character 37 in a position that matches the position and orientation of a real object, such as in a state where the game character 37 is sitting in an actual chair. Next, this process will be described in detail.

[0054] 4. First Embodiment In the first embodiment, details of the processing when a game character is arranged to match the position and orientation of a real object will be described. The processing content of this embodiment is executed by switching modes in the communication processing in step S50 described above, for example, when the player issues a call such as "let's sit down." Note that, although not described below, in this embodiment, after the game character is arranged to match the position and orientation of a real object, it is possible to engage in various types of communication with the game character described in step S50 described above.

[0055] (4-1. Functional configuration of information processing device) First, an example of the functional configuration of the information processing device 3 according to the first embodiment will be described with reference to Fig. 12. Note that although the configuration of the head-mounted display 7 is not shown in Fig. 12, it has the same configuration as that shown in Fig. 2 described above.

[0056] As shown in Figure 12, the information processing device 3 has an object selection processing unit 55, an object display processing unit 57, an object adjustment processing unit 59, a character posture setting processing unit 61, a first character display processing unit 63, and an object non-display processing unit 65.

[0057] The object selection processing unit 55 selects the type of virtual object based on the player's input. Virtual objects that mimic various real objects existing in real space may be prepared in advance, and the player may be allowed to select a desired virtual object from among them. The virtual objects may include, for example, virtual seating objects such as chairs, sofas, balls, and cushions, virtual furniture objects such as tables and dressers, virtual tableware objects such as glasses, cups, and plates, and virtual wall objects on which the game character can lean. For example, if the player wants to have the game character sit on a real seating object (an example of a seating object) such as a chair, sofa, ball, or cushion in real space, the player may select a virtual seating object.

[0058] The object display processing unit 57 displays an image of the virtual object selected by the object selection processing unit 55 on the display unit 13 by superimposing it on the image of the real space.

[0059] The object adjustment processing unit 59 adjusts at least one of the position and orientation of the virtual object relative to the image of real space based on the player's input. For example, if the player wants to have a game character sit on a real seating object, the object adjustment processing unit 59 adjusts at least one of the position and orientation of the virtual seating object relative to the real seating object included in the image of real space based on the player's input. As described above, the player can adjust the position of the virtual object displayed in the center of the field of view of the display unit 13 in the up, down, left, or right directions relative to real space by, for example, changing the direction of his or her head. The depth position of the virtual object is a position near the collision point between the direction of the player's head and real space, as described above. Note that the depth position along the line of sight may be adjusted by, for example, a hand operation or an operation using the game controller 5. The player can also adjust the orientation of the virtual object by, for example, the rotation operation using the hand movement described above or a rotation operation using the game controller 5. The player may adjust both the position and orientation of the virtual object, or only one of the position and orientation.

[0060] The character posture setting processing unit 61 sets the posture of the game character based on the type of virtual object selected by the object selection processing unit 55. For example, if the selected object is a virtual seating object such as a chair, sofa, ball, or cushion, the posture is set to a posture of sitting on the object. For example, if the selected object is a virtual table object, the posture is set to a posture of sitting facing the player with the virtual table object between them. For example, if the selected object is a virtual tableware object such as a glass, cup, or plate, the posture is set to a posture of placing a hand on the tableware object. For example, if the selected object is a virtual wall object, the posture is set to a posture of leaning against the virtual wall object.

[0061] The first character display processing unit 63 (an example of a character display processing unit) displays an image of a virtual game character on the display unit 13 so that the image is positioned to match the position or orientation of a virtual object, at least one of whose position or orientation has been adjusted by the object adjustment processing unit 59. For example, when at least one of the position or orientation of a virtual seating object is adjusted with respect to a real seating object, the first character display processing unit 63 displays an image of the game character on the display unit 13 so that the image is positioned to match the position or orientation of the real seating object. The first character display processing unit 63 also displays an image of the game character in the posture set by the character posture setting processing unit 61 on the display unit 13.

[0062] The object non-display processing unit 65 hides the image of the virtual object when the first character display processing unit 63 displays the image of the game character.

[0063] Note that the processing in each processing unit described above is not limited to these examples of division of processing, and may be performed by, for example, a smaller number of processing units (e.g., one processing unit), or by further subdivided processing units. The functions of each processing unit described above are implemented by a game program executed by a CPU 301 (see FIG. 38 described below), but some of them may be implemented by actual devices such as dedicated integrated circuits (ASICs, FPGAs, etc.) or other electrical circuits. Furthermore, the processing units described above are not limited to being implemented entirely on the information processing device 3 side, and some or all of them may be implemented on the head-mounted display 7 side (e.g., the control unit 27). In this case, the control unit 27 of the head-mounted display 7 is an example of an information processing device.

[0064] (4-2. Example of virtual object adjustment and character display screen) Next, an example of a screen on which a game character is displayed by adjusting the position and orientation of a virtual object will be described with reference to FIGS.

[0065] In the example shown in FIG. 13, a virtual seating object 67, for example, a chair with a backrest, is selected by the player. As shown in the upper part of FIG. 13, the orientation of the virtual seating object 67 is adjusted to match the orientation of a real seating object 69, which is a chair with a backrest that exists in real space. Next, as shown in the middle part of FIG. 13, the position of the virtual seating object 67 is adjusted to match the position of the real seating object 69. At this time, for example, the heights of the seats of the virtual seating object 67 and the real seating object 69 may be adjusted to match. The orientation and position adjustments may be performed in the reverse order to the above, or may be performed simultaneously (similarly to FIGS. 14 to 19). Then, as shown in the lower part of FIG. 13, a game character 71 set in a seated posture is positioned to match the position and orientation of the adjusted virtual seating object 67, and the virtual seating object 67 is hidden. This allows the game character 71 to be displayed as if seated on the real seating object 69.

[0066] In the example shown in FIG. 14 , a virtual seating object 73, such as a balance ball, is selected by the player. The orientation of the virtual seating object 73 is also displayed. As shown in the upper part of FIG. 14 , the orientation of the virtual seating object 73 is adjusted to match the orientation of a real seating object 75, which is a balance ball existing in real space. In this example, since the real seating object 75 is ball-shaped and has no orientation, the player may adjust the direction in which the game character faces. Next, as shown in the middle part of FIG. 14 , the position of the virtual seating object 73 is adjusted to match the position of the real seating object 75. At this time, for example, the heights of the seat surfaces (top surfaces) of the virtual seating object 73 and the real seating object 75 may be adjusted to match. Thereafter, as shown in the lower part of FIG. 14 , a game character 77 set in a seated posture is positioned to match the position and orientation of the adjusted virtual seating object 73, and the virtual seating object 73 is hidden. This allows the game character 77 to appear as if it is sitting on the real seating object 75.

[0067] In the example shown in FIG. 15 , a virtual seating object 79, such as a seat cushion, is selected by the player. As shown in the upper part of FIG. 15 , the orientation of the virtual seating object 79 is adjusted to match the orientation of a real seating object 81, which is a seat cushion existing in real space. Next, as shown in the middle part of FIG. 15 , the position of the virtual seating object 79 is adjusted to match the position of the real seating object 81. At this time, for example, the heights of the seats of the virtual seating object 79 and the real seating object 81 may be adjusted to match. Thereafter, as shown in the lower part of FIG. 15 , a game character 83 set in a seated posture is positioned to match the position and orientation of the adjusted virtual seating object 79, and the virtual seating object 79 is hidden. This allows the game character 83 to be displayed as if it is sitting on the real seating object 81.

[0068] Although not shown in the figures, it is also possible to, for example, have a square cushion that can be sat on in four directions, and have a game character sit facing one direction in the same way as above, with a player sitting next to the character facing the other direction.

[0069] In the example shown in FIG. 16 , a virtual object 85, for example, a body pillow, is selected by the player. As shown in the upper part of FIG. 16 , the orientation of the virtual object 85 is adjusted to match the orientation of a real object 87, which is a body pillow that exists in real space. At this time, for example, the longitudinal directions (axial directions) of the virtual object 85 and the real object 87 may be adjusted to match. Next, as shown in the middle part of FIG. 16 , the position of the virtual object 85 is adjusted to match the position of the real object 87. At this time, for example, the heights or central positions of the top surfaces of the virtual object 85 and the real object 87 may be adjusted to match. Thereafter, as shown in the lower part of FIG. 16 , a game character 89 set in a pose of lying down and hugging a pillow is positioned to match the adjusted position and orientation of the virtual object 85, and the virtual object 85 is hidden. This makes it possible to display the game character 89 as if it were hugging the real object 87.

[0070] In the example shown in FIG. 17 , a player selects a virtual object 91, which is, for example, a glass with a straw. As shown in the upper part of FIG. 17 , the orientation of the virtual object 91 is adjusted to match the orientation of a real object 93, which is a glass with a straw that exists in real space. At this time, for example, the orientations of the straws of the virtual object 91 and the real object 93 may be adjusted to match. Next, as shown in the middle part of FIG. 17 , the position of the virtual object 91 is adjusted to match the position of the real object 93. At this time, for example, the heights of the straws of the virtual object 91 and the real object 93 may be adjusted to match. Thereafter, as shown in the lower part of FIG. 17 , a game character 95 is positioned with its hand on the glass and the straw in its mouth so as to match the adjusted position and orientation of the virtual object 91, and the virtual object 91 is hidden. This makes it possible to display the game character 95 as if it is holding the real object 93 in its hand and drinking a drink.

[0071] In the example shown in FIG. 18 , a virtual object 97, for example, a table, is selected by the player. As shown in the upper part of FIG. 18 , the orientation of the virtual object 97 is adjusted to match the orientation of a real object 99, which is a table existing in real space. At this time, for example, the longitudinal directions of the virtual object 97 and the real object 99 may be adjusted to match. Next, as shown in the middle part of FIG. 18 , the position of the virtual object 97 is adjusted to match the position of the real object 99. At this time, for example, the height of the top surfaces of the virtual object 97 and the real object 99 may be adjusted to match. Thereafter, as shown in the lower part of FIG. 18 , a game character 101 is positioned facing the player across the table, with its hands or elbows on the table, to match the adjusted position and orientation of the virtual object 97, and the virtual object 97 is hidden. This makes it possible to display the game character 101 as if it were having a meal with the player. Note that by making similar adjustments to glasses, plates, etc. on the table in addition to the table, it is possible to display the game character 101 as if he or she is holding glasses, plates, etc. and eating or drinking at the table. This allows the player to feel like they're on a date.

[0072] In the example shown in FIG. 19 , a player selects virtual object 103, which is, for example, a wall. As shown in the upper part of FIG. 19 , the orientation of virtual object 103 is adjusted to match the orientation of real object 105, which is a wall existing in real space. At this time, for example, adjustment may be made so that virtual object 103 and real object 105 are parallel. Next, as shown in the middle part of FIG. 19 , the position of virtual object 103 is adjusted to match the position of real object 105. At this time, for example, adjustment may be made so that the wall surfaces (positions in the depth direction) of virtual object 103 and real object 105 are aligned. Thereafter, as shown in the lower part of FIG. 19 , game character 107, which is set to lean against the wall, is positioned to match the adjusted position and orientation of virtual object 103, and virtual object 103 is hidden. As a result, for example, when a player touches his or her right hand 109 to the wall of real object 105, the player can experience the feeling of performing a so-called kabedon.

[0073] (4-3. Processing procedure executed by the information processing device) Next, an example of a processing procedure executed by the CPU 301 of the information processing device 3 according to the first embodiment will be described with reference to FIG.

[0074] In step S110, the information processing device 3 causes the object selection processing unit 55 to select the type of virtual object based on the input from the player.

[0075] In step S120, the information processing device 3 causes the object display processing unit 57 to display the image of the virtual object selected in step S110 on the display unit 13, superimposed on the image of real space.

[0076] In step S130, the information processing device 3 causes the object adjustment processing unit 59 to adjust, based on the input of the player, at least one of the position and orientation of the virtual object displayed in step S120 above with respect to the image in real space.

[0077] In step S140, the information processing device 3 causes the character posture setting processing unit 61 to set the posture of the game character based on the type of virtual object selected in step S110.

[0078] In step S150, the information processing device 3 causes the first character display processing unit 63 to display on the display unit 13 an image of the game character assuming the posture set in step S140 so that the image is positioned in accordance with the position or orientation of the virtual object adjusted in step S130.

[0079] In step S160, the information processing device 3 hides the image of the virtual object by the object hide processing unit 65. This ends the process of this flowchart.

[0080] The above-described processing procedure is merely an example, and at least some of the above procedures may be deleted or changed, or other procedures may be added. Furthermore, the order of at least some of the above procedures may be changed, or multiple procedures may be combined into a single procedure. For example, step S120, step S130, and step S140 may be executed in the reverse order, or may be executed simultaneously in parallel.

[0081] (4-4. Effects of the First Embodiment) As described above, the game program of this embodiment causes the display unit 13, which is configured to be wearable on the player's head and displays a virtual image superimposed on an image of real space, and the information processing device 3, which is configured to transmit and receive signals, to function as an object display processing unit 57 that displays an image of a virtual object on the display unit 13, an object adjustment processing unit 59 that adjusts at least one of the position and orientation of the virtual object with respect to the image of real space based on input from the player, a first character display processing unit 63 that displays an image of a virtual game character on the display unit 13 so that the image is positioned in accordance with at least one of the position and orientation of the adjusted virtual object, and an object non-display processing unit 65 that hides the image of the virtual object when the image of the game character is displayed.

[0082] In this embodiment, an image of a virtual object is displayed by display unit 13 so as to be superimposed on the image of real space, and at least one of the position and orientation of the virtual object relative to the image of real space is adjusted by a player's input. Then, an image of a virtual game character is displayed by display unit 13 so as to be superimposed on the image of real space so as to be positioned in accordance with the adjusted position or orientation of the virtual object, and the image of the virtual object is hidden.

[0083] This allows the position and orientation of the virtual object to be adjusted to match the position and orientation of an actual object in real space, so that the game character can be displayed so that it is positioned to match the position and orientation of the actual object. This makes it possible for the player to feel as if the game character is present in the real world, allowing the player to feel as if they are sharing an experience in the real world with the game character.

[0084] In addition, in this embodiment, the object adjustment processing unit 59 may adjust at least one of the position or orientation of the virtual seated object relative to the real seated object included in the image of real space based on the player's input, and the first character display processing unit 63 may display an image of the game character on the display unit 13 so that the game character is positioned in accordance with at least one of the position or orientation of the real seated object.

[0085] In this case, the game character can be displayed as if seated on an actual seating object in real space, allowing the player to communicate with the game character seated in a real chair, for example, and giving the player the feeling that the game character is actually present in the real world.

[0086] In this embodiment, the game program may also cause the information processing device 3 to further function as an object selection processing unit 55 that selects the type of virtual object based on an input from the player.

[0087] In this case, the player can select the type of virtual object to match the type of actual object that exists in real space. For example, if a chair exists in real space, the player can select a virtual chair object and adjust the position and orientation of the chair object to match the actual chair that exists in real space, thereby displaying the game character as if it were sitting in an actual chair. This allows the player to feel that the game character exists in the real world more realistically.

[0088] In addition, in this embodiment, the game program may further cause the information processing device 3 to function as a character posture setting processing unit 61 that sets the posture of the game character based on the type of selected virtual object, and in that case, the first character display processing unit 63 may cause the display unit 13 to display an image of the game character in the set posture.

[0089] In this case, the game character can be displayed in a pose corresponding to the type of virtual object selected by the player. For example, if the player selects a glass with a straw, the game character can be displayed in a pose such that it places its hand on the glass and clings to the straw, and is positioned in accordance with the position and orientation of the actual glass with a straw. This allows the player to feel that the game character exists in the real world with a more realistic feel.

[0090] (4-5. Modifications of the First Embodiment) In the first embodiment described above, the adjusted position and orientation of the virtual object may be registered, and the behavior of the game character may be changed based on the registered content.

[0091] An example of the functional configuration of an information processing device 3 according to this modification is shown in Fig. 21. As shown in Fig. 21, the information processing device 3 has an object registration processing unit 111 and a first character action setting processing unit 113 in addition to the configuration shown in Fig. 12 above.

[0092] The object registration processing unit 111 registers at least one of the position and orientation of the virtual object adjusted by the object adjustment processing unit 59 in an appropriate recording medium (for example, the RAM 305, the recording device 317, etc.; see FIG. 38 described later).

[0093] First character action setting processing unit 113 (an example of a character action setting processing unit) sets the action of the game character based on at least one of the position and orientation of the virtual object registered by object registration processing unit 111. "Action" includes the movement, body movement, facial expressions and gestures, speech, etc. of the game character. First character display processing unit 63 displays an image of the game character on display unit 13 so that the game character performs the action set by first character action setting processing unit 113.

[0094] FIG. 22 shows an example of a real space in which virtual objects are registered. FIG. 22 illustrates, for example, a player's room, which is equipped with a sofa 115, a table 117, a chest of drawers 119, a television 121, a calendar 123, a window 125, and the like. The player selects virtual objects corresponding to each real object, adjusts the position and orientation as described above, and registers the type, position, orientation, and the like of each virtual object. This allows the game character to perform actions in accordance with the arrangement of furniture and the like in the player's room. For example, the game character can perform a wide variety of actions in accordance with the registered contents, such as sitting on the sofa 115, sitting in front of the television 121 and looking in the direction of the television, resting its elbows on the table 117, eating and drinking, opening a drawer in front of the chest of drawers 119, looking at the calendar 123 and saying something like, "Our anniversary is coming up soon," or looking outside in front of the window 125 and saying something like, "The weather is nice today."

[0095] According to this modification, the player can adjust and register the position and orientation of the virtual object according to the arrangement of furniture, etc. in the player's room, for example, and thereby make the game character behave in accordance with the arrangement of furniture, etc. in the player's room. This makes it possible for the player to feel as if the game character is actually present in the player's room, and as if the player is sharing a real-life experience with the game character.

[0096] 5. Second Embodiment In the second embodiment, details of the process for setting the line of sight of a game character based on the line of sight of a player will be described. Note that the process content of this embodiment is executed in the communication process in step S50 described above.

[0097] (5-1. Functional configuration of information processing device) First, an example of the functional configuration of the information processing device 3 according to the second embodiment will be described with reference to Fig. 23. Note that although the configuration of the head-mounted display 7 is not shown in Fig. 23, it has the same configuration as that of Fig. 2 described above.

[0098] As shown in FIG. 23, the information processing device 3 has a line of sight direction setting processing unit 127, a second character display processing unit 129, an additional information acquisition processing unit 131, and a second character action setting processing unit 133.

[0099] The gaze direction setting processor 127 sets the gaze direction of a virtual game character having eyes based on the gaze direction of the player detected by the gaze direction detector 15 of the head-mounted display 7. As mentioned above, the "virtual game character having eyes" is not limited to a human, but also includes non-human animal characters, virtual living creature characters other than humans or animals, non-living robots, etc. Also, as mentioned above, the "player's gaze direction" is a direction that includes both the direction of the player's eyes and the direction of the head, and the gaze direction setting processor 127 sets the gaze direction of the game character based on both the detected direction of the player's eyes and the direction of the head. Note that the "game character's gaze direction" refers to a direction that combines the direction of the game character's eyes, head, and body.

[0100] The gaze direction setting processing unit 127 may set the gaze direction of the game character to follow the detected gaze direction of the player, for example. "Following" refers to, for example, when the player and game character are facing each other (for example, when the intersection angle between the player's front direction and the game character's front direction is greater than 90 degrees), the gaze direction of the game character intersects with the gaze direction of the player at a position between the player and the game character (for example, an intermediate position). Also, for example, when the player and game character are side-by-side (for example, when the player's front direction and the game character's front direction are parallel or the intersection angle is 90 degrees or less), the gaze direction of the game character and the gaze direction of the player are approximately parallel. Furthermore, when the detected gaze direction of the player corresponds to the game character, the gaze direction setting processing unit 127 may set the gaze direction of the game character to the direction corresponding to the player.

[0101] Second character display processing unit 129 (an example of a character display processing unit) causes display unit 13 to display an image of the game character superimposed on an image of real space so that the game character changes its direction of gaze based on the direction of gaze of the game character set by gaze direction setting processing unit 127. At this time, second character display processing unit 129 displays the image of the game character so that the game character changes its direction of gaze by changing at least one of the direction of the game character's eyes, head direction, and body direction. Therefore, the game character may change its direction of gaze only by the direction of its eyes, or by the direction of its eyes and head direction, or by the direction of its eyes and body direction, or by the direction of its head and body direction without changing its direction of its eyes, or by all of the direction of its eyes, head direction, and body direction.

[0102] 24 and 25 show an example of changing the gaze direction of a game character to follow the gaze direction of a player. Note that FIGS. 24 and 25 are not game play screens displayed on the display unit 13, but are explanatory diagrams for explaining the relationship between the gaze direction of a game character and the gaze direction of a player (the same applies to FIGS. 28 and 29 described later). FIG. 24 shows an example of a case where a player and a game character face each other. As shown in the upper part of FIG. 24, when the gaze direction 137 of the player 135 faces, for example, to the right as seen from the player, the gaze direction 141 of the game character 139 faces to the right as seen from the player so as to intersect with the gaze direction 137 of the player 135, and follows the player. As shown in the middle part of FIG. 24, when the gaze direction 137 of the player 135 faces, for example, to the left as seen from the player, the gaze direction 141 of the game character 139 faces to the left as seen from the player so as to intersect with the gaze direction 137 of the player 135, and follows the player. As shown in the lower part of FIG. 24, when the line of sight 137 of the player 135 faces the direction of the game character 139, the line of sight 141 of the game character 139 faces the direction of the player 135, and they stare at each other.

[0103] FIG. 25 shows an example in which a player and a game character are side-by-side. As shown in the upper part of FIG. 25, when the line of sight 137 of the player 135 turns, for example, to the right when viewed from the front of the player, the line of sight 141 of the game character 139 changes to become substantially parallel to the line of sight 137 of the player 135 and follows it. As shown in the middle part of FIG. 25, when the line of sight 137 of the player 135 turns, for example, to the left when viewed from the front of the player, the line of sight 141 of the game character 139 changes to become substantially parallel to the line of sight 137 of the player 135 and follows it. As shown in the lower part of FIG. 25, when the line of sight 137 of the player 135 turns toward the game character 139, the line of sight 141 of the game character 139 turns toward the player 135, and the two characters look at each other.

[0104] Although not shown in the drawings, even when the line of sight 137 of the player 135 changes in the direction of elevation or depression, the line of sight 141 of the game character 139 follows so as to intersect with the line of sight 137 of the player 135, as described above. Furthermore, although the above describes a case in which the game character 139 changes the line of sight 141 by, for example, changing the direction of the eyes, the line of sight 141 may also be changed by changing the direction of the head or the direction of the body instead of or in addition to the direction of the eyes. For example, if the change in the line of sight 137 of the player 135 is small, the direction of the game character's eyes may be changed; if the change is moderate, the direction of the head may be changed instead of or in addition to the direction of the eyes; and if the change is large, the direction of the body may be changed instead of or in addition to the direction of the eyes or the direction of the head.

[0105] Returning to FIG. 23, the incidental information acquisition processing unit 131 connects to a network NW such as the Internet via, for example, a communication device 323 (see FIG. 38 described later) and acquires various types of information from information distribution sites that distribute information, databases, etc. The incidental information acquisition processing unit 131 may also acquire information about the player's current location based on signals from GPS satellites, for example. The incidental information acquisition processing unit 131 may also acquire information by receiving information detected by various sensors of the head-mounted display 7 or information acquired by the information acquisition unit 25. For example, it is possible to acquire information related to the date and time, weather, news, entertainment (concert and movie schedules, etc.), shopping (bargain sales, etc.), etc.

[0106] The second character action setting processing unit 133 sets the action of the game character based on the information acquired by the supplementary information acquisition processing unit 131. "Action" includes the movement, body movements, facial expressions and gestures, speech, etc. of the game character. The second character display processing unit 129 displays an image of the game character on the display unit 13 so that the game character performs the action set by the second character action setting processing unit 133.

[0107] The second character behavior setting processing unit 133 allows the game character to perform, for example, the following behavior. For example, a brightness sensor may be provided in the head-mounted display 7, and if the player sees a shooting star while looking up at the night sky, the game character may follow the player's gaze as the player changes his or her gaze toward the shooting star. At this time, the shooting star may be determined based on the detection information of the brightness sensor, and the game character may make a statement appropriate to the situation, such as "The stars are beautiful," or "There's another shooting star." Similarly, when the player is watching fireworks, the game character may follow the player's gaze as the player changes his or her gaze toward the fireworks. At this time, the fireworks may be determined based on the detection information of the brightness sensor, and the game character may make a statement appropriate to the situation, such as "The fireworks are beautiful."

[0108] For example, when the doorbell rings, the game character may follow the player's gaze as he or she changes direction toward the front door. At this time, the game character may determine that the doorbell is ringing based on information detected by the audio input unit 19 of the head-mounted display 7, and make a statement appropriate to the situation, such as "It looks like someone's here."

[0109] Furthermore, by combining multiple pieces of acquired information, more advanced communication between the player and the game character becomes possible. For example, by adding time-of-day information to the information detected by the brightness sensor, it becomes possible to determine whether the light is sunlight or starlight. Furthermore, by adding current location information and line-of-sight information from GPS to the information detected by the brightness sensor and time-of-day information, it becomes possible to determine which constellation the starlight belongs to. This makes it possible for the game character to engage in short stories about constellations.

[0110] Furthermore, for example, by obtaining news distribution information according to current location information, coupon information that can be used at nearby stores can be obtained, and if the player makes a statement such as "I'm hungry," the content of the statement can be determined based on the information detected by the voice input unit 19, and the game character can make a suggestion such as "X-box lunches are cheap."

[0111] Also, if the current location information is a movie theater, for example, it is possible to obtain the theater's screening schedule information to determine the movie currently being shown, and have the game character be scared if it is a horror movie, or laugh if it is a comedy movie, allowing the player to experience the feeling of watching a movie together with the game character.

[0112] Furthermore, for example, when watching the fireworks described above, by acquiring current location information, it becomes possible to determine the location of the fireworks display and to tell a short story related to the fireworks display.

[0113] Note that the processing in each processing unit described above is not limited to these examples of division of processing, and may be performed by, for example, a smaller number of processing units (e.g., one processing unit), or by further subdivided processing units. The functions of each processing unit described above are implemented by a game program executed by a CPU 301 (see FIG. 38 described below), but some of them may be implemented by actual devices such as dedicated integrated circuits (ASICs, FPGAs, etc.) or other electrical circuits. Furthermore, the processing units described above are not limited to being implemented entirely on the information processing device 3 side, and some or all of them may be implemented on the head-mounted display 7 side (e.g., the control unit 27). In this case, the control unit 27 of the head-mounted display 7 is an example of an information processing device.

[0114] (5-2. Processing procedure executed by the information processing device) Next, using Figure 26, we will explain an example of a processing procedure executed by the CPU 301 of the information processing device 3 according to the second embodiment when setting and changing the line of sight of a game character based on the line of sight of a player.

[0115] In step S210, the information processing device 3 acquires information relating to the direction of the player's line of sight detected by the line of sight direction detection unit 15 of the head-mounted display .

[0116] In step S220, the information processing device 3 causes the line of sight direction setting processing unit 127 to set the line of sight direction of the game character based on the line of sight direction of the player.

[0117] In step S230, the information processing device 3 causes the second character display processing unit 129 to display an image of the game character on the display unit 13 so that the game character changes its line of sight based on the line of sight direction of the game character set in step S220. This ends this flowchart.

[0118] The above-described processing procedure is an example, and at least some of the procedures may be deleted or changed, or other procedures may be added. Furthermore, the order of at least some of the procedures may be changed, or multiple procedures may be combined into a single procedure.

[0119] (5-3. Effects of the Second Embodiment) As described above, the game program of this embodiment causes display unit 13, which is configured to be wearable on the player's head and displays a virtual image superimposed on an image of real space, gaze direction detection unit 15, which detects the direction of the player's gaze, and information processing device 3, which is configured to send and receive signals, to function as gaze direction setting processing unit 127, which sets the gaze direction of a virtual game character having eyes based on the detected direction of the player's gaze, and second character display processing unit 129, which displays an image of the game character on display unit 13 so that the game character changes its gaze direction based on the set gaze direction of the game character.

[0120] In this embodiment, the line of sight of the game character is set based on the line of sight of the player, an image of the game character is generated so that the game character changes its line of sight in the set direction, and the image is displayed superimposed on the image of real space by the display unit 13. In this way, when the player changes his line of sight in response to an event in the real world, the line of sight of the game character can be changed to correspond to the change. This allows the player to feel as if he or she is sharing an experience in the real world with the game character.

[0121] In this embodiment, the line of sight direction setting processing unit 127 may set the line of sight direction of the game character so as to follow the detected line of sight direction of the player.

[0122] In this case, when the player changes the direction of their gaze in response to an event in the real world, the direction of the game character's gaze can be changed to follow that direction of their gaze, making the player feel as if they are seeing the same thing in the real world as the game character, enhancing the sense of sharing.

[0123] Furthermore, in this embodiment, when the detected direction of the player's gaze corresponds to the direction of the game character, the gaze direction setting processing unit 127 may set the gaze direction of the game character to the direction corresponding to the player.

[0124] In this case, for example, the direction of the game character's gaze can be changed to follow the direction of the player's gaze, and when the player looks in the direction of the game character, the game character can also look in the direction of the player and the two can gaze at each other, creating the feeling of being lovers or on a date.

[0125] In addition, in this embodiment, the second character display processing unit 129 may display an image of the game character so that the game character changes its direction of gaze by changing at least one of the direction of the game character's eyes, head, and body.

[0126] In this case, for example, if the change in the gaze direction of the set game character is small, the direction of the eyes can be changed, if it is medium, the direction of the head can be changed instead of or in addition to the direction of the eyes, and if it is large, the direction of the body can be changed instead of or in addition to the direction of the eyes or the direction of the head, etc. In this way, by changing the manner of the action that changes the gaze direction of the game character according to the magnitude of the change in the gaze direction of the set game character, a more natural expression can be achieved.

[0127] (5-4. Modification of the second embodiment) In the second embodiment described above, when a game character is caused to follow the direction of the player's line of sight, the face of the game character may be made more visible to the player.

[0128] An example of the functional configuration of an information processing device 3 according to this modification is shown in Figure 27. As shown in Figure 27, the information processing device 3 has a time difference setting processor 143 in addition to the configuration shown in Figure 23 above.

[0129] When the direction of the player's gaze detected by the gaze direction detection unit 15 of the head-mounted display 7 changes from a direction not corresponding to the game character to a direction corresponding to the game character, the time difference setting processing unit 143 sets a predetermined time difference between the change in the player's gaze direction and the change in the game character's gaze direction. The "predetermined time difference" may be set to, for example, one second to several seconds.

[0130] FIG. 28 shows an example of the change in the gaze direction of the game character and the gaze direction of the player when a time difference is set, for example, when the player and game character are standing side by side. As shown in the upper part of FIG. 28, if the gaze direction 137 of the player 135 turns to the right when viewed from the front of the player, the gaze direction 141 of the game character 139 changes to be approximately parallel to the gaze direction 137 of the player 135 and follows it. Next, as shown in the middle part of FIG. 28, if the gaze direction 137 of the player 135 turns toward the game character 139, the gaze direction 141 of the game character 139 does not change for the duration of the time difference set by the time difference setting processing unit 143. Then, after the time difference has passed, the gaze direction 141 of the game character 139 turns toward the player 135, and the two characters look at each other, as shown in the lower part of FIG. 28.

[0131] According to this modification, for example, if the game character's line of sight is changed to follow the player's line of sight, and the player looks in the direction of the game character, the game character will look in the player's direction after a predetermined time lag has passed. This allows the player to watch and confirm for a while that the game character is looking at the same thing as the player, further enhancing the player's sense of sharing a real-world experience with the game character.

[0132] In addition to the above-mentioned methods, for example, the gaze direction setting processing unit 127 may set the gaze direction of the game character based on either the direction of the player's eyes or the direction of the head, for example, the direction of the head, detected by the gaze direction detection unit 15 of the head-mounted display 7.

[0133] FIG. 29 shows an example of changes in the gaze direction of the game character and the gaze direction of the player in this case. As shown in the upper part of FIG. 29, when the head direction 137a of the player 135 turns, for example, to the right when viewed from the front of the player, the gaze direction 141 of the game character 139 changes to be approximately parallel to the head direction 137a of the player 135 and follows it. Next, as shown in the lower part of FIG. 29, when the player 135 turns only the eye direction 137b toward the game character 139 without changing the head direction 137a, the gaze direction 141 of the game character 139 does not change. This allows the player to change only the direction of their eyes and visually recognize that the game character is looking at the same thing as the player.

[0134] According to this modification, it is possible to change the direction of the game character's gaze so that it follows only the direction of the player's head, for example. By controlling the direction of the game character's gaze by independently detecting the direction of the player's eyes and head in this way, it becomes possible for the game character's gaze direction to follow the direction of the player's head, and for the player to change only the direction of their eyes to look at the game character and confirm it, an action known as "glancing," which further enhances the player's sense of sharing a real-world experience with the game character.

[0135] 6. Third Embodiment In the third embodiment, details of processing will be described for detecting the state of wind in the real space around the player and changing the manner of deformation of flexible objects (such as hair and clothing) associated with the game character in accordance with the detected state of wind. The processing content of this embodiment is executed in the communication processing in step S50 described above. For example, it may be executed when switching to the fan mode described above.

[0136] (6-1. Functional configuration of information processing device) First, an example of the configuration of the head mounted display 7 and the functional configuration of the information processing device 3 according to the third embodiment will be described with reference to Fig. 30. Note that the head mounted display 7 has the same configuration as that shown in Fig. 2 described above in addition to the configuration shown in Fig. 30, but the same configuration as that shown in Fig. 2 is not shown.

[0137] As shown in FIG. 30, the head mounted display 7 has a wind detection unit 145. The wind detection unit 145 detects the state of wind in the real space around the player. The wind detection unit 145 detects at least one of the wind volume, wind speed, and wind direction around the player as the "wind state." The wind detection unit 145 may be installed on the front, side, or rear of the head mounted display 7, or may be installed in multiple locations. The wind detection unit 145 may be installed integrally with the head mounted display 7, or may be installed separately so as to be able to send and receive signals.

[0138] The wind detection unit 145 has a wind volume detection unit 147, a wind speed detection unit 149, and a wind direction detection unit 151. The wind volume detection unit 147 is, for example, an anemometer, and detects the volume of wind around the player. The wind speed detection unit 149 is, for example, an anemometer, and detects the wind speed around the player. The wind direction detection unit 151 is, for example, a wind vane, and detects the wind direction around the player. Note that the wind volume detection unit 147, the wind speed detection unit 149, and the wind direction detection unit 151 do not need to be separate units, and two or more detection units may be integrated together, for example, like a wind vane and anemometer.

[0139] The information processing device 3 has a third character display processing unit 153, an object transformation processing unit 155, a third character action setting processing unit 157, and an additional information acquisition processing unit 131.

[0140] Third character display processing unit 153 (an example of a character display processing unit) causes display unit 13 to display an image of a virtual game character superimposed on an image of real space.

[0141] The object transformation processing unit 155 changes the deformation mode of a flexible object attached to a game character based on the wind conditions (at least one of wind volume, wind speed, and wind direction) detected by the wind detection unit 145 of the head-mounted display 7. The "flexible object" is, for example, the hair of the game character, clothes or swimsuit worn by the game character, etc. If the game character is a female character, it may also include a body part such as the chest. The "deformation mode" is, for example, the magnitude of the deformation, the direction of the deformation, shaking, vibration, flipping up, etc. If the flexible object is, for example, clothes or swimsuit, it may also include tearing, breaking, disassembly, etc.

[0142] Third character action setting processing section 157 (an example of a character action setting processing section) sets the action of the game character to change based on the wind conditions (at least one of wind volume, wind speed, and wind direction) detected by wind detection section 145 of head-mounted display 7. Third character display processing section 153 displays an image of the game character on display section 13 so that the game character performs the action set by third character action setting processing section 157.

[0143] The incidental information acquisition processing unit 131 is the same as in the second embodiment, and therefore a description thereof will be omitted. The third character action setting processing unit 157 sets the action of the game character based on the information acquired by the incidental information acquisition processing unit 131. The third character display processing unit 153 displays an image of the game character on the display unit 13 so that the game character performs the action set by the third character action setting processing unit 157.

[0144] 31 and 32 show examples of game screens in this embodiment. The example shown in FIG. 31 illustrates a case where an event occurs in the real world that causes wind around the player, and the strength of the wind is relatively weak. The event that causes wind is not particularly limited, but may include, for example, the player lifting their head with their hand or a fan, breathing out, being exposed to the wind from a fan, being exposed to the wind from a hair dryer, or being exposed to the wind from outside. Note that in the cases of lifting their head, breathing out, being exposed to the wind from a hair dryer, etc., the wind detection unit 145 may be installed separately from the head-mounted display 7 so as to be able to detect these winds. As shown in FIG. 31, the hair 161 of the game character 159 is blowing in the direction of the wind. At this time, the game character 159 may be made to utter a line corresponding to the strength of the wind, such as "It's cool and comfortable."

[0145] The example shown in Figure 32 is a case where an event occurs in the real world that causes wind around the player, and the wind is relatively strong. As shown in Figure 32, hair 161 of game character 159 is blowing wildly in the direction of the wind, and skirt 163 is flipped up. In this case, game character 159 may be made to scream "Eek!" and hold skirt 163, or act embarrassed.

[0146] Furthermore, by setting actions based on information acquired by the incidental information acquisition processing unit 131, more advanced communication between the player and the game character becomes possible. For example, in the case shown in FIG. 31 above, it may be determined that it is summer from calendar information, and that the wind from a fan is coming from a camera image or the player's voice, and the character may be made to say something like, "In summer, all you need is a fan." Also, it may be determined that the character is near the ocean at night based on time zone information and current location information obtained by GPS, and the character may be made to say something like, "The ocean breeze at night is so pleasant." Also, for example, in FIG. 32 above, it may be determined that a typhoon is approaching based on weather information, and the character may be made to say something like, "A typhoon is approaching."

[0147] Note that the processing in each processing unit described above is not limited to these examples of division of processing, and may be performed by, for example, a smaller number of processing units (e.g., one processing unit), or by further subdivided processing units. The functions of each processing unit described above are implemented by a game program executed by a CPU 301 (see FIG. 38 described below), but some of them may be implemented by actual devices such as dedicated integrated circuits (ASICs, FPGAs, etc.) or other electrical circuits. Furthermore, the processing units described above are not limited to being implemented entirely on the information processing device 3 side, and some or all of them may be implemented on the head-mounted display 7 side (e.g., the control unit 27). In this case, the control unit 27 of the head-mounted display 7 is an example of an information processing device.

[0148] (6-2. Processing procedure executed by the information processing device) Next, using Figure 33, we will explain an example of a processing procedure executed by the CPU 301 of the information processing device 3 according to the third embodiment when changing the deformation pattern and behavior of a flexible object associated with a game character according to the detected wind conditions.

[0149] In step S310, the information processing device 3 causes the third character display processing unit 153 to display on the display unit 13 an image of a virtual game character superimposed on the image of real space.

[0150] In step S320, the information processing device 3 acquires information related to the state of wind in the real space around the player detected by the wind detection unit 145 of the head-mounted display 7 (at least one of wind volume, wind speed, and wind direction).

[0151] In step S330, the information processing device 3 sets the object transformation processing unit 155 to change the deformation pattern of the flexible object associated with the game character based on the wind conditions acquired in step S320, and causes the third character display processing unit 153 to display an image of the flexible object on the display unit 13 so that the flexible object undergoes the set deformation.

[0152] In step S340, information processing device 3 causes third character behavior setting processing unit 157 to set the behavior of the game character to change based on the wind conditions acquired in step S320, and causes third character display processing unit 153 to display an image of the game character on display unit 13 so that the game character performs the set behavior. This ends this flowchart.

[0153] The above-described processing procedure is an example, and at least some of the procedures may be deleted or changed, or other procedures may be added. Furthermore, the order of at least some of the procedures may be changed, or multiple procedures may be combined into a single procedure.

[0154] (6-3. Effects of the Third Embodiment) As described above, the game program of this embodiment causes the display unit 13, which is configured to be wearable on the player's head and displays a virtual image superimposed on an image of real space, the wind detection unit 145, which detects the wind conditions in the real space around the player, and the information processing device 3, which is configured to send and receive signals, to function as a third character display processing unit 153, which displays an image of a virtual game character on the display unit 13, and an object transformation processing unit 155, which changes the deformation mode of a flexible object associated with the game character based on the detected wind conditions.

[0155] In this embodiment, an image of a virtual game character is superimposed on an image of real space and displayed by display unit 13. At this time, the wind conditions in the real space around the player are detected, and the deformation mode of a flexible object associated with the game character changes according to the detected wind conditions.

[0156] This allows for the deformation of the game character's hair, clothing, etc. to be changed depending on the strength and direction of the wind when an event that creates wind occurs in the real world. For example, if the wind is strong, the hair will flutter wildly in the wind direction and the skirt will be turned up, while if the wind is weak, only the hair will flutter. This allows the player to feel as if they are sharing an experience in the real world with the game character.

[0157] In addition, in this embodiment, the game program may further cause the information processing device 3 to function as a third character behavior setting processing unit 157 that sets the behavior of the game character to change based on the detected wind conditions, and in that case, the third character display processing unit 153 may display an image of the game character on the display unit 13 so that the game character performs the set behavior.

[0158] In this case, if an event that causes wind occurs in the real world, the game character's behavior can be changed depending on the wind conditions. For example, if the wind is strong, the character can scream "Eek!" and act embarrassed while trying to prevent her skirt from flipping up, while if the wind is weak, the character can say something appropriate to the wind strength, such as "It's cool and comfortable." This allows the player to feel more realistic, as if they are sharing a real-world experience with the game character.

[0159] In this embodiment, the wind detection unit 145 may detect at least one of the wind volume, wind speed, and wind direction around the player, and the object transformation processing unit 155 may change the manner of deformation of the flexible object based on at least one of the detected wind volume, wind speed, and wind direction.

[0160] In this case, when an event that causes wind occurs in the real world, it is possible to change not only the strength of the wind (wind volume and wind speed) but also the manner in which the game character's hair, clothing, etc., deform (including the direction of deformation) depending on the wind direction. For example, when the wind volume and wind speed are strong, the hair will flutter widely in the wind direction and the skirt will be turned up in the direction corresponding to the wind direction, while when the wind volume and wind speed are weak, only the hair will flutter in the wind direction. This makes it possible for the player to feel more realistic, as if they are sharing a real-world experience with the game character.

[0161] 7. Fourth Embodiment In the fourth embodiment, details of processing will be described for detecting humidity in the real space around the player and changing the wetness of the game character's skin or objects associated with the game character (hair, clothes, etc.) according to the detected humidity. Note that the processing content of this embodiment is executed in the communication processing in step S50 described above. For example, it may be executed when switching to the spraying mode described above.

[0162] (7-1. Functional configuration of information processing device) First, an example of the configuration of the head mounted display 7 and the functional configuration of the information processing device 3 according to the fourth embodiment will be described with reference to Fig. 34. Note that the head mounted display 7 has the same configuration as that shown in Fig. 2 described above in addition to the configuration shown in Fig. 34, but the same configuration as that shown in Fig. 2 is not shown.

[0163] As shown in FIG. 34 , the head mounted display 7 has a humidity detection unit 165 and a temperature detection unit 167. The humidity detection unit 165 is, for example, a hygrometer and detects the humidity in the real space around the player. The temperature detection unit 167 is, for example, a thermometer and detects the temperature in the real space around the player. The humidity detection unit 165 and the temperature detection unit 167 may be installed on the front, side, or rear of the head mounted display 7, or may be installed in multiple locations. Furthermore, the humidity detection unit 165 and the temperature detection unit 167 may be installed integrally with the head mounted display 7, or may be installed separately so as to be able to send and receive signals. Note that only one of the humidity detection unit 165 and the temperature detection unit 167 may be installed, rather than both.

[0164] The information processing device 3 has a fourth character display processing unit 169 , a wetness processing execution processing unit 171 , a sweating processing execution processing unit 173 , a fourth character action setting processing unit 175 , and an additional information acquisition processing unit 131 .

[0165] A fourth character display processing unit 169 (an example of a character display processing unit) causes the display unit 13 to display an image of a virtual game character superimposed on an image of real space.

[0166] The wetness processing execution processor 171 executes wetness processing to change the wetness of the game character's skin or objects associated with the game character based on the humidity detected by the humidity detection unit 165 of the head-mounted display 7. "The game character's skin" includes the skin of the game character's face and body. "Objects associated with the game character" include, for example, the game character's hair, the clothes and swimsuit worn by the game character, etc. For example, wetness processing may be performed so that the wetness (e.g., degree of wetness, range of wetness, etc.) and the transparency of the clothes (e.g., degree of transparency, range of transparency, etc.) change depending on the humidity level. For example, when the humidity is high, the game character's skin and hair become soaked or the clothes worn by the game character become wet and transparent, and when the humidity is low, the game character's skin and hair become slightly wet and the clothes do not become transparent. In addition to the wetness processing, the degree of flushing of the game character's face may also be changed depending on the humidity level.

[0167] The sweating process execution processor 173 executes sweating process to change the degree of sweating on the game character's skin based on the temperature detected by the temperature detection unit 167 of the head-mounted display 7. For example, the sweating process may be performed so that the game character does not sweat or sweats only a small amount over a small area when the temperature is low, and sweats a large amount over a large area when the temperature is high, changing the amount of sweat or the area where sweating occurs depending on the temperature. In addition to the sweating process, the degree of flushing on the game character's face may also be changed depending on the temperature.

[0168] Fourth character action setting processing unit 175 (an example of a character action setting processing unit) sets the game character's action to change based on at least one of the humidity and the temperature detected by humidity detection unit 165 and temperature detection unit 167 of head-mounted display 7. Fourth character display processing unit 169 displays an image of the game character on display unit 13 so that the game character performs the action set by fourth character action setting processing unit 175.

[0169] The incidental information acquisition processing unit 131 is the same as in the second embodiment, and therefore a description thereof will be omitted. The fourth character action setting processing unit 175 sets the action of the game character based on the information acquired by the incidental information acquisition processing unit 131. The fourth character display processing unit 169 displays an image of the game character on the display unit 13 so that the game character performs the action set by the fourth character action setting processing unit 175.

[0170] 35 and 36 show examples of game screens in this embodiment. The example shown in FIG. 35 illustrates a case where an event occurs in the real world that increases humidity around the player, and the humidity is relatively high. Events that increase humidity are not particularly limited, and examples include spraying with a spray bottle, spraying water from a hose, taking a shower, and being caught in the rain. In the case of the above-mentioned spraying, spraying water from a hose, showering, etc., the humidity detection unit 165 may be installed separately from the head-mounted display 7 so that changes in humidity caused by these events can be detected. In the example shown in FIG. 35, the face 179, hair 181, clothes 183, etc. of the game character 177 are soaked. In this case, the game character 177 may utter a line corresponding to the humidity, such as "I got wet in the rain."

[0171] The example shown in Figure 36 is a case where an event occurs in the real world that causes the temperature around the player to rise, and that temperature is relatively high. As shown in Figure 36, the face 187 of game character 185 is flushed and sweating, and the game character 185 is wiping his face with a handkerchief 189. At this time, the game character 185 may say something in response to the temperature, such as "It's really hot today."

[0172] Furthermore, by setting behavior based on information acquired by the incidental information acquisition processing unit 131, it becomes possible to carry out more advanced communication between the player and the game character. For example, in the case shown in FIG. 35 above, it may be determined that water is being sprayed from a hose based on a camera image or the player's voice, and the character may be made to say something like, "I'm soaked!" to make the character angry. Also, it may be determined that there is an evening shower based on time zone information and weather information, and the character may be made to say something like, "I got caught in an evening shower." Also, for example, in FIG. 36 above, it may be determined that the season is summer based on calendar information, and the character may be made to say something like, "It's going to be extremely hot this summer."

[0173] In this embodiment, the case where both humidity and temperature are detected has been described, but it is not necessary to detect both, and it is also possible to detect only humidity or only temperature and perform processing based on the detection results.

[0174] Note that the processing in each processing unit described above is not limited to these examples of division of processing, and may be performed by, for example, a smaller number of processing units (e.g., one processing unit), or by further subdivided processing units. The functions of each processing unit described above are implemented by a game program executed by a CPU 301 (see FIG. 38 described below), but some of them may be implemented by actual devices such as dedicated integrated circuits (ASICs, FPGAs, etc.) or other electrical circuits. Furthermore, the processing units described above are not limited to being implemented entirely on the information processing device 3 side, and some or all of them may be implemented on the head-mounted display 7 side (e.g., the control unit 27). In this case, the control unit 27 of the head-mounted display 7 is an example of an information processing device.

[0175] (7-2. Processing procedure executed by information processing device) Next, using Figure 37, we will explain an example of a processing procedure executed by the CPU 301 of the information processing device 3 related to the fourth embodiment when changing the wetness or sweating of the skin of a game character or an object associated with the game character in accordance with the detected humidity and temperature.

[0176] In step S410, the information processing device 3 causes the fourth character display processing unit 169 to display on the display unit 13 an image of the virtual game character superimposed on the image of real space.

[0177] In step S420, the information processing device 3 acquires information relating to the humidity and temperature of the real space around the player detected by the humidity detection unit 165 and the temperature detection unit 167 of the head-mounted display .

[0178] In step S430, the information processing device 3 executes wetness processing by the wetness processing execution processing unit 171 so as to change the wetness of the game character's skin or an object associated with the game character based on the humidity detected by the humidity detection unit 165 of the head-mounted display 7, and displays the result on the display unit 13.

[0179] In step S440, the information processing device 3 executes a sweating process by the sweating process execution processing unit 173 so as to change the degree of sweating on the game character's skin based on the temperature detected by the temperature detection unit 167 of the head-mounted display 7, and displays the result on the display unit 13.

[0180] In step S450, the information processing device 3 causes the fourth character behavior setting processing unit 175 to set the behavior of the game character to change based on the humidity and temperature detected by the humidity detection unit 165 and the temperature detection unit 167 of the head-mounted display 7, and causes the fourth character display processing unit 169 to display an image of the game character on the display unit 13 so that the game character performs the set behavior. This ends this flowchart.

[0181] The above-described processing procedure is an example, and at least some of the above procedures may be deleted or changed, or other procedures may be added. Furthermore, the order of at least some of the above procedures may be changed, or multiple procedures may be combined into a single procedure. For example, although the above describes a case where both humidity and temperature are detected, it is not necessary to detect both. Only humidity or only temperature may be detected, and processing may be performed based on the detection results. If only humidity is detected, step S440 is unnecessary, and if only temperature is detected, step S430 is unnecessary.

[0182] (7-3. Effects of the Fourth Embodiment) As described above, the game program of this embodiment causes the display unit 13, which is configured to be wearable on the player's head and displays a virtual image superimposed on an image of real space, the humidity detection unit 165, which detects the humidity in the real space around the player, and the information processing device 3, which is configured to send and receive signals, to function as a fourth character display processing unit 169, which displays an image of a virtual game character on the display unit 13, and a wetness processing execution processing unit 171, which executes wetness processing to change the wetness of the game character's skin or an object associated with the game character based on the detected humidity.

[0183] In this embodiment, an image of a virtual game character is superimposed on an image of real space and displayed by display unit 13. At this time, the humidity of the real space around the player is detected, and the wetness of the game character's skin or an object associated with the game character changes depending on the detected humidity.

[0184] This allows, for example, when an event occurs in the real world that increases humidity, the wetness of the game character's skin, hair, clothes, etc. to be changed according to the humidity level. For example, when the humidity is high, the skin and hair will be soaked and the clothes will be transparent, and when the humidity is low, the skin and hair will be only slightly wet and the clothes will not be transparent. This allows the player to feel as if they are sharing an experience in the real world with the game character.

[0185] In addition, in this embodiment, the game program may further cause the information processing device 3 to function as a fourth character behavior setting processing unit 175 that sets the behavior of the game character to change based on the detected humidity, and in that case, the fourth character display processing unit 169 may display an image of the game character on the display unit 13 so that the game character performs the set behavior.

[0186] In this case, for example, if an event occurs in the real world that causes humidity to rise, the game character's behavior can be changed depending on the humidity level. For example, if the humidity is high, the character's clothes will become transparent and the character will scream "Eek!" in embarrassment, while if the humidity is low, only the skin will become wet and the character will say something like "It's cold and feels good." This makes it feel more realistic for the player to feel as if they are sharing a real-world experience with the game character.

[0187] In addition, in this embodiment, the game program may further cause the information processing device 3 to function as a sweating process execution processing unit 173 that executes sweating process so as to change the degree of sweating on the game character's skin based on the detected temperature.

[0188] In this case, for example, if an event occurs in the real world that causes a rise in temperature (such as taking a walk during the day in summer), the degree of sweating of the game character's skin can be changed depending on the temperature, making the player feel as if they are sharing a real-world experience with the game character.

[0189] Furthermore, in addition to what has already been described above, the methods according to the above-described embodiments and modifications may be used in appropriate combinations. Although not specifically illustrated, the above-described embodiments and modifications may be implemented with various modifications within the scope of their spirit.

[0190] <8. Hardware configuration of information processing device> Next, an example of a hardware configuration of the information processing device 3 that realizes each processing unit implemented by a program executed by the CPU 301 etc. described above will be described with reference to Fig. 38. Note that the control unit 27 of the head-mounted display 7 may have a similar hardware configuration.

[0191] 38, the information processing device 3 includes, for example, a CPU 301, a ROM 303, a RAM 305, a GPU 306, a dedicated integrated circuit 307 constructed for a specific application, such as an ASIC or an FPGA, an input device 313, an output device 315, a recording device 317, a drive 319, a connection port 321, and a communication device 323. These components are connected to each other via a bus 309, an input / output interface 311, etc., so that signals can be transmitted between them.

[0192] The game program can be recorded in, for example, the ROM 303, the RAM 305, or a recording device 317 such as a hard disk.

[0193] The game program may also be temporarily or permanently (non-temporarily) recorded on a removable recording medium 325, such as a magnetic disk such as a flexible disk, various optical disks such as CDs, MO disks, and DVDs, or a semiconductor memory. Such recording medium 325 may also be provided as a so-called package software. In this case, the game program recorded on such recording medium 325 may be read by the drive 319 and recorded on the recording device 317 via the input / output interface 311, the bus 309, etc.

[0194] The game program may also be stored, for example, on a download site, another computer, or another storage device (not shown). In this case, the game program is transferred via a network NW such as a LAN or the Internet, and the communication device 323 receives the program. The program received by the communication device 323 may then be recorded in the storage device 317 via the input / output interface 311, the bus 309, etc.

[0195] The game program may also be recorded in, for example, an appropriate externally connected device 327. In this case, the game program may be transferred via an appropriate connection port 321 and recorded in the recording device 317 via the input / output interface 311, the bus 309, etc.

[0196] Then, CPU 301 executes various processes in accordance with the programs recorded in the recording device 317, thereby realizing processes by the object selection processing unit 55, object display processing unit 57, object adjustment processing unit 59, character posture setting processing unit 61, first character display processing unit 63, object non-display processing unit 65, object registration processing unit 111, first character action setting processing unit 113, line of sight direction setting processing unit 127, second character display processing unit 129, incidental information acquisition processing unit 131, second character action setting processing unit 133, time difference setting processing unit 143, third character display processing unit 153, object transformation processing unit 155, third character action setting processing unit 157, fourth character display processing unit 169, wetness processing execution processing unit 171, sweating processing execution processing unit 173, fourth character action setting processing unit 175, etc. In this case, CPU 301 may directly read out and execute the programs from the recording device 317, or may temporarily load the programs into RAM 305 and then execute them. Furthermore, when the CPU 301 receives a program via the communication device 323 , the drive 319 , or the connection port 321 , the CPU 301 may directly execute the received program without recording it in the recording device 317 .

[0197] Furthermore, the CPU 301 may perform various processes based on signals and information input from an input device 313, such as a microphone, mouse, keyboard, etc. (not shown), including the game controller 5 described above, as needed.

[0198] The GPU 306 performs processing for image display, such as rendering processing, in response to instructions from the CPU 301 .

[0199] Then, the CPU 301 and the GPU 306 output the results of the above processing from an output device 315 including, for example, the display unit 13 of the head-mounted display 7. Furthermore, the CPU 301 and the GPU 306 may transmit the processing results via the communication device 323 or the connection port 321 as necessary, or may record the results in the recording device 317 or the recording medium 325. [Explanation of symbols]

[0200] 3. Information processing equipment 7. Head-mounted display 13 Display section 15 Gaze direction detection unit 27 Control unit (information processing device) 55 Object selection processing section 57 Object display processing section 59 Object Adjustment Processing Unit 61 Character posture setting processing unit 63 First character display processing unit 65 Object hiding processing section 67 Virtual Seated Object (Virtual Object) 71 Game Characters 73 Virtual Seated Object (Virtual Object) 77 Game Characters 79 Virtual Seated Object (Virtual Object) 83 Game Characters 85 Virtual Objects (Virtual Objects) 89 Game Characters 91 Virtual Objects (Virtual Objects) 95 Game Characters 97 Virtual Objects 101 Game Characters 103 Virtual Objects (Virtual Objects) 107 Game Characters 111 Object registration processing unit 113 First character action setting processing unit (character action setting processing unit) 127 Gaze direction setting processing unit 129 Second character display processing unit (character display processing unit) 131 Additional information acquisition processing unit 133 Second character action setting processing unit 143 Time difference setting processing unit 145 Wind detection unit 147 Air volume detection unit 149 Wind speed detector 151 Wind direction detector 153 third character display processing unit (character display processing unit) 155 Object transformation processing section 157 Third character action setting processing unit (character action setting processing unit) 165 Humidity detection unit 167 Temperature detection unit 169 Fourth character display processing unit (character display processing unit) 171 Wetting process execution processing unit 173 Sweat processing execution processing unit 175 Fourth character action setting processing unit (character action setting processing unit) 325 Recording Media

Claims

1. a display unit configured to be wearable on the head of a player and displaying a virtual image superimposed on an image of real space; a gaze direction detection unit detecting the direction of the player's gaze; and an information processing device configured to transmit and receive signals, a gaze direction setting processing unit that sets the gaze direction of a virtual game character having eyes based on the detected gaze direction of the player; a character display processing unit that displays an image of the game character on the display unit so that the game character changes its line of sight based on the set line of sight direction of the game character; an object selection processing unit that selects a type of virtual object based on an input from the player; a character posture setting processing unit that sets a posture of the game character based on the selected type of the virtual object; an object adjustment processing unit that adjusts at least one of a position and an orientation of the virtual object with respect to the image in real space based on an input from the player; a character action setting processing unit that sets an action of the game character based on at least one of the position and the orientation of the virtual object; It functions as The character display processing unit When an image of the game character is displayed on the display unit so as to be positioned in accordance with at least one of the position and orientation of the virtual object, the image of the virtual object is hidden, and the image of the game character is displayed on the display unit so as to cause the game character to perform the set action. Game program.

2. The information processing device an additional information acquisition processing unit that acquires additional information from at least one of an information distribution site, a database, a GPS satellite, and a sensor; It further functions as The character behavior setting processing unit setting the behavior of the game character based on the supplementary information; The game program according to claim 1 .

3. The line of sight direction setting processing unit setting the line of sight direction of the game character so as to follow the detected line of sight direction of the player; The game program according to claim 2 .

4. The line of sight direction setting processing unit when the detected direction of the player's line of sight is a direction corresponding to the game character, setting the direction of the game character's line of sight to a direction corresponding to the player; The game program according to claim 3 .

5. The information processing device a time difference setting processing unit that, when the detected direction of the player's line of sight changes from a direction not corresponding to the game character to a direction corresponding to the game character, sets a predetermined time difference between the change in the direction of the player's line of sight and the change in the direction of the game character's line of sight; To further function as The game program according to claim 4.

6. The character display processing unit displaying an image of the game character so that the game character changes its line of sight by changing at least one of the direction of the game character's eyes, head, and body; 6. The game program according to claim 1.

7. The gaze direction detection unit an eye direction detection unit that detects the direction of the player's eyes; a head direction detection unit that detects the direction of the player's head, The line of sight direction setting processing unit setting a line of sight direction of the game character based on at least one of the detected eye direction or head direction of the player; 7. The game program according to claim 1.

8. 8. A recording medium readable by an information processing device, on which the game program according to claim 1 is recorded.

9. A game processing method executed by an information processing device configured to be wearable on a player's head and to display a virtual image superimposed on an image of real space, a gaze direction detection unit that detects the direction of the player's gaze, and to transmit and receive signals, comprising: setting a gaze direction of a virtual game character having eyes based on the detected gaze direction of the player; displaying an image of the game character on the display unit so that the game character changes its line of sight based on the set line of sight direction of the game character; selecting a type of virtual object based on an input from the player; setting a posture of the game character based on the selected type of the virtual object; adjusting at least one of a position and an orientation of the virtual object with respect to the image of real space based on an input from the player; setting an action of the game character based on at least one of the position and the orientation of the virtual object; and The step of displaying the image of the game character on the display unit includes: When an image of the game character is displayed on the display unit so as to be positioned in accordance with at least one of the position and orientation of the virtual object, the image of the virtual object is hidden, and the image of the game character is displayed on the display unit so as to cause the game character to perform the set action. Game processing method.

Images