Program and image display system

Abstract

To enable a user to easily grasp positional relation between an object in a virtual space and a reference line.SOLUTION: A program makes a computer function as: a display control part for displaying an image of a virtual space taken by a virtual camera whose direction is controlled according to direction of a head-mounted display on the head-mounted display; and an operation control part for making a character perform an action according to a position where a reference line that changes direction in the virtual space according to the direction of the head-mounted display intersects with the character that moves in the virtual space.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a technique for displaying an image captured in a virtual space. [Background technology]

[0002] A technique for displaying an image captured by a virtual camera (hereinafter referred to as a "virtual camera") installed in a virtual space on a display device has been proposed. For example, Patent Document 1 discloses a configuration for displaying a menu image in a virtual space according to the tilt of a head-mounted display. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-115122 Summary of the Invention [Problem to be solved by the invention]

[0004] Content such as games that allow communication between characters and other objects in a virtual space has been proposed. When this type of content is displayed using a head-mounted display, there is a problem in that it is difficult for the user to grasp the positional relationship between a virtual straight line (hereinafter referred to as a "reference line"), such as the optical axis of a virtual camera or the user's line of sight, and an object such as a character in the virtual space. In consideration of the above circumstances, the present invention aims to make it easier for the user to grasp the positional relationship between an object in a virtual space and the reference line. [Means for solving the problem]

[0005] In order to solve the above problems, a preferred embodiment of the present invention provides a program that causes a computer to function as a display control unit that displays, on a display device of the head-mounted display, a stereoscopic image that utilizes binocular parallax, an image of a virtual space captured by a virtual camera whose direction is controlled according to the direction of the head-mounted display, wherein the display control unit displays an indication image in the direction of a reference line whose direction in the virtual space changes according to the direction of the head-mounted display, and changes the display mode of the indication image according to the positional relationship between the reference line and an object in the virtual space.

[0006] An image display system according to a preferred embodiment of the present invention is an image display system comprising a head-mounted display and an information processing device, wherein the information processing device comprises a display control unit that causes a display device of the head-mounted display to display a stereoscopic image utilizing binocular parallax, the stereoscopic image being an image captured of a virtual space by a virtual camera whose direction is controlled according to the direction of the head-mounted display, and the display control unit displays an indication image in the direction of a reference line whose direction in the virtual space changes according to the direction of the head-mounted display, and changes the display mode of the indication image according to the positional relationship between the reference line and an object in the virtual space. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a perspective view illustrating an example of the appearance of an image display system according to a first embodiment of the invention. [Figure 2] FIG. 1 is a block diagram illustrating a configuration of an image display system. [Figure 3] FIG. 1 is a block diagram illustrating an example of a functional configuration of an image display system. [Figure 4] FIG. 1 is an explanatory diagram of a virtual space. [Figure 5] FIG. 2 is an explanatory diagram of an image displayed by a display device. [Figure 6] 4 is a flowchart illustrating the content of a process executed by a control device. [Figure 7]FIG. 10 is an explanatory diagram of the rotation of the HMD in the roll direction. [Figure 8] FIG. 2 is an explanatory diagram of an image displayed by a display device. [Figure 9] FIG. 10 is an explanatory diagram of the movement of the virtual camera in Modification A4. [Figure 10] FIG. 10 is an explanatory diagram of a change in the imaging range in Modification A5. [Figure 11] FIG. 10 is a block diagram illustrating an example of the functional configuration of an image display system according to a second embodiment. [Figure 12] FIG. 11 is an explanatory diagram of an image displayed when the reference line does not intersect with the character in the second embodiment. [Figure 13] FIG. 11 is an explanatory diagram of an image displayed when a reference line intersects with a character in the second embodiment. [Figure 14] FIG. 10 is an explanatory diagram of the relationship between a character and an instruction image in the second embodiment. [Figure 15] 10 is a flowchart illustrating the content of a process executed by a control device of a second embodiment. [Figure 16] FIG. 10 is an explanatory diagram of a change over time in an instruction image in Modification B2. [Figure 17] FIG. 10 is an explanatory diagram of an instruction image in Modification Example B3. [Figure 18] FIG. 10 is an explanatory diagram of an instruction image in Modification Example B4. [Figure 19] FIG. 10 is a perspective view illustrating an example of the appearance of an image display system in Modification C5. DETAILED DESCRIPTION OF THE INVENTION

[0008] [A: First embodiment] FIG. 1 is a perspective view illustrating an example of the appearance of an image display system 1A according to a first embodiment of the present invention. The image display system 1A is a video device for displaying images, and includes a terminal device 10 and a wearing attachment 20. The terminal device 10 is a portable information terminal such as a smartphone or a tablet terminal. The terminal device 10 is detachably attached to the wearing attachment 20. The wearing attachment 20 is a device for wearing the terminal device 10 on the head of a user U. For example, as illustrated in FIG. 1, a goggle-type attachment having a belt that is wrapped around the head of the user U is suitable as the wearing attachment 20.

[0009] Fig. 2 is a block diagram illustrating an example of the configuration of terminal device 10. As illustrated in Fig. 2, terminal device 10 is a computer system including a control device 11, a storage device 12, a display device 13, and a detection device 14. Control device 11 is a processor such as a CPU (Central Processing Unit) and performs overall control of each element of image display system 1A. Storage device 12 stores programs executed by control device 11 and various data used by control device 11. For example, storage device 12 is configured by a known storage medium such as a magnetic storage medium or a semiconductor storage medium, or a combination of multiple types of storage media.

[0010] The display device 13 displays images under the control of the control device 11. For example, a flat display such as a liquid crystal display panel or an organic EL (Electroluminescence) display panel is used as the display device 13. As can be seen from FIG. 1 , when the wearing device 20 is worn on the head of the user U, the display device 13 is disposed in front of both eyes of the user U with the display surface facing the head of the user U. The display device 13 of the first embodiment displays a stereoscopic image that allows the user U to perceive a stereoscopic effect. The stereoscopic image is composed of an image for the right eye and an image for the left eye that are provided with binocular parallax. The user U perceives a stereoscopic effect by having the right eye image viewed by the right eye of the user U and the left eye image viewed by the left eye of the user U.

[0011] 2, the display device 13, the detection device 14, and the wearing device 20 constitute a head-mounted display (hereinafter referred to as "HMD") 30, and the control device 11 and the storage device 12 constitute an information processing device 40 that displays an image on the HMD 30. That is, the image display system 1A includes the HMD 30 and the information processing device 40. As described above, in the first embodiment, the information processing device 40, the display device 13, and the detection device 14 are realized by a single terminal device 10. Note that the information processing device 40 may be understood as an element of the HMD 30.

[0012] The detection device 14 in FIG. 2 is a sensor that outputs a detection signal corresponding to the attitude of the HMD 30. Specifically, the detection device 14 is composed of one or more types of sensors selected from a plurality of types of sensors, such as a gyro sensor that detects angular velocity, an acceleration sensor that detects acceleration, an inclination sensor that detects tilt angle, and a geomagnetic sensor that detects direction using geomagnetism. By analyzing the detection signal output from the detection device 14, it is possible to identify the attitude or a change in attitude of the HMD 30. In other words, the detection device 14 is a sensor that outputs a detection signal corresponding to the attitude of the display device 13 or the terminal device 10. For convenience, an amplifier that amplifies the detection signal and an A / D converter that converts the detection signal from analog to digital are not shown.

[0013] As illustrated in FIG. 1 , the orientation of the HMD 30 is defined by three mutually orthogonal axes (X-axis, Y-axis, and Z-axis) at the origin O. The X-axis is an axis perpendicular to the display surface of the display device 13 and corresponds to the front-to-back direction of the user U. The Y-axis and Z-axis are axes parallel to the display surface of the display device 13. The Y-axis corresponds to the left-to-right direction of the user U, and the Z-axis corresponds to the up-to-down direction of the user U. Focusing on the display surface of the display device 13, the X-axis corresponds to the depth direction of the display surface, the Y-axis corresponds to the horizontal direction of the display surface, and the Z-axis corresponds to the vertical direction of the display surface. The direction of the circumference around the X-axis is the roll direction, the direction of the circumference around the Y-axis is the pitch direction, and the direction of the circumference around the Z-axis is the yaw direction. The direction of the X-axis is determined by the pitch angle and the yaw angle. Rotation in the roll direction corresponds to the movement of the user U tilting their head left or right while facing forward. That is, for example, when the user U tilts their head, the HMD 30 rotates in the roll direction. In the following description, one side in the roll direction will be referred to as the "first side" and the other side as the "second side" as shown in Figure 1. One of the clockwise and counterclockwise directions around the X axis is the first side, and the other is the second side.

[0014] 2 executes a program stored in the storage device 12, thereby functioning as a posture analysis unit 41, a display control unit 42A, and a motion control unit 43A, as illustrated in Fig. 3. Note that some or all of the functions of the control unit 11 may be realized by a dedicated electronic circuit.

[0015] The posture analysis unit 41 identifies the posture of the HMD 30 by analyzing the detection signal output by the detection device 14. Specifically, the posture analysis unit 41 sequentially generates posture data relating to the posture of the HMD 30 at a predetermined cycle. The posture data is data representing the posture of the HMD 30 or a change in posture.

[0016] The display control unit 42A causes the display device 13 to display images captured by a virtual camera E in a virtual space V. FIG. 4 is an explanatory diagram of the virtual space V. As illustrated in FIG. 4, the virtual camera E is installed in the virtual space V and captures an image of a specific range (hereinafter referred to as an "imaging range") R in the virtual space V. The imaging range R is a range that spans a predetermined angle in the vertical and horizontal directions with respect to the optical axis of the virtual camera E. The optical axis of the virtual camera E corresponds to the virtual line of sight of the user U in the virtual space V. The display control unit 42A of the first embodiment causes the display device 13 to display a stereoscopic image (an image for the right eye and an image for the left eye) utilizing binocular parallax. Therefore, the virtual camera E includes a first virtual camera that captures an image for the left eye and a second virtual camera that captures an image for the right eye. Image data representing the stereoscopic image captured by the virtual camera E is sequentially supplied from the display control unit 42A to the display device 13, whereby the stereoscopic image is displayed on the display device 13.

[0017] The display control unit 42A controls the direction of the virtual camera E in the virtual space V according to the direction of the HMD 30 identified by the posture analysis unit 41. Specifically, the direction of the optical axis of the virtual camera E (and further the imaging range R) changes in conjunction with the rotation of the X axis around the origin O. For example, when the HMD 30 rotates in the yaw direction, the optical axis of the virtual camera E rotates left and right around the origin O, and when the HMD 30 rotates in the pitch direction, the optical axis of the virtual camera E rotates up and down around the origin O. In the first embodiment, the posture of the virtual camera E (the angle around the optical axis) does not change even when the HMD 30 rotates in the roll direction. However, the tilt of the virtual camera E around the optical axis may be changed according to the rotation of the HMD 30 in the roll direction.

[0018] As illustrated in Fig. 4, a character C is placed in a virtual space V in which a virtual camera E is installed. The character C in the first embodiment is an object representing a virtual creature (e.g., a human, an animal, or a monster) active in the virtual space V, and is configured to include a head and a torso. The image display system 1A provides a user U with a game in which the user can interact with the character C by making contact (skinship) with the character C as needed in the virtual space V.

[0019] A reference line Q is set within the virtual space V. The reference line Q is a straight line whose direction within the virtual space V changes depending on the direction of the HMD 30 (the direction of the X-axis). In the first embodiment, as illustrated in FIG. 4, the optical axis of the virtual camera E is exemplified as the reference line Q. For example, if the user U turns his head to the right, the reference line Q rotates to the right, and if the user U turns his head to the left, the reference line Q rotates to the left. As can be understood from the above explanation, the reference line Q is the virtual line of sight of the user U within the virtual space V.

[0020] 5 is a schematic diagram of an image displayed on display device 13. As illustrated in FIG. 5, display control unit 42A causes display device 13 to display a stereoscopic image captured by virtual camera E in virtual space V. When character C is located within imaging range R, character C is displayed on display device 13. Display control unit 42A also causes display device 13 to display an image (hereinafter referred to as "indication image") G representing the direction of reference line Q. Indication image G is placed at a predetermined position on reference line Q within virtual space V. Since reference line Q in the first embodiment is the optical axis of virtual camera E, indication image G is displayed at a predetermined position within the display surface of display device 13.

[0021] The movement control unit 43A in FIG. 3 controls the movement of the character C in the virtual space V. The movement control unit 43A not only controls the movement of the character C in accordance with the progress of the game, but also causes the character C to perform a movement (hereinafter referred to as a "reaction movement") in response to contact by the user U in the virtual space V. Specifically, the movement control unit 43A determines whether the user U has contacted the character C in the virtual space V, and causes the character C to perform a reaction movement in response to the contact by the user U. Typical examples of reaction movements include various movements such as a change in posture, a change in facial expression, or the utterance of a specific line.

[0022] The movement control unit 43A determines whether the user U has contacted the character C in the virtual space V, depending on the change in the posture of the HMD 30 identified by the posture analysis unit 41. The movement control unit 43A of the first embodiment determines that the user U has contacted the character C in the virtual space V when the HMD 30 has rotated in the roll direction. That is, the rotation of the HMD 30 in the roll direction corresponds to an instruction (movement instruction) for the user U to contact the character C in the virtual space V. Specifically, when the HMD 30 has rotated in the roll direction, the movement control unit 43A determines that the user U has contacted a point P where the reference line Q and the surface of the character C intersect (hereinafter referred to as the "contact point"). The contact point P is located on the surface of the character C. Note that if the reference line Q intersects with the surface of the character C at multiple points, the point closest to the virtual camera E (i.e., the user U) among the multiple points is selected as the contact point P.

[0023] As described above, in the first embodiment, the movement of the character C in the virtual space V is controlled in accordance with the rotation of the HMD 30 in the roll direction. Therefore, it is possible to reflect instructions from the user U in the movement of the character C without the need to prepare an operation device, separate from the HMD 30, through which the user U inputs instructions to control the movement of the character C. In particular, in the first embodiment, it is determined that the user U has come into contact with the character C when the reference line Q and the character C intersect, so that the user U can operate the character C while keeping the character C within their field of vision. In other words, there is an advantage that the user U can operate the character C more intuitively.

[0024] FIG. 6 is a flowchart illustrating the specific contents of the process (hereinafter referred to as "first control process") executed by the movement control unit 43A. The movement control unit 43A repeatedly executes the first control process of FIG. 6 at a predetermined cycle. The storage device 12 stores contact determination data indicating whether or not the user U is in contact with the character C in the virtual space V (hereinafter referred to as "contact state"). The contact determination data is, for example, a flag indicating either a contact state or a non-contact state.

[0025] When the first control process starts, the operation control unit 43A determines whether or not the contact determination data indicates a contact state (Sa1). If the contact determination data indicates a non-contact state (Sa1: NO), the operation control unit 43A determines whether or not the HMD 30 has rotated in the roll direction (Sa2). Specifically, the operation control unit 43A determines whether or not the HMD 30 has rotated to a first side in the roll direction (for example, clockwise) by referring to the posture data sequentially supplied from the posture analysis unit 41.

[0026] FIG. 7 is an explanatory diagram of a process for determining whether the HMD 30 has rotated in the roll direction. As illustrated in FIG. 7, the operation control unit 43A determines that the HMD 30 has rotated in the roll direction when the angle θ by which the HMD 30 has rotated toward the first side in the roll direction exceeds a threshold θt. The angle θ is the angle by which the Z-axis or Y-axis has rotated around the X-axis, and is defined, for example, with the vertical direction as the reference (θ=0). On the other hand, when the rotation angle θ of the HMD 30 is below the threshold θt, the operation control unit 43A does not determine that the HMD 30 has rotated in the roll direction. As described above, in the first embodiment, if the rotation is below the threshold θt, the HMD 30 is not determined to have rotated in the roll direction. Therefore, it is possible to reduce the possibility that the HMD 30 will be determined to have rotated excessively frequently. Another advantage is that it is possible to reduce the possibility that the HMD 30 will be determined to have rotated even when the user U does not intend to rotate the HMD 30 (i.e., the possibility of erroneous operation).

[0027] In addition, not only when only a roll rotation occurs in the HMD30, but also when a roll rotation and a pitch or yaw rotation occur together, if the angle of the roll rotation exceeds the threshold value θt, it is determined that the HMD30 has rotated in the roll direction.

[0028] In the first control process of FIG. 6, when it is determined that the HMD 30 has rotated to the first side in the roll direction (Sa2: YES), the movement control unit 43A determines whether or not the reference line Q intersects with the character C in the virtual space V (Sa3). That is, it is determined whether or not the character C exists on the reference line Q. As illustrated in the example of FIG. 8, when the reference line Q intersects with the character C in the virtual space V (Sa3: YES), the movement control unit 43A determines that the user U is in contact with the contact point P of the character C (Sa4). Specifically, the movement control unit 43A changes the contact determination data to a numerical value indicating the contact state. As can be understood from the above description, when the HMD 30 has rotated in the roll direction, the movement control unit 43A determines that the user U has contacted the contact point P of the character C in the virtual space V. That is, the user U can instruct contact with the contact point P of the character C by rotating the HMD 30 in the roll direction.

[0029] When the state transitions to the contact state, the action control unit 43A causes the character C to perform a reaction action (Sa5). Specifically, the action control unit 43A causes the character C to perform a reaction action according to the position of the contact point P of the character C where the reference line Q intersects. That is, the reaction action performed by the character C changes depending on the position of the contact point P. For example, a table specifying the type of reaction action for each of a plurality of regions obtained by dividing the surface of the character C is stored in the storage device 12. The action control unit 43A causes the character C to perform a reaction action specified in the table for the region of the surface of the character C that includes the contact point P. For example, when the contact point P is located on the head of the character C, the action control unit 43A causes the character C to perform a reaction action that accepts contact by the user U (for example, a positive action such as being happy, laughing, or moving closer). On the other hand, when the contact point P is located on the torso of the character C, the action control unit 43A causes the character C to perform a reaction action that rejects contact by the user U (for example, a negative action such as being angry, sad, or moving away). According to the above configuration, it is possible to vary the reaction action of the character C in a variety of ways depending on the position of the contact point P.

[0030] On the other hand, if the HMD 30 does not rotate to the first side in the roll direction (Sa2: NO), or if the reference line Q does not intersect with the character C (Sa3: NO), the operation control unit 43A maintains the contact determination data at a value indicating a non-contact state. That is, even if the HMD 30 rotates to the first side in the roll direction, if the reference line Q does not intersect with the character C as shown in the example of FIG. 5, it is determined that the user U is not in contact with the character C.

[0031] When the user U comes into contact with the character C in the virtual space V, in step Sa1 of the subsequent first control process, it is determined that the contact determination data indicates a contact state. When the contact determination data indicates a contact state (Sa1: YES), the movement control unit 43A determines whether the HMD 30 has rotated to the second side in the roll direction (e.g., counterclockwise) by referring to the posture data sequentially supplied from the posture analysis unit 41. Specifically, the movement control unit 43A determines that the HMD 30 has rotated to the second side in the roll direction when the rotation angle θ with respect to the second side in the roll direction exceeds a predetermined threshold θt. When the HMD 30 has rotated to the second side in the roll direction (Sa6: YES), the movement control unit 43A determines that contact with the character C has been released (Sa7). Specifically, the movement control unit 43A changes the contact determination data to a value indicating a non-contact state. On the other hand, when the HMD 30 has not rotated to the second side in the roll direction (Sa6: NO), the movement control unit 43A maintains the contact determination data as a value indicating a contact state.

[0032] It is preferable that when contact with character C is released, the action control unit 43A causes character C to perform an action that responds to the release of contact, but a configuration in which character C does not perform an action that responds to the release of contact is also envisioned.

[0033] As can be understood from the above explanation, the action control unit 43A determines that the user U has made contact with the character C (Sa4) when the HMD 30 has rotated to the first side in the roll direction (Sa2: YES). On the other hand, when the HMD 30 has rotated to the second side in the roll direction (Sa6: YES) while the user U is in contact with the character C (Sa1: YES), the action control unit 43A determines that contact with the character C has been released. With the above configuration, the user U can instruct the occurrence and release of contact with the character C in the virtual space V according to the direction of rotation of the HMD 30 in the roll direction. Since rotations in opposite directions in the roll direction correspond to opposite actions of the occurrence and release of contact with the character C, there is also the advantage that the user U can easily intuitively grasp the occurrence and release of contact with the character C.

[0034] In the first embodiment, for example, the following configuration may be adopted.

[0035] [Variation A1] The movement control unit 43A in the modification A1 causes the character C to perform a reaction action according to the length of time that contact with the character C is maintained in the virtual space V. Specifically, the movement control unit 43A changes the reaction action over time during the period during which the contact state continues (hereinafter referred to as the "contact period"). The contact period is the period from when the HMD 30 rotates to the first side in the roll direction to when it rotates to the second side. Note that the contact period may also be the period from when the contact determination data is set to a value indicating a contact state to when it is changed to a value indicating a non-contact state. The movement control unit 43A changes the reaction action of the character C over time from the start point to the end point of the contact period. Therefore, the reaction action of the character C changes depending on the length of the contact period. According to the above configuration, the reaction action of the character C can be varied in a variety of ways depending on the length of time that the user U contacts the character C.

[0036] [Variation A2] In the modification A2, parameters related to the character C are stored in the storage device 12. Specifically, the likeability of the character C toward the user U is stored as a parameter in the storage device 12. The likeability is a parameter indicating the degree of favorable feelings that the character C has toward the user U. The action control unit 43A changes the likeability as the game progresses, and also changes the likeability of the character C toward the user U depending on the contact with the character C. Specifically, the greater the degree of contact (e.g., number of times or duration) that the user U has with the character C, the greater the likeability of the character C toward the user U is set to. Note that, although the above example has been described as the likeability of the character C toward the user U, the likeability stored in the storage device 12 may also be the likeability of the character C toward the character (player character) used by the user U. The user U may possess multiple player characters in the virtual space V.

[0037] Furthermore, the action control unit 43A changes the reaction action according to the likeability level stored for character C. That is, the type of reaction action that character C performs (Sa4) when user U comes into contact with character C (Sa2: YES, Sa3: YES) changes according to the likeability level of character C. For example, when the number of times or time of contact with character C is insufficient and the likeability level is low, character C will perform a reaction action of rejecting the contact when user U comes into contact with its head. On the other hand, when the number of times or time of contact with character C is sufficient and the likeability level is high, character C will perform a reaction action of accepting the contact when user U comes into contact with its head. With the above configuration, the reaction action of character C can be varied in a variety of ways according to the parameters related to character C.

[0038] Note that the parameters that affect the reaction behavior of character C are not limited to the favorability rating exemplified above. For example, various parameters such as the growth level (level) of character C or the intimacy between user U and character C change depending on the contact of character C by user U, and the reaction behavior of character C is controlled depending on the parameters.

[0039] [Variation A3] The movement control unit 43A in the modification A3 causes the character C to perform a reaction action according to the angle θ by which the HMD 30 is rotated in the roll direction. For example, the rotation angle θ of the HMD 30 corresponds to the virtual pressure of the user U contacting the character C in the virtual space V. For example, a small rotation angle θ indicates a state in which the user U is lightly contacting the character C, and a large rotation angle θ indicates a state in which the user U is strongly pressing the character C. For example, when the rotation angle θ is small, the movement control unit 43A causes the character C to perform a reaction action to accept contact by the user U, and when the rotation angle θ is large, the movement control unit 43A causes the character C to perform a reaction action to reject contact by the user U. With the above configuration, the reaction action of the character C can be varied in a variety of ways according to the rotation angle θ of the HMD 30 (i.e., the virtual pressure of the user U contacting the character C).

[0040] [Variation A4] The display control unit 42A in modification A4 moves the virtual camera E (i.e., the virtual viewpoint in the virtual space V) within the virtual space V in accordance with the rotation of the HMD 30 in the roll direction. Specifically, the display control unit 42A changes the positional relationship between the virtual camera E and the character C in the virtual space V in conjunction with the rotation of the HMD 30. For example, the distance between the virtual camera E and the character C is controlled in accordance with the rotation angle θ of the HMD 30. That is, the virtual camera E moves toward the object in conjunction with the rotation of the HMD 30 in the roll direction.

[0041] For example, when the HMD 30 rotates to a first side in the roll direction, the display control unit 42A moves the virtual camera E closer to the character C in the virtual space V by an amount corresponding to the rotation angle θ (θ>0) of the HMD 30, as shown by an arrow a1 in Fig. 9. On the other hand, when the HMD 30 rotates to a second side in the roll direction, the display control unit 42A moves the virtual camera E away from the character C in the virtual space V by an amount corresponding to the rotation angle θ (θ<0) of the HMD 30, as shown by an arrow a2 in Fig. 9. With the above configuration, the user U can move the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[0042] [Variation A5] The display control unit 42A in Modification A5 changes the imaging range R of the virtual camera E (i.e., the angle of view of the virtual camera E) in response to the rotation of the HMD 30 in the roll direction. That is, the range displayed on the display device 13 in the virtual space V changes in response to the rotation of the HMD 30 in the roll direction. For example, when the HMD 30 rotates to a first side in the roll direction, the display control unit 42A reduces the imaging range R at a ratio corresponding to the rotation angle θ of the HMD 30, as shown by the arrow b1 in FIG. 10 . Therefore, for example, the character C in the virtual space V is zoomed in (enlarged). On the other hand, when the HMD 30 rotates to a second side in the roll direction, the display control unit 42A expands the imaging range R at a ratio corresponding to the rotation angle θ of the HMD 30, as shown by the arrow b2 in FIG. Therefore, for example, the character C in the virtual space V is zoomed out (reduced). According to the above configuration, the user U can change the imaging range R of the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[0043] [Variation A6] In the first embodiment, it is determined that contact with the character C has been released when the HMD 30 has rotated to the second side in the roll direction (Sa6: YES). However, the condition for determining that contact with the character C has been released is not limited to the above example. For example, the movement control unit 43A may determine that contact with the character C has been released when the HMD 30 has returned to its original state from a state in which it has rotated in the roll direction (Sa2: YES). Specifically, in step Sa6 of FIG. 6, the movement control unit 43A determines that contact with the character C has been released when the rotation angle θ of the HMD 30 has changed to the initial angle (e.g., θ = 0) at the time when the rotation for contacting the character C started (Sa7). On the other hand, in step Sa2, the movement control unit 43A determines that the user U has contacted the character C when the rotation angle θ has changed, regardless of the direction of rotation of the HMD 30 (first side / second side). As can be understood from the above explanation, by monitoring the rotation angle θ of the HMD 30, it is possible to determine the occurrence and release of contact with the character C without determining the direction of rotation (first side / second side) of the HMD 30. In other words, in determining contact with the character C, the action control unit 43A does not necessarily need to determine the direction of rotation of the HMD 30.

[0044] [Variation A7] In the first embodiment, when the reference line Q intersects with the character C in the virtual space V (Sa3: YES), the character C is caused to perform a reaction action in response to contact by the user U. However, the condition regarding the positional relationship between the reference line Q and the character C is not limited to the above example. For example, the character C may be caused to perform a reaction action on the condition that the distance between the reference line Q and the character C is below a predetermined threshold. The case where the distance between the reference line Q and the character C is below the predetermined threshold includes not only the case where the reference line Q intersects with the character C, but also the case where the reference line Q is separated from the character C by a distance below the threshold. Furthermore, the character C may be caused to perform a reaction action on the condition that the reference line Q intersects with a specific part of the character C. If the reference line Q intersects with a part other than the specific part of the character C, the movement control unit 43A does not cause the character C to perform a reaction action. As can be understood from the above description, the movement control unit 43A in the first embodiment is comprehensively expressed as an element that controls the movement of the character C according to the positional relationship between the reference line Q and the character C (for example, the relationship where the reference line Q intersects with the character C).

[0045] [B: Second embodiment] A second embodiment of the present invention will be described. In the following examples, elements that have the same functions as those in the first embodiment will be designated by the same reference numerals as those in the first embodiment, and detailed descriptions of these elements will be omitted where appropriate.

[0046] The image display system in the second embodiment has the same configuration as the first embodiment illustrated in Figures 1 and 2. That is, the HMD 30 in the second embodiment includes the display device 13 and the detection device 14 of the terminal device 10 and the wearing attachment 20, and the information processing device 40 includes the control device 11 and the storage device 12 of the terminal device 10.

[0047] 11 is a block diagram illustrating an example of the functional configuration of the terminal device 10 of the second embodiment. As illustrated in FIG. 11, the control device 11 of the second embodiment functions as a posture analysis unit 41, a display control unit 42B, and a motion control unit 43B by executing a program stored in the storage device 12. Note that part or all of the control device 11 may be realized by a dedicated electronic circuit. As in the first embodiment, the posture analysis unit 41 analyzes the detection signal output by the detection device 14 to sequentially generate posture data related to the posture of the HMD 30.

[0048] Similar to the display control unit 42A in the first embodiment, the display control unit 42B causes the display device 13 to display an image within the imaging range R of the virtual space V captured by the virtual camera E and an instruction image G indicating the direction of the reference line Q. As described above in the first embodiment, the reference line Q is a virtual straight line whose direction in the virtual space V changes depending on the orientation of the HMD 30.

[0049] The display control unit 42B of the second embodiment changes the display mode of the instruction image G according to the positional relationship between the reference line Q and a character C (an example of an object) in the virtual space V. Specifically, the display control unit 42B changes the display mode of the instruction image G depending on whether the reference line Q intersects with the character C or not. The display mode refers to the characteristics of an image that can be visually distinguished by the user U. For example, the three attributes of color, that is, hue (tone), saturation, and brightness (gradation), as well as size and image content (e.g., pattern or shape) are also included in the concept of the display mode.

[0050] When the reference line Q does not intersect with the character C, the display control unit 42B causes the display device 13 to display an image G1 as an instruction image G, as illustrated in FIG. 12. The image G1 is a circular or point-like object located in the virtual space V, and is located on the reference line Q within the virtual space V. On the other hand, when the reference line Q intersects with the character C, the display control unit 42B causes the display device 13 to display an image G2 different from the image G1 as an instruction image G, as illustrated in FIG. 13. The image G2 is a planar object that schematically represents the hand of the user U. The display control unit 42B places the image G2 at a point on the surface of the character C in the virtual space V that intersects with the reference line Q (i.e., the contact point P), as illustrated in FIG. 14. That is, the image G2 contacts the contact point P on the surface of the character C in the virtual space V. The contact point P can also be described as the point on the surface of the character C where the image G2 contacts.

[0051] Although the above description illustrates switching between image G1 and image G2, changes in the display mode of instruction image G also include switching between display and non-display. That is, the display control unit 42B may display image G2 as instruction image G when reference line Q intersects with character C, and may not display instruction image G when reference line Q does not intersect with character C.

[0052] As can be understood from the above explanation, in the second embodiment, the display mode of the instruction image G, which indicates the direction of the reference line Q in the virtual space V, changes depending on the positional relationship between the reference line Q and the character C. Therefore, the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V. Furthermore, because the instruction image G (image G2) is placed on the surface of the character C in the virtual space V, there is an advantage that the user U can easily perceive the state in which he or she is in contact with the surface of the character C in the virtual space V. Placing the instruction image G on the surface of the character C also has the advantage that the user U can easily and accurately visually grasp the contact point P.

[0053] 11 controls the movement of the character C in the virtual space V, similar to the movement control unit 43A in the first embodiment. Specifically, the movement control unit 43B causes the character C to perform a movement (i.e., a reaction movement) that reacts to contact by the user U in the virtual space V. Typical examples of the reaction movement are a change in posture, a change in facial expression, or the utterance of a specific line.

[0054] Specifically, the movement control unit 43B causes the character C to perform a movement according to the position of the contact point P where the reference line Q intersects with the character C. For example, when the contact point P is located on the head of the character C, the movement control unit 43B causes the character C to perform a reaction movement to accept contact by the user U, and when the contact point P is located on the torso of the character C, the movement control unit 43B causes the character C to perform a movement to reject contact by the user U. With the above configuration, it is possible to vary the movement of the character C in a variety of ways according to the position of the contact point P.

[0055] 15 is a flowchart illustrating the specific contents of the process (hereinafter referred to as "second control process") executed by the display control unit 42B and the operation control unit 43B according to the second embodiment. The second control process in FIG. 15 is repeatedly executed at a predetermined cycle.

[0056] When the second control process starts, the display control unit 42B determines whether the reference line Q intersects with the character C in the virtual space V (Sb1). If the reference line Q intersects with the character C (Sb1: YES), the display control unit 42B causes the display device 13 to display, as the instruction image G, an image G2 that is placed at the contact point P where the reference line Q intersects with the character C (Sb2). On the other hand, if the reference line Q does not intersect with the character C (Sb1: NO), the display control unit 42B causes the display device 13 to display, as the instruction image G, an image G1 that is placed on the reference line Q (Sb3).

[0057] In a state where image G2 overlapping character C is displayed, action control unit 43B determines whether an action instruction (hereinafter referred to as "action instruction") for character C has been received from user U (Sb4). The action instruction is an instruction to cause an action to occur on character C in virtual space V. In the second embodiment, the action instruction means an instruction for user U to touch character C in virtual space V.

[0058] The movement control unit 43B of the second embodiment accepts a specific change in the posture of the HMD 30 as a movement instruction from the user U. Specifically, the movement control unit 43B determines that a movement instruction has been received when the change in posture of the HMD 30 satisfies a predetermined condition (hereinafter referred to as an "instruction determination condition"). For example, similar to the first embodiment, the movement control unit 43B determines that a movement instruction has been given from the user U when the HMD 30 has rotated in the roll direction. That is, the rotation of the HMD 30 in the roll direction is the instruction determination condition. On the other hand, when the change in posture of the HMD 30 does not satisfy the instruction determination condition, the movement control unit 43B determines that a movement instruction has not been received. Note that, as described above with reference to FIG. 7, a configuration is preferred in which it is determined that the HMD 30 has rotated when the rotation angle θ of the HMD 30 exceeds a threshold value θt.

[0059] When an action instruction is received from the user U (Sb4: YES), the action control unit 43B determines that the user U has contacted the character C in the virtual space V, and causes the character C to perform an action in response to the contact by the user U (Sb5). Specifically, as described above, the action control unit 43B causes the character C to perform an action according to the position of the contact point P of the character C where the reference line Q intersects. As can be understood from the above explanation, in the second embodiment, the user U can contact a desired position of the character C in the virtual space V by giving an action instruction while moving the reference line Q so that it intersects with the desired position of the character C.

[0060] In a configuration in which a change in the posture of the HMD 30 is recognized as an action instruction, there is a problem in that direct operation such as a touch operation on the display surface is not required, making it difficult for the user U to actually feel the sensation of contact. In the second embodiment, the display mode of the instruction image G changes depending on the positional relationship between the reference line Q and the character C (specifically, whether or not the reference line Q intersects with the character C), which has the advantage that the user U can easily actually feel the sensation of contacting the character C in the virtual space V, compared to a configuration in which the instruction image G is displayed in a fixed mode.

[0061] In the second embodiment, for example, the following configuration may be adopted.

[0062] [Variation B1] In the second embodiment, the character C is caused to perform a reaction action according to the position of the contact point P. However, as in the first embodiment, the character C may be caused to perform a reaction action according to the amount of change in the attitude of the HMD 30 (for example, the rotation angle θ in the roll direction). For example, the amount of change in the attitude of the HMD 30 is assumed to be a virtual pressure of the user U contacting the character C in the virtual space V. The action control unit 43B of the modification B1 causes the character C to perform a reaction action to accept contact by the user U when the amount of change in the attitude of the HMD 30 is small, and causes the character C to perform a reaction action to reject contact by the user U when the amount of change in the attitude is large. According to the above aspect, it is possible to cause the character C to perform a variety of reaction actions according to the attitude of the HMD 30.

[0063] [Variation B2] The movement control unit 43B of Modification B2 accepts movement instructions from the user U within a specific period on the time axis (hereinafter referred to as the "instruction acceptance period"). In other words, movement instructions given by the user U during the instruction acceptance period are effectively reflected in the movement of the character C, but movement instructions given by the user U after the instruction acceptance period has elapsed are ignored. The instruction acceptance period is, for example, a period of a predetermined length of time (e.g., several seconds) starting from the point at which the reference line Q begins to intersect with the character C. Note that the start and end points of the instruction acceptance period are not limited to the above examples. For example, the start point of the instruction acceptance period may be the point at which a specific event begins in the game. By limiting the acceptance of movement instructions to the instruction acceptance period as described above, an appropriate sense of tension is imparted to the user U, which ultimately makes it possible to prevent boredom with the game.

[0064] The display control unit 42B changes the display mode of the instruction image G (image G2) over time during the instruction acceptance period. For example, as illustrated in FIG. 16, assume that image G2, which is displayed as the instruction image G when the reference line Q intersects with the character C, includes image G21 and image G22. Image G21 is an image representing the hand of the user U, similar to image G2 in the first embodiment, and image G22 is a circular image placed behind image G21. The display control unit 42B reduces the size of image G22 over time from the start point to the end point of the instruction acceptance period. The size of image G21 does not change. The above configuration has the advantage that the user U can visually grasp the instruction acceptance period (e.g., the remaining time) during which acceptance of action instructions is permitted from the display mode of the instruction image G.

[0065] [Variation B3] The display control unit 42B of the modification B3 changes the display mode of the instruction image G according to the position of the contact point P of the character C in the virtual space V where the reference line Q intersects. Specifically, as illustrated in Fig. 17, the display mode of the instruction image G differs when the contact point P in the virtual space V is at a first position on the surface of the character C and when it is at a second position on the surface of the character C. The first position is, for example, a position where the character C accepts contact by the user U (for example, the torso of the character C), and the second position is, for example, a position where the character C rejects contact by the user U (for example, the head of the character C).

[0066] 17 illustrates an example in which the contact point P is at the first position or the second position, the display control unit 42B may change the display mode of the instruction image G over time in parallel with the change in the position of the contact point P on the character C. That is, when the user U moves the contact point P on the surface of the character C in accordance with the posture of the HMD 30, the display control unit 42B changes the display mode of the instruction image G over time in conjunction with the movement of the contact point P. For example, during the period in which the contact point P moves from the first position to the second position in FIG. 17, the display control unit 42B changes the display mode of the instruction image G continuously or stepwise from the display mode at the first position to the display mode at the second position.

[0067] In modification B3, the display mode of the instruction image G changes depending on the position of the contact point P. Therefore, by visually checking the display mode of the instruction image G, the user U can guess the action of the character C at the time of contact. In other words, the user U can be provided with the fun of gradually moving the contact point P while guessing the action of the character C at the time of contact by checking the display mode of the instruction image G, and giving an action instruction while maintaining the contact point P at a position where the desired action is expected.

[0068] [Variation B4] The display control unit 42B of modification B4 controls the character C so that the line of sight of the character C tracks the reference line Q. For example, the character C is controlled so that the line of sight of the character C is directed toward the contact point P where the reference line Q intersects with the character C. Specifically, as illustrated in FIG. 18, the eyeballs (e.g., pupils) and head of the character C rotate so as to track the reference line Q. Note that only one of the eyeballs and the head of the character C may track the reference line Q. Modification B4 has the advantage that the user U can easily grasp the sense of the influence that the user U is having on the character C.

[0069] [Variation B5] In the second embodiment, the display mode of the instruction image G was changed depending on whether the reference line Q intersects with the character C. However, the conditions for changing the display mode of the instruction image G are not limited to the above example. For example, the display control unit 42B may change the display mode of the instruction image G continuously or stepwise depending on the distance between the reference line Q and the character C. The intersection of the reference line Q with the character C is not an essential condition for changing the display mode of the instruction image G. Furthermore, the display mode of the instruction image G may be changed depending on whether the reference line Q intersects with a specific part of the character C. When the reference line Q intersects with a part other than the specific part of the character C, the instruction image G is displayed in the same display mode as when the reference line Q does not intersect with the character C. As can be understood from the above description, the movement control unit 43B in the second embodiment is comprehensively expressed as an element that changes the display mode of the instruction image G depending on the positional relationship between the reference line Q and the character C.

[0070] [Variation B6] The configuration of the second embodiment, in which the display mode of the instruction image G is changed according to the positional relationship between the reference line Q and the character C, is also applicable to the case where the user U uses the terminal device 10 (for example, a smartphone) while holding it in his / her hand. As in the second embodiment, the imaging range R of the virtual space V displayed on the display device 13 changes according to the attitude of the terminal device 10.

[0071] The terminal device 10 of Modification B6 includes a touch panel that detects contact of the user U with the display surface of the display device 13. In the second embodiment, the position of the contact point P in the virtual space V was changed depending on the posture of the HMD 30. However, in Modification B6, an instruction image G is displayed with the point where the user U contacts the display surface of the display device 13 as the contact point P. For example, a reference line Q passing through the contact point P where the user U contacted is set in the virtual space V. That is, a virtual straight line in the direction indicated by the user U by contacting the display surface of the display device 13 (i.e., by a touch operation) is set in the virtual space V as the reference line Q. The display control unit 42B causes the display device 13 to display an image G2 as the instruction image G when the reference line Q intersects with the character C, and causes the display device 13 to display an image G1 as the instruction image G when the reference line Q does not intersect with the character C. The above configuration also has the advantage that the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V (for example, the state in which the user U is contacting the character C in the virtual space V).

[0072] As can be understood from the example of Modification Example B6, in the second embodiment, the configuration in which the display device 13 is worn on the head of the user U can be omitted. In a configuration that does not assume that the display device 13 is worn on the head of the user U, the display control unit 42B is expressed as an element that causes the display device 13 to display an image of the virtual space V captured by a virtual camera E whose attitude is controlled according to the attitude of the display device 13. Furthermore, in a display device 13 that is used while being held in the hand of the user U, a virtual straight line in a direction indicated by an operation (for example, a touch operation) on the display device 13 is the "reference line."

[0073] [C: Other variations] Specific examples of modifications to the first or second embodiment are given below. Two or more aspects arbitrarily selected from the examples below may be combined as appropriate within the scope of not contradicting each other.

[0074] [Variation C1] The reference line Q is not limited to the optical axis of the virtual camera E exemplified in each of the above-described embodiments. For example, a straight line perpendicular to the display surface of the display device 13 is used as the reference line Q. Alternatively, a straight line forming a predetermined angle with respect to the optical axis of the virtual camera E or a straight line perpendicular to the display surface may be used as the reference line Q. Note that, in an HMD 30 equipped with an eye tracking (gaze measurement) function that estimates the direction of the gaze of the user U, the gaze estimated by this function may be used as the reference line Q. As can be understood from the above examples, the reference line Q is comprehensively expressed as a virtual straight line set in the virtual space V.

[0075] [Variation C2] In each of the above-described embodiments, a character C is exemplified as an object placed in the virtual space V, but the objects that the user U comes into contact with in the virtual space V are not limited to the character C. For example, inanimate elements such as buildings and natural objects in the virtual space V are also included in the concept of objects. For a living object (character C), an "action" refers to, for example, the behavior, action, appearance, or attitude of the object. Furthermore, for an inanimate object, an "action" refers to, for example, a change in the form of the element. For example, the action of opening a door in a building, or the action of a natural object such as a rock deforming or moving are examples of object actions.

[0076] [Variation C3] In the above-described embodiments, when the HMD 30 rotates in the roll direction, it is determined that the user U has contacted the character C in the virtual space V, causing the character C to perform a reaction action. However, the trigger for the reaction action (i.e., the action instruction) is not limited to the above examples. For example, the rotation of the HMD 30 in the pitch direction or yaw direction may be considered as contact by the user U, causing the character C to perform a reaction action. Furthermore, for example, the character C may be caused to perform a reaction action when the reference line Q intersects within a specific range of the character C (i.e., when the user U gazes within that range for a long period of time) for a predetermined period of time. The character C may be caused to perform a reaction action when the user U operates an operating device (not shown) connected to the HMD 30. As can be understood from the above examples, the configuration in which the reaction action of the character C is triggered by a change in the posture of the HMD 30 (particularly rotation in the roll direction) may be omitted.

[0077] [Variation C4] In the above-described embodiments, rotation of the HMD 30 in the roll direction is accepted as a motion instruction (an instruction to contact the character C), but the change in the posture of the HMD 30 that is determined as a motion instruction is not limited to rotation in the roll direction. For example, rotation of the HMD 30 in the pitch direction or yaw direction may be accepted as a motion instruction.

[0078] [Variation C5] The modified examples (modified examples A1 to A7) exemplified for the first embodiment are also applicable to the second embodiment. Moreover, the modified examples (modified examples B1 to B6) exemplified for the second embodiment are also applicable to the first embodiment.

[0079] [Variation C6] In the above-described embodiments, the display device 13 and detection device 14 of the HMD 30 and the information processing device 40 that controls the HMD 30 are implemented as a single terminal device 10, but the HMD 30 and the information processing device 40 may be implemented as separate devices, as in the image display system 1B of FIG. 19. The HMD 30 and the information processing device 40 can communicate with each other via wire or wirelessly. Any method of communication between the HMD 30 and the information processing device 40 may be used, but short-range wireless communication such as Bluetooth (registered trademark) is preferred.

[0080] As in the first embodiment, the HMD 30 includes a display device 13, a detection device 14, and a wearing attachment 20, and is worn on the head of the user U. The information processing device 40 is a control device that communicates with the HMD 30 to display various images on the HMD 30, and includes a control device 11 and a storage device 12. For example, various information terminals such as a mobile phone, a smartphone, a tablet terminal, a personal computer, or a home game console are used as the information processing device 40. It does not matter whether the information processing device 40 is portable or stationary. As in the first or second embodiment, the control device 11 functions as a posture analysis unit 41, a display control unit 42 (42A, 42B), and a motion control unit 43 (43A, 43B) by executing a program stored in the storage device 12. Therefore, the configuration of FIG. 19 also achieves the same effects as the first or second embodiment.

[0081] [Variation C7] A preferred embodiment of the present invention is realized by cooperation between a computer (specifically, the control device 11) and a program, as exemplified in the above-described embodiments. The program according to each of the above-described embodiments is provided in a form stored on a computer-readable recording medium and installed on the computer. The recording medium is, for example, a non-transitory recording medium, and a good example is an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium, such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except for a transient, propagating signal, and does not exclude volatile recording media. The program can also be provided to the computer in the form of distribution via a communication network.

[0082] [D: Notes] From the above description, for example, the following preferred embodiments of the present invention can be understood. Note that, in order to facilitate understanding of each embodiment, reference numerals in the drawings are written in parentheses for convenience, but the present invention is not intended to be limited to the illustrated embodiments.

[0083] [Aspect 1] A program according to a preferred embodiment (embodiment 1) of the present invention is a program that causes a computer (11) to function as a display control unit (42B) that causes a display device (13) of the head-mounted display (30) to display a stereoscopic image utilizing binocular parallax, the stereoscopic image being an image of a virtual space (V) captured by a virtual camera (E) whose direction is controlled according to the direction of a head-mounted display (30), and the display control unit (42B) displays an instruction image (G) in the direction of a reference line (Q) whose direction in the virtual space (V) changes according to the direction of the head-mounted display (30), and changes the display mode of the instruction image (G) according to the positional relationship between the reference line (Q) and an object (C) in the virtual space (V). In the above configuration, the display mode of the instruction image (G) indicating the direction of the reference line (Q) in the virtual space (V) changes depending on the positional relationship between the reference line (Q) and the object (C), which has the advantage that the user (U) can easily understand the positional relationship between the reference line (Q) and the object (C) in the virtual space (V).

[0084] The "reference line" is a virtual straight line set in the virtual space (V). For example, the optical axis of the virtual camera (E), the center line of the display screen of the display device (13), or a straight line forming a predetermined angle with these straight lines are suitable examples of the reference line (Q). In a configuration in which an eye tracking function for estimating the gaze of the user (U) can be used, in addition to the straight lines exemplified above, the gaze of the user (U) may also be used as the reference line (Q).

[0085] An "object" is a virtual object placed in a virtual space (V). A typical example of an object (C) is a character (e.g., a human, animal, or monster), but the concept of an object can also include inanimate elements such as buildings or natural objects in the virtual space (V).

[0086] "Display mode" refers to the visually distinguishable characteristics of an image. For example, the three attributes of color—hue (tone), saturation, and brightness (gradation)—as well as size and image content (e.g., pattern or shape) are included in the concept of display mode. Changes in display mode also include switching between display and non-display.

[0087] [Aspect 2] In a preferred example (Aspect 2) of Aspect 1, the display control unit (42B) displays the instruction image (G) at a point (P) on the surface of the object (C) that intersects with the reference line (Q). According to the above aspect, the instruction image (G) is placed on the surface of the object (C) in the virtual space (V), which has the advantage that the user (U) can easily perceive the state of contact with the surface of the object (C) from the instruction image (G).

[0088] [Aspect 3] A program according to a preferred example (aspect 3) of aspect 1 or aspect 2 causes the computer 11 to function as a motion control unit 43B that controls the motion of the object C in accordance with the positional relationship between the reference line Q and the object C. According to the above aspect, it is possible to cause the object C to perform a variety of motions that reflect instructions from the user U.

[0089] The "movement of an object" refers to, for example, the behavior, action, appearance, or attitude of a character that is an example of an object (C). With respect to an inanimate element, the "movement of an object" refers to, for example, a change in the form of the element (for example, a door or window of a building opening, or a natural object such as a rock deforming or moving).

[0090] [Aspect 4] In a preferred example (Aspect 4) of Aspect 3, when the action control unit (43B) receives an action instruction from the user (U), it controls the action of the object (C) in accordance with the positional relationship between the reference line (Q) and the object (C). According to the above aspect, the user (U) can cause the object (C) to perform an action corresponding to the desired position by giving an action instruction after moving the reference line (Q) to a desired position.

[0091] An "action instruction" is an instruction to cause an action (e.g., contact) to occur on an object (C) in the virtual space (V). It can also be described as an instruction to confirm the provisional direction represented by the instruction image (G) as the direction desired by the user (U). For example, it is determined that an action instruction has been accepted when a specific change (e.g., rotation in the roll direction) occurs in the direction of the head-mounted display (30). It may also be determined that an action instruction has been accepted when, for example, the reference line (Q) remains stationary in the virtual space (V) for a predetermined period of time (e.g., 2 seconds).

[0092] [Aspect 5] In a preferred example (Aspect 5) of Aspect 4, the action control unit (43B) determines that the action instruction has been received when a change in the attitude of the head-mounted display (30) satisfies a predetermined condition, and causes the object (C) to perform an action corresponding to the amount of change in attitude, and the display control unit (42B) changes the display mode of the instruction image (G) according to the change in attitude of the head-mounted display (30) before receiving the action instruction. According to the above aspect, it is possible to cause the object (C) to perform a variety of actions according to the attitude of the head-mounted display (30). Furthermore, because the display mode of the instruction image (G) changes according to the attitude of the head-mounted display (30), there is an advantage in that the user (U) can easily grasp the sensation of the user (U) influencing the object (C) (e.g., the sensation of contact). The "predetermined condition" refers to a condition for a change in attitude that is determined to be an action instruction from the user (U), such as a rotation in the roll direction in this embodiment.

[0093] [Aspect 6] In a preferred example (aspect 6) of aspect 4 or aspect 5, the display control unit (42B) changes the display mode of the instruction image (G) over time during an instruction acceptance period during which acceptance of the operation instruction from the user (U) is permitted. According to the above aspect, the user (U) can visually grasp the instruction acceptance period during which acceptance of the operation instruction is permitted (e.g., the remaining time of the instruction acceptance period) from the display mode of the instruction image (G). Note that the "instruction acceptance period" is, for example, the remaining time during which one contact with a character is permitted, or the remaining time of an operation mode during which contact with a character is permitted.

[0094] [Aspect 7] In a preferred example (Aspect 7) of any of Aspects 3 to 6, the display control unit (42B) changes the display mode of the instruction image (G) according to the position of the point (P) of the object (C) where the reference line (Q) intersects. According to the above aspects, the display mode of the instruction image (G) changes according to the position of the point (P) of the object (C) where the reference line (Q) intersects, so the user (U) can infer the movement of the object (C) from the display mode of the instruction image (G).

[0095] [Aspect 8] In a preferred example of aspect 7 (aspect 8), the display control unit (42B) changes the display mode of the indication image (G) over time in parallel with a change in the position of the point (P) of the object (C) where the reference line (Q) intersects. According to the above aspect, the display mode of the indication image (G) changes over time in parallel with a change in the position of the point (P) of the object (C) where the reference line (Q) intersects, providing the user (U) with the enjoyment of gradually changing the reference line (Q) while checking the display mode of the indication image (G). Note that "changing the display mode over time" does not mean that the display mode changes only in a binary manner at a specific point in time, but rather that the display mode changes gradually over time. Note that it does not matter whether the display mode changes continuously or in stages.

[0096] [Aspect 9] In a preferred example (Aspect 9) of any of Aspects 1 to 8, the operation control unit (43B) controls the object (C) so that the line of sight of the object (C) tracks the reference line (Q). According to the above aspects, the user (U) can easily grasp the sensation of the user (U) influencing the object (C) (e.g., the sensation of contact). Note that "the line of sight of the object tracks the reference line" includes not only the case where the eyeballs (e.g., pupils) of the object (C) rotate to track the reference line (Q), but also the case where the head of the object (C) rotates in accordance with the reference line (Q).

[0097] [Aspect 10] An image display system (1A, 1B) according to a preferred embodiment (embodiment 10) of the present invention is an image display system (1A, 1B) comprising a head-mounted display (30) and an information processing device (40), wherein the information processing device (40) comprises a display control unit (42B) that causes a display device (13) of the head-mounted display (30) to display a stereoscopic image utilizing binocular parallax, the stereoscopic image being an image captured of a virtual space (V) by a virtual camera (E) whose direction is controlled according to the direction of the head-mounted display (30), and the display control unit (42B) displays an instruction image (G) in the direction of a reference line (Q) whose direction in the virtual space (V) changes according to the direction of the head-mounted display (30), and changes the display mode of the instruction image (G) according to the positional relationship between the reference line (Q) and an object (C) in the virtual space (V). In the above configuration, the display mode of the instruction image (G) indicating the direction of the reference line (Q) in the virtual space (V) changes depending on the positional relationship between the reference line (Q) and the object (C), which has the advantage that the user (U) can easily grasp the positional relationship between the reference line (Q) and the object (C) in the virtual space (V). Note that the head-mounted display (30) and the information processing device (40) may be integrated or separate. [Explanation of symbols]

[0098] 1A, 1B...image display system, 10...terminal device, 11...control device, 12...storage device, 13...display device, 14...detection device, 20...wearing device, 30...HMD, 40...information processing device, 41...posture analysis unit, 42A, 42B...display control unit, 43A, 43B...motion control unit, V...virtual space, E...virtual camera, C...character, R...imaging range, Q...reference line, G...instruction image, P...contact point.

Claims

1. a display control unit that causes an image of a virtual space captured by a virtual camera whose direction is controlled according to a direction of a head mounted display to be displayed on the head mounted display; and a motion control unit that causes a character to perform a motion according to a position at which a reference line, the direction of which in the virtual space changes according to the direction of the head mounted display, intersects with the character moving in the virtual space; A program that makes a computer function as a

2. The action corresponding to the position where the reference line intersects with the moving character is one of a change in posture, a change in facial expression, and the pronunciation of a specific line. The program of claim 1.

3. An image display system including a head mounted display and an information processing device, The information processing device includes: a display control unit that causes the head mounted display to display an image of a virtual space captured by a virtual camera whose direction is controlled according to a direction of the head mounted display; and a motion control unit that causes the character to perform a motion according to a position at which a reference line, the direction of which in the virtual space changes according to the direction of the head mounted display, intersects with the character moving in the virtual space. Image display system.

Images