Information processing device, information processing system, and information processing method

Abstract

This information processing device displays content in a virtual space. The information processing device is provided with a control unit. Here, the control unit acquires area information regarding the area where a user acts in relation to the content, and predicted action information regarding an action predicted on the basis of the area information. Furthermore, upon determining that the action predicted on the basis of the predicted action information cannot be performed within the area, the control unit determines a preparatory movement related to a movement for changing the user's position and / or orientation. The control unit then notifies the user of the content of the determined preparatory movement.

Description

Information processing device, information processing system, and information processing method

[0001] The present invention relates to an information processing device, an information processing system, and an information processing method that enable playing content related to a virtual space in a space-saving manner.

[0002] In recent years, virtual reality (VR) technology, which creates virtual objects that resemble reality and allows users to experience the sensation of being in those objects, has been widely used, and information processing devices that allow users to easily handle virtual objects have become widespread. One example is a head-mounted display (HMD), which is worn on the head. A user wearing an HMD can sit, stand, and move around to play content such as a game in a virtual space and fully enjoy the content.

[0003] However, when playing content, whether seated, standing, or room-scale, it is not always possible to secure the recommended play area for the content due to space constraints within the home. If the recommended play area cannot be secured, the content available to users or movement during play may be restricted, making it difficult to fully enjoy the desired content. For example, there have been cases where users have no choice but to choose a stationary play style, even though they would normally like to move around a lot while playing.

[0004] Here, as an example of a method for identifying the play area state using an information processing device, Patent Document 1 states that it "includes a position information acquisition unit that acquires position information of the head-mounted display, an information processing unit that performs information processing using the position information, an output data generation unit that generates and outputs data for an image to be displayed as a result of the information processing, and a user guide generation unit that uses the position information to generate and output data for a user guide image that represents the user's position information in real space."

[0005] Japanese Patent Application Laid-Open No. 2017-130793

[0006] Patent Literature 1 discloses that by acquiring a user's position information from a captured image and presenting information related to the user's surroundings in the real world as a user guide, the user can recognize their location and direction of movement in the real world even when the outside world is invisible. However, the technology described in Patent Literature 1 has a problem in that it only presents a guide for determining the current location, and does not consider or suggest any guide for enabling the user to perform play operations within the recommended play area of ​​the content. Furthermore, the technology described in Patent Literature 1 has a problem in that the guide is displayed only when the user performs an operation for displaying the guide or when there is a danger, such as an obstacle, in the play area, which is insufficient as a guide for performing user play operations within the recommended play area.

[0007] Taking the above-mentioned problems into consideration, the present invention provides an information processing device, an information processing system, and an information processing method that enable the user to play content recommended in the content in the same manner as when the recommended play area is secured, even if the recommended play area for the content cannot be secured, thereby expanding the range of content selection and allowing the user to enjoy the content to the fullest without restricting their movements.

[0008] Among the inventions disclosed in this application, the outline of representative inventions will be briefly explained as follows.

[0009] (1) An information processing device that displays content in a virtual space, comprising a control unit, wherein the control unit acquires area information regarding an area in which a user will behave in relation to the content and predicted behavior information regarding behavior predicted based on the area information, and when it determines that the behavior predicted based on the predicted behavior information cannot be performed within the area, determines a preparatory action regarding an action to change the user's position and / or orientation, and notifies the user of the determined preparatory action.

[0010] (2) An information processing device according to (1), characterized in that the control unit temporarily suspends the reflection of the user's preparatory movement in the content when it detects the start of the preparatory movement by the user after notifying the preparatory movement, and resumes the reflection of the user's play movement in the content after it detects the end of the user's preparatory movement.

[0011] (3) The information processing device according to (2), wherein the control unit, when not all predicted actions have been performed, repeats the process of determining preparatory movements and notifying the content of the determined preparatory movements until all predicted actions have been performed.

[0012] (4) An information processing device according to (1), wherein the control unit, if the predicted action is being performed within the area after the notification and without detecting the start of the user's preparatory action, issues a prompting notification to the user to perform a preparatory action, and if the predicted action is no longer being performed within the play area, issues a warning and stops the execution of the content.

[0013] (5) The information processing device described in (1) is characterized in that, when there is another information processing device whose area overlaps with the area of ​​the information processing device and the users of the two information processing devices collide due to the predicted actions of the two information processing devices, the information processing device determines a preparatory action to avoid the collision and notifies the user of the determined preparatory action.

[0014] (6) An information processing system of an information processing device that displays content in a virtual space, wherein the information processing device acquires area information regarding an area in which a user will behave in relation to the content, acquires predicted behavior information regarding behavior predicted based on the area information, and, when it determines that the behavior predicted based on the predicted behavior information cannot be performed within the area, determines a preparatory movement regarding an action to change the position and / or orientation of the user, and notifies the user of the content of the determined preparatory movement.

[0015] (7) An information processing method performed using an information processing device that displays content in a virtual space, comprising: a step in which the information processing device acquires area information regarding an area in which a user will behave in relation to the content; a step in which the information processing device acquires predicted behavior information regarding behavior predicted based on the area information; a step in which the information processing device determines, when it is determined that the behavior predicted based on the predicted behavior information cannot be performed within the area, a step in which the information processing device determines a preparatory movement regarding an action to change the position and / or orientation of the user; and a step in which the information processing device notifies the user of the content of the determined preparatory movement.

[0016] By using the technology of the present invention, it is possible to realize an information processing device, an information processing system, and an information processing method that enable a user to play content recommended in the content in the same manner as when the recommended play area is secured, even if the recommended play area for the content cannot be secured, thereby widening the range of content selection and allowing the user to enjoy the content to the fullest without restricting their movements. Furthermore, problems, configurations, and effects other than those described above will become clear from the description of the embodiments below.

[0017] FIG. 1B is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1C is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1D is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1E is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1F is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1G is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1H is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1H is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1H is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1H is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; FIG. 1H is a diagram illustrating a display screen of the information processing device and information processing system according to the embodiment of the present invention; 5B is a diagram illustrating a display screen of the information processing device and information processing system according to the present embodiment; FIG. 5C is a diagram illustrating a display screen of the information processing device and information processing system according to the present embodiment; FIG. 5D is a diagram illustrating a display screen of the information processing device and information processing system according to the present embodiment; FIG. 5E is a diagram illustrating a display screen of the information processing device and information processing system according to the present embodiment;FIG. 1 is an example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a standing state. FIG. 2 is another example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a standing state. FIG. 3 is another example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a standing state. FIG. 4 is an example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a seated state. FIG. 5 is another example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a seated state. FIG. 6 is another example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a seated state. FIG. 7 is an example of a diagram schematically illustrating the appearance of an example of an information processing device and an information processing system according to the present embodiment in a case where multiple users coexist. FIG. 10 is another example of a diagram for schematically explaining the appearance of an example in which a plurality of users coexist in an information processing device and an information processing system according to the present embodiment. FIG. 11 is an example of a flowchart for explaining the basic operation of an information processing device and an information processing system according to the present embodiment. FIG. 12 is a diagram for explaining an example of predicted behavior in an information processing device and an information processing system according to the present embodiment. FIG. 13 is a diagram for explaining another example of predicted behavior in an information processing device and an information processing system according to the present embodiment. FIG. 14 is a block diagram for explaining an example of the configuration of an information processing device and an information processing system according to the present embodiment.

[0018] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. As a specific example of an information processing device according to this embodiment, an HMD that is worn on a user's head and displays content played by the user in a virtual space will be described. FIGS. 1A and 1B are diagrams showing the exterior of an example of an information processing device and an information processing system according to this embodiment. Also, FIGS. 2A, 2B, and 2C are diagrams showing the display screen of the HMD in the example embodiment shown in FIGS. 1A and 1B.

[0019] 1A and 1B , an HMD 100 is worn on the head of a user 10. To avoid clutter, the HMD 100 is depicted enlarged and at a distance from the user 10. The HMD 100 includes a camera 101 that captures the external world around the user, a distance sensor 102 that measures the distance and angle to an object and captures the object's shape in three dimensions, a positioning sensor 103 that detects the user's current location, an attitude sensor 104 that captures the user's attitude, such as direction and tilt, a microphone 106 that collects speech from the user 10, a speaker 107 that produces sound based on audio data, and a vibrator 108 that generates vibrations. The HMD 100 also wirelessly communicates with an external information server 112 via an external network 111. The information server 112 stores content information indicating the content of games and other such content, and the HMD 100 transmits and receives content information to and from the information server 112.

[0020] Before the user 10 plays content, the HMD 100 detects the spatial environmental conditions around the user 10 using the camera 101 and the distance measurement sensor 102, and acquires user environment information indicating the environmental conditions around the user. As shown in FIGS. 1A and 1B , if the camera 101 and the distance measurement sensor 102 are positioned facing forward of the user 10, the user 10 turns around and looks around, and the camera 101 and the distance measurement sensor 102 detect the environmental conditions around the entire environment. The camera 101 and the distance measurement sensor 102 may be located at a top position 115 of the HMD 100 that provides a panoramic view. Based on the user environment information acquired by the HMD 100, the user 10 sets a user play area 120 in which the user 10 can move and play. The HMD 100 may also set the user play area 120 based on the user environment information. The play style of the content may be, for example, a room-scale play style. Therefore, the HMD 100 is configured with a user play area acquisition means that acquires a user-defined user play area in which the user can move during play.

[0021] After the user 10 selects content to play, the HMD 100 acquires content information for the selected content and starts playing the content. The HMD 100 acquires predicted behavior information indicating predicted behavior (predicted behavior) developed in the content from the information server 112. Thus, the HMD 100 is configured with a content predicted behavior information acquisition means that acquires predicted behavior information indicating predicted behavior of the content from the content server while the content is being played. The HMD 100 also acquires, for example, the current position of the user 10 using the positioning sensor 103 and the user's orientation and other posture information using the posture sensor 104 as user position / posture information.

[0022] As illustrated in FIG. 1A , the content information acquired by the HMD 100 forms a virtual space 130 in which a building 131 and first and second towers 132 and 133 are arranged. Assume that a scenario is progressing in the virtual space 130 in which a user character (avatar) 136, which is an avatar of the user, moves straight toward the building 131. At this time, the virtual distance (arrow 135) between the position of the user character 136 in the virtual space 130 and the building 131 is a distance Va in the virtual space. Distance Va is, for example, the distance of 10 steps of the user character in the virtual space 130. This distance is only within the virtual space 130 and does not necessarily coincide with a distance Ra (arrow 137) in the real space 141. However, in content in which the user's actions in the real space are reflected in the actions of the user character, which is the user's avatar in the virtual space, a scenario is achieved in which the user character also moves 10 steps forward in the virtual space and arrives at the destination, the building 131, by the user taking 10 steps in the real space.

[0023] Here, to achieve this scenario, the user 10 in the real world needs to take 10 steps forward from position 134. However, in the example of Figure 1A, the user 10 wearing the HMD 100 cannot move forward a distance Ra of 10 steps from position 134 within the play area 120 in the real world. In other words, the user cannot perform the predicted behavior within the user play area 120 to achieve the scenario of reaching the building 131 in the virtual space.

[0024] In such a case, conventionally, in order to achieve the scenario, the user would move forward the number of steps that can be taken in the real space 141 (for example, five steps). As a result, the user's character would also move forward five steps in the virtual space and approach the building 131. After that, the user would no longer be able to move forward in the real space 141, so it was necessary to switch from operation system 1 (room-scale play style) in which the user's own movements are linked to the movements of the user character, which is the user's alter ego in the virtual space, to another operation system 2 (standing play style) in which the user moves the user character in the virtual space using a controller held in the user's hand, and then, while remaining standing, move the user character forward another five steps by operating the controller to achieve the scenario of reaching the building 131 in the virtual space.

[0025] In response to this, when the HMD 100 recognizes that the predicted action for achieving the scenario in the virtual space cannot be executed in the user play area in the real world without changing posture or position, the HMD 100 determines a preparatory action to change the user's position or orientation in the real space so that the predicted action for achieving the scenario in the virtual space can be executed in the user play area in the real world, and provides a guidance notification to the user 10 to perform the determined preparatory action in the real world. For example, the HMD 100 determines a preparatory action to rotate the orientation of the user 10 in the real world 180 degrees to the right, as indicated by arrow 140, and provides a guidance notification to the user 10 by displaying on the screen or by voice from the speaker 107. Note that, for example, the HMD 100 provides a guidance notification when the user 10 walking within the play area approaches the boundary of the play area.

[0026] Furthermore, by the user 10 performing the preparatory movement in the real world as notified by the guide, the user 10 is able to secure a distance Ra of 10 steps from the position 134 in the play area in the real world, as shown in FIG. 1B . The user 10 can then perform a movement to move forward 10 steps (i.e., the distance Ra) from the position 134 in the real world. That is, the user 10 walks from the position 134 toward the boundary of the play area, performs a preparatory movement when approaching the boundary of the play area, and continues walking, thereby performing a movement to move forward 10 steps from the position 134. The user character 136 in the virtual space also moves forward 10 steps in conjunction with this user movement in the real world, thereby achieving a scenario in which the user character 136 arrives at the destination building 131. That is, the scenario in the virtual space can be progressed solely by movements in the real world that are linked to the character's movements, without the need to switch control systems or play styles as in the past. This allows the user to enjoy the content with the feeling that they are moving within the virtual space.

[0027] Regarding the screen display of the HMD 100, after the preparatory movement is determined, the HMD 100 displays a message guide 201 saying "Preparatory movement guide: Rotate your orientation 180 degrees" on the virtual space display screen 200, along with a building 131, first and second towers 132 and 133, and feet 203 of a user character 136, which is an alter ego of the user 10, in the virtual space 130, as shown in FIG. 2A . Note that the play area 120 indicated by the dashed dotted line in FIG. 2A indicates the user's play area and is not displayed on the HMD 100, but is shown to indicate that the user 10 in the real world is located near the boundary of the play area in the situation shown in FIG. 2A . While the user 10, who has received the message guide 201, is performing a preparatory movement in the real world, the HMD 100 does not reflect the user's movement in the movement of the user character 136, which is an alter ego of the user 10, in the virtual space 130.

[0028] Then, when the HMD 100 detects the completion of the preparatory movement by the user 10 in the real world, as shown in Fig. 2B , the display states of the building 131, the first and second towers 132 and 133, and the feet 203 of the user character 136 remain unchanged, and a message guide 202 saying "Preparatory movement guide: 180 degree rotation of orientation detected" is displayed on the display screen 200. Here, the play area 120 indicated by the dashed dotted line in Fig. 2B is not displayed on the HMD 100 as in Fig. 2A , but is illustrated to indicate that the change in the orientation of the user in the real world due to the preparatory movement has secured a distance within the play area to achieve the predicted action. That is, even if the user 10 turns 180 degrees in the real world, the user character 136 in the virtual space does not change its direction, and therefore the content display screen 200 seen by the user 10 does not change before or after the preparatory movement, and remains as an image of the user facing the building 131, the first and second towers 132, 133 at a distance Va in the virtual space 130. In other words, even if the user 10 makes a preparatory movement in the real world, it does not affect the scenario in the virtual space.

[0029] After detecting the completion of the preparatory movement by the user 10 in the real world, the HMD 100 resumes the process of reflecting the user 10's movement in the real world on the movement of the user character 136, which is the user's alter ego in the virtual space. When the user 10 takes 10 steps forward in the real world (forward a distance Ra), the user character 136 in the virtual space also moves forward a distance Va in conjunction with the movement, achieving a scenario in which the user character 136 reaches the building 131. The content display at this time shows a scene with the building 131 right in front of the user, as shown in FIG. 2C . The play area 120 indicated by the dashed dotted line in FIG. 2C is not displayed on the HMD 100, as in FIGS. 2A and 2B . However, it is shown to indicate that the position of the user character 136 in the virtual space changes in conjunction with the user 10's forward movement in the real world, and that the position in the displayed image and the real world also changes accordingly. The 180-degree rotation guide is not limited to a clockwise direction as indicated by the arrow 140, but may be a counterclockwise direction. A preferred rotation direction may be set for each user, and guidance may be provided according to that setting. If a user is good at turning left, guidance may be provided with a preference for turning left when turning 180 degrees. Note that Figures 2A, 2B, and 2C each include a mark 210 indicating the direction. Mark 210 makes it clear that although the user has turned 180 degrees in the real world, the direction has not changed in the virtual space.

[0030] Therefore, even if the predicted behavior developed in the content cannot be executed within the user play area as it is, the user can play the content within the user play area by receiving a guide notification of the preparatory movements and executing the preparatory movements themselves, allowing the user to fully enjoy the content. This in turn makes it possible to select content that requires large movements when playing, thereby broadening the range of content choices.

[0031] Next, other examples of the information processing device and information processing system according to this embodiment will be described using FIGS. 3A, 3B, 4A, 4B, and 4C. FIGS. 3A and 3B are schematic diagrams illustrating the exterior appearance of the information processing device and information processing system according to this embodiment. FIGS. 4A, 4B, and 4C are diagrams illustrating the display screen of the HMD in the embodiment shown in FIGS. 3A and 3B. In FIGS. 3A, 3B, 4A, 4B, and 4C, parts with the same reference numerals as those shown in FIGS. 1A, 1B, 2A, 2B, and 2C perform the same operations as those already described, and therefore detailed description thereof will be omitted. In FIG. 3A, as in FIG. 1A, a scenario is assumed to be progressing in a virtual space 130 in which a user character 136, representing the user's alter ego, moves straight toward a building 131. In this case, the virtual distance (arrow 135) between the position of the user character 136 in the virtual space 130 and the building 131 is set to be a distance Va in the virtual space. Distance Va is, for example, the distance of 10 steps taken by the user character in the virtual space 130. This distance is only a distance within the virtual space 130 and does not necessarily coincide with distance Ra (arrow 137) in the real space 141. However, in content in which the user's actions in the real space are reflected in the actions of the user character, which is the user's alter ego in the virtual space 130, a scenario is achieved in which the user's character also moves 10 steps forward in the virtual space 130 and arrives at the destination, building 131.

[0032] Here, to achieve this scenario, the user 10 in the real world needs to take 10 steps forward from position 134. However, in the example of Figure 3A, the user 10 wearing the HMD 100 cannot move forward a distance Ra equivalent to 10 steps from position 134 within the play area 120 in the real world. In other words, the user cannot perform the predicted behavior within the user play area to achieve the scenario of reaching the building 131 in the virtual space.

[0033] In response to this, when the HMD 100 recognizes that the predicted action for achieving the scenario in the virtual space cannot be executed in the user play area in the real world without changing the posture or position, the HMD 100 determines a preparatory action of moving the position 134 of the user 10 in the real world back by a distance L to a position 301, as indicated by arrow 302, and notifies the user 10 of the determined preparatory action by displaying it on the screen or by voice via the speaker 107. Furthermore, when the user 10 executes the preparatory action for which the user 10 has been notified in the real world, the user 10 is able to secure a distance Ra of 10 steps from the position 301 in the play area in the real world, as shown in FIG. 3B . Thus, the user 10 can take an action of moving forward 10 steps (i.e., the distance Ra) from the position 301 in the real world, and the user character 136 in the virtual space also moves forward 10 steps in conjunction with this user action in the real world, thereby achieving the scenario of arriving at the destination building 131. In other words, the scenario in the virtual space can be progressed solely by actions in the real world linked to the character's movements, without the need to switch between control systems or play styles as in the past, so the user 10 can enjoy the content as if they were moving within the virtual space themselves.

[0034] Regarding the screen display of the HMD 100, after the preparatory movement is determined, the HMD 100 displays a message guide 401 saying "Preparatory movement guide: Please step back a distance L" on the display screen 200 along with the building 131, the first and second towers 132 and 133, and the feet 203 of the user character 136, which is the alter ego of the user 10, in the virtual space 130, as shown in FIG. 4A . Note that the play area 120 indicated by the dashed dotted line in FIG. 4A indicates the user play area and is not displayed on the HMD 100, but is shown to indicate that the user 10 in the real world is located within the play area in the situation of FIG. 4A . While the user 10, who has received the display of the message guide 401, is performing a preparatory movement in the real world, the HMD 100 does not reflect the user's movement in the movement of the user character 136, which is the alter ego of the user 10, in the virtual space 130. Then, when the HMD 100 detects the completion of the preparatory movement by the user 10 in the real world, as shown in Fig. 4B , the display states of the building 131, the first and second towers 132 and 133, and the feet 203 of the user character 136 remain unchanged, and a message guide 402 stating "Preparatory movement guide: Completion of backward movement of distance L has been detected" is displayed on the display screen 200. Here, the play area 120 indicated by the dashed dotted line in Fig. 4B is not displayed on the HMD 100 as in Fig. 4A , but is illustrated to indicate that the change in the user's orientation in the real world due to the preparatory movement has secured a distance within the play area to achieve the predicted action. That is, even if the user 10 performs a preparatory movement in the real world, such as retreating without changing his or her orientation, the distance and orientation between the user character 136 and the building 131, and the first and second towers 132 and 133 in the virtual space do not change, and therefore the content display screen 200 seen by the user 10 does not change before or after the preparatory movement, and remains as an image of the user 10 facing the building 131, the first and second towers 132 and 133 directly at a distance Va in the virtual space 130. In other words, performing a preparatory movement in the real world does not affect the scenario in the virtual space.After detecting the completion of the preparatory movement by the user 10 in the real world, the HMD 100 resumes the process of reflecting the user 10's movement in the real world on the movement of the user character 136, which is the user's alter ego in the virtual space. When the user 10 takes 10 steps forward in the real world (forward a distance Ra), the user character 136 in the virtual space also moves forward a distance Va in conjunction with the movement, achieving a scenario in which the user character 136 reaches the building 131. The content display at this time shows a scene with the building 131 right in front of the user, as shown in FIG. 4C . The play area 120 indicated by the dashed dotted line in FIG. 4C is not displayed on the HMD 100, as in FIGS. 4A and 4B . However, it is shown to indicate that the position of the user character 136 in the virtual space changes in conjunction with the user 10's forward movement in the real world, and that the position in the displayed image and the real world also changes accordingly. Note that FIGS. 4A , 4B , and 4C are marked with a mark 210 indicating a direction. Mark 210 makes it clear that even though the user is moving backward in the real world, their orientation is not changing in the virtual space.

[0035] Therefore, as in the cases of Figures 1A and 1B, even if the user is unable to perform the predictive actions developed in the content within the user play area as they are, the user can play the content within the user play area by receiving guide notifications of preparatory actions and by performing the preparatory actions themselves, allowing the user to enjoy the content to the fullest.

[0036] In addition, if the HMD is unable to detect the user's movements due to an HMD malfunction or if the HMD's grasp of the user's position, orientation, or other status differs from the actual situation, continuing to provide guidance notifications will not be effective and may result in a collision with an object. In this case, the HMD may prompt the user to move to the origin (e.g., the center of the play area) during a break in content, or may have a function to reset and initialize the content play. Furthermore, if the user makes a movement that is not instructed in the guidance notification, the HMD can issue a warning guidance notification because it detects and understands the user's movements using a camera or various sensors. Furthermore, in this example, a guidance notification for a preparatory movement to retreat from position 134 to position 301 has been described. However, the HMD may also issue a guidance notification for a preparatory movement to retreat when the user approaches the boundary of the play area.

[0037] Next, another example of the information processing device and information processing system according to this embodiment, in which preparatory movements are repeatedly performed, will be described with reference to the accompanying drawings. Figures 5A, 5B, 5C, and 5D are schematic diagrams illustrating the exterior of an embodiment of the information processing device and information processing system according to this embodiment. Figures 6A, 6B, 6C, 6D, and 6E are diagrams illustrating the display screen of the HMD in the embodiment shown in Figures 5A, 5B, 5C, and 5D. In Figures 5A, 5B, 5C, 5D, 6A, 6B, 6C, 6D, and 6E, components designated with the same reference numerals as those shown in Figures 1A, 1B, 2A, 2B, and 2C perform the same operations as those already described, and therefore detailed descriptions thereof will be omitted. In Figures 5A, 5B, 5C, and 5D, the HMD 100, external network 111, and information server 112 are omitted to avoid clutter. As illustrated in FIG. 5A , the content information acquired by the HMD 100 forms a virtual space 530 in which a building 531 and first and second towers 532 and 533 are arranged. Assume that a scenario is progressing in the virtual space 530 in which a user character 136, an avatar of the user, moves forward straight, then turns right and moves toward the building 531. At this time, the virtual distance (arrow 535) between the position of the user character 136 and the building 531 in the virtual space 530 is distance Vb in the virtual space. Distance Vb is, for example, the distance of 10 steps of the user character moving forward straight and then 5 steps after turning right, totaling 15 steps in the virtual space 530. This distance is merely a distance within the virtual space 530 and does not necessarily coincide with distance Rb (arrow 537) in the real space 141. However, in content in which the user's actions in the real space are reflected in the actions of the user character, which is the user's avatar in the virtual space, the scenario is achieved in which the user takes a total of 15 steps in the real space, causing the user character to take a total of 15 steps in the virtual space and arrive at the destination, building 531.

[0038] Here, to achieve the scenario, the user 10 in the real world needs to take 10 steps straight from position 534 to right-turn position 542. However, in the example of FIG. 5A , the user 10 wearing the HMD 100 cannot move forward the 10 steps straight from position 534 to right-turn position 542 within the play area 520 in the real world. That is, the first part of the predicted action to achieve the scenario, which is to reach building 531 in the virtual space, cannot be performed within the play area. Note that arrow 511 indicates the direction of the user 10. In response to this, when the HMD 100 recognizes that the first part of the predicted action to achieve the scenario in the virtual space cannot be performed within the user play area in the real world without changing posture or position, the HMD 100 determines a first preparatory action to rotate the direction of the user 10 in the real world by 90 degrees to the right, as indicated by arrow 501, and notifies the user 10 of the determined first preparatory action by displaying it on the screen or by audio from the speaker 107. Furthermore, by the user 10 performing the first preparatory movement of a 90-degree right turn in the real world, as indicated in the guide notification, the user 10 can secure a distance of 10 steps from position 534 to right turn position 542 in the play area in the real world, as shown in FIG. 5B . Therefore, the user 10 can take 10 steps forward in the real world from position 534 to right turn position 542, as indicated by arrow 544. The user character 136 in the virtual space also takes 10 steps forward in conjunction with this user movement in the real world, as indicated by arrow 543, and as a result, can reach right turn position 541. Note that arrow 512 indicates the direction of the user character 136. Furthermore, for example, when the user 10 walking within the play area approaches the boundary of the play area, the HMD 100 provides a guide notification.

[0039] As shown in FIG. 6A , after the first preparatory movement is determined, the HMD 100 displays a message guide 601 on the display screen 200 stating "Preparatory movement guide: Rotate your orientation 90 degrees right," along with a building 531, first and second towers 532 and 533, and the foot 203 of the user character 136, which is the user's alter ego, in the virtual space 530. The play area 520 indicated by the dashed dotted line in FIG. 6A indicates the user's play area and is not displayed on the HMD 100. However, it is shown to indicate that the user 10 in the real world is located near the boundary of the play area in the situation shown in FIG. 6A . While the user 10, who has received the message guide 601, is performing the first preparatory movement in the real world, the HMD 100 does not reflect the user's movement in the movement of the user character, which is the user's alter ego, in the virtual space 530. Then, when the HMD 100 detects the completion of the first preparatory movement by the user 10 in the real world, as shown in Fig. 6B , the display states of the building 531, the first and second towers 532 and 533, and the feet 203 of the user character 136 remain unchanged, and a message guide 602 stating "Preparatory movement guide: A 90-degree rotation to the right has been detected" is displayed on the display screen 200. Here, the play area 520 indicated by the dashed dotted line in Fig. 6B is not displayed on the HMD 100 as in Fig. 6A , but is illustrated to indicate that the change in the user's orientation in the real world due to the first preparatory movement has secured a distance within the play area to accomplish the first part of the predicted action. That is, even if the user 10 turns 90 degrees to the right in the real world, the user character in the virtual space does not change its orientation, and therefore the content display screen 200 that the user 10 sees does not change before or after the first preparatory movement, and remains the same image as before the user 10 started moving straight ahead, with the first and second towers 532 and 533 ahead (north in the content) and building 531 on the right hand side of the three-way intersection (east in the content). In other words, performing preparatory movements in the real world does not affect the scenario in the virtual space.

[0040] Next, in order to achieve this scenario, the user 10 in the real world turns right to turn right at the right turn position 542. As shown in FIG. 5C , the user 10 turns right in the real world, and the user character 136 in the virtual space also turns right in conjunction with this. Note that an arrow 512 indicates the direction of the user character 136. In other words, the content display screen of the HMD 100 displays an image of the building 531 facing forward.

[0041] Here, to achieve the scenario, the user 10 in the real world needs to take five steps forward from the right turn position 542. However, in the example of FIG. 5C , the user 10 wearing the HMD 100 cannot move forward five steps forward in a straight line from the right turn position 542 within the play area 520 in the real world, as indicated by arrow 545. In other words, the remaining part of the predicted action to achieve the scenario of reaching the building 531 in the virtual space cannot be performed within the play area. Therefore, when the HMD 100 recognizes that the remaining part of the predicted action to achieve the scenario in the virtual space cannot be performed within the user play area in the real world without changing posture or position, the HMD 100 determines a second preparatory action to rotate the orientation of the user 10 in the real world by 90 degrees to the right, as indicated by arrow 504, and notifies the user 10 of the determined second preparatory action by displaying it on the screen or by audio from the speaker 107. Furthermore, when the user 10 performs the second preparatory movement of turning 90 degrees to the right in the real world, as indicated by the guide notification, the user 10 is able to secure a distance of five steps from the right turn position 542 in the play area in the real world, as shown in Fig. 5D . Note that, for example, the HMD 100 issues a guide notification when the user 10 walking within the play area approaches the boundary of the play area.

[0042] After determining the second preparatory movement, the HMD 100 displays a message guide 603 on the display screen 200, reading "Preparatory movement guide: Rotate your orientation 90 degrees right," along with a building 531, a second tower 532, and the feet 203 of the user character 136, which is the alter ego of the user 10, in the virtual space 530, as shown in FIG. 6C . Note that the play area 520 indicated by the dashed dotted line in FIG. 6C indicates the user's play area, as in FIGS. 6A and 6B , and is not displayed on the HMD 100, but is shown to indicate that the user 10 in the real world is located near the boundary of the play area in the situation shown in FIG. 6C . While the user 10, who has received the display of the message guide 603, is performing the second preparatory movement in the real world, the HMD 100 does not reflect the user's movement in the movement of the user character, which is the alter ego of the user 10, in the virtual space 530. Then, when the HMD 100 detects the completion of the second preparatory movement by the user 10 in the real world, as shown in FIG. 6D , the display states of the building 531, the first tower 532, and the feet 203 of the user character 136 remain unchanged, and a message guide 604 reading "Preparatory movement guide: A 90-degree rotation to the right has been detected" is displayed on the display screen 200. Here, the play area 520 indicated by the dashed dotted line in FIG. 6D is not displayed on the HMD 100, as in FIGS. 6A , 6B , and 4C . However, it is illustrated to indicate that the change in the user's orientation in the real world due to the second preparatory movement has secured a distance within the play area to accomplish the remaining part of the predicted behavior. In other words, even if the user 10 rotates their orientation 90 degrees to the right in the real world, the user character in the virtual space does not change their orientation. Therefore, the content displayed on the display screen 200 of the content viewed by the user 10 does not change before and after the second preparatory movement, and the image remains the same, with the first tower 532 to the front left and the building 531 directly ahead. In other words, even if the user 10 performs a preparatory movement in the real world, the preparatory movement does not affect the scenario in the virtual space. After detecting the completion of the second preparatory movement by the user 10 in the real world, the HMD 100 resumes the process of reflecting the movement of the user 10 in the real world on the movement of the user character.Thus, the user 10 can move forward the remaining distance in the real world as indicated by arrow 546. In conjunction with this user movement in the real world, the user character 136 in the virtual space also moves forward the remaining distance as indicated by arrow 547, thereby achieving a scenario in which the user reaches the destination building 531. At this time, the content display shows a scene with the building 531 right in front of them, as shown in FIG. 6E . Here, the play area 520 indicated by the dashed dotted line in FIG. 6E is not displayed on the HMD 100, as in FIGS. 6A , 6B , 6C , and 6D . However, it is shown to indicate that the position of the user character 136 in the virtual space changes in conjunction with the user 10's forward movement in the real world, and that the displayed image and the real world position also change accordingly. Therefore, the scenario in the virtual space can be progressed solely by movements in the real world linked to the character's movements, without the need to switch control systems or play styles as in the past. This allows the user to enjoy the content with the feeling that they are actually moving within the virtual space. 6A, 6B, 6C, 6D, and 6E are provided with marks 611 and 612 indicating directions. The marks 611 and 612 clearly show that the user character 136, which is the user's alter ego in the virtual space, faces north and moves straight ahead, then turns right and faces east, advancing toward the building 531.

[0043] On the other hand, if the user 10 does not initiate the first and second preparatory movements even after receiving a guide notification for the first and second preparatory movements, the user 10 continues playing the predicted action, ignoring the guide notification. Therefore, while the predicted action is still being played within the user play area 520, the HMD 100 provides a guide notification message such as "Perform preparatory movements," urging the user to quickly perform the first and second preparatory movements. On the other hand, if the user 10 strays outside the user play area 520, a dangerous situation may arise, such as colliding with an object outside the user play area 520. Depending on the content, the user may hit the wall of the guardian area surrounding the user play area. The HMD 100 provides a guide notification message such as "Play not possible, content execution stopped," warning that play is no longer possible and indicating that content execution will be stopped, and stops content execution. This can promote prompt execution of preparatory movements and safe content play. Note that the content execution may not be stopped; instead, a warning that play is not possible may be issued, a notification to change the play style may be issued, or the play style may be forcibly changed. This has the advantage of allowing the user to continue playing. Thus, even if the user is unable to perform the predictive actions developed by the content within the user play area, a guide notification of preparatory actions that allows the user to perform the predictive actions within the user play area is issued, and the user repeatedly executes preparatory actions, enabling the user to play the content within the user play area and fully enjoy the content. While Figures 5A, 5B, 5C, 5D, 6A, 6B, 6C, 6D, and 6E illustrate the case where two preparatory actions are performed, they may be repeated multiple times. Therefore, regardless of the narrowness of the user play area or the wide range of the content's predictive actions, the user can fully enjoy the content without any restrictions on their movements.Ultimately, even if there is not enough user play area, the effect is that the selected content can be played without having to switch from a room-scale play style to a seated or standing play style where movement is restricted by the small area.

[0044] Next, as other examples of the information processing device and information processing system according to this embodiment, cases in which preparatory movements are performed in a standing or seated playing style will be described with reference to the drawings. Figures 7A, 7B, 7C, and 7D are schematic diagrams showing the exterior appearance of an embodiment of the information processing device and information processing system for a standing playing style. Figures 8A and 8B are schematic diagrams showing the exterior appearance of an embodiment of the information processing device and information processing system for a seated playing style.

[0045] In Fig. 7A , a user 10 wearing an HMD 100 and attempting to play content in a standing play style has reserved a space, called a user play area 701, for moving their arms to play the content in the virtual space. In this state, when the HMD 100 recognizes that a predictive action in the virtual space will cause the arm 702 to extend outside the user play area 701, as shown in Fig. 7B , the HMD 100 provides a guide notification of a preparatory action to rotate the orientation of the user 10 in the real world 90 degrees to the right, as indicated by arrow 703. When the user 10 performs the preparatory action in accordance with the guide notification, the user 10 is able to thrust the arm 702 forward within the user play area 701, as shown in Fig. 7C , and the predictive action in the virtual space linked to thrusting the arm 702 forward can be performed without any hindrance within the play area.

[0046] Furthermore, when the HMD 100 recognizes that the predicted action in the virtual space will cause the arm 702 to extend outside the user play area 701, it notifies the user 10 of a preparatory action to move back a certain distance in the real world as indicated by the arrow 704. When the user 10 performs the preparatory action in accordance with the guide notification, the user 10 is again able to thrust the arm 702 forward within the user play area 701 as shown in FIG. 7D , and the predicted action in the virtual space can be performed safely within the play area.

[0047] 8A , a user 10 wearing an HMD 100 and sitting in a chair 800 intending to play content in a seated play style has reserved a space, called a user play area 801, for moving their arms to play the content in the virtual space. It is preferable that the chair 800 has a swivel seat. In this state, when the HMD 100 recognizes that arms 802 and 803, which are spread out to the left and right due to predictive behavior in the virtual space, will extend outside the user play area 801, as shown in FIG. 8B , the HMD 100 provides a guide notification of a preparatory action to rotate the orientation of the user 10 90 degrees to the right in the real world, as indicated by arrow 804. When the user 10 performs the preparatory action of rotating the chair they are sitting on 90 degrees to the right in accordance with the guide notification, the user can spread their arms 802 and 803 to the left and right within the user play area 801, as shown in FIG. 8C . This allows the user 10 to safely perform the predictive behavior in the virtual space associated with spreading their arms 802 and 803 to the left and right within the play area. Therefore, whether playing in a standing or seated position, the user can play content within the user play area by receiving guide notifications for preparatory movements and performing the preparatory movements themselves, allowing the user to enjoy the content to the fullest.

[0048] Next, as another example of the information processing device and information processing system according to this embodiment, a case where multiple users coexist will be described with reference to the accompanying drawings. Figures 9A and 9B are schematic diagrams illustrating the exterior of an embodiment of the information processing device and information processing system in which two users play content in overlapping play areas. In Figure 9A, users 11 and 12 wearing HMDs 901 and 902 and attempting to play content have secured user play areas 911 and 912, respectively, in which they can move. In this state, if user 901 moves forward in a straight line as indicated by movement 921 based on predictive behavior in the virtual space, and user 902 moves forward in a straight line as indicated by movement 922 based on predictive behavior in the virtual space, the two users will move toward the same position 930 in the real world, creating a risk of collision. Therefore, when the HMDs 901 and 902 recognize a risk of collision between the two in the real world through predictive actions, for example, the HMD 901 provides a guide notification for a preparatory movement to rotate the orientation of the user 11 in the real world to the right as indicated by arrow 931, and the HMD 902 provides a guide notification for a preparatory movement to rotate the orientation of the user 12 in the real world to the right as indicated by arrow 932. When the users 11 and 12 perform their respective preparatory movements in accordance with their respective guide notifications, as shown in FIG. 9B , the users 11 and 12 move straight after the preparatory movements as indicated by arrows 921 and 922, thereby preventing a collision between the two in the real world. Note that while FIGS. 9A and 9B show an example in which a guide notification is provided to instruct both users 11 and 12 to change their orientation, a guide notification for a preparatory movement may be provided to only one of the users. For example, it is clear that a collision between the two can be similarly avoided even if a guide notification is provided to only the user 11 and the user 11 performs a preparatory movement. Furthermore, the effect of avoiding collisions between the users 11 and 12 in the real world by the guide notification of the present invention is equally effective whether both users are playing the same content or different content.

[0049] Next, the basic operation of the information processing device and information processing system according to this embodiment will be described using a flowchart. FIG. 10 is an example of a flowchart illustrating the basic operation of the information processing device and information processing system according to this embodiment. In FIG. 10 , the HMD 100 (specifically, the processor) acquires user environment information indicating the ambient environmental state of the user 10 based on the spatial environment around the user 10 detected by the camera 101 and the distance sensor 102 (S1001). Based on the user environment information acquired by the HMD 100, the user 10 sets a user play area in which the user 10 can move and play, and the HMD 100 acquires the settings of the user 10 (S1001). Furthermore, after the user 10 selects content to play, the HMD 100 acquires content information for the selected content from the information server 112 (S1002). When content play selected by the user 10 begins (S1003), the HMD 100 acquires predicted behavior (predicted behavior) developed in the content from the information server 112 as predicted behavior information (S1004), and acquires the user's 10 current position using the positioning sensor 103 and the user's 10 orientation and other posture information using the posture sensor 104 as user position / posture information (S1005). If the HMD 100 recognizes that the predicted behavior in the virtual space can be performed within the user play area based on the user's position and posture within the user play area (S1006), the user 10 performs an action corresponding to the predicted behavior (S1007). If content play continues, the process returns to sequence S1004. If content play ends, all sequences end (S1008).

[0050] On the other hand, if the HMD 100 recognizes that some or all of the predicted action will not be performed within the user play area (S1006), it determines preparatory actions that can be performed within the user play area for some or all of the predicted action and notifies the guide of the determined preparatory actions (S1011). When the HMD 100 detects that the user 10 has started the preparatory actions for which the guide notified (S1012), the HMD 100 temporarily suspends the process of reflecting the user 10's actions in the real world on the character actions in the content (the character actions in the virtual space) (S1013). Note that although the flowchart shows that S1013 is executed after S1012, in reality, once the start of the preparatory action is detected in S1012, the HMD 100 immediately temporarily suspends the reflection of the user 10's actions in the real world on the character actions in the content in S1013. Thereafter, when the HMD 100 detects the end of the preparatory movement (S1014), the HMD 100 resumes the process of reflecting the user's 10's movements in the real world on the character's movements in the content (S1015). Because steps S1013 to S1015 prevent the user's 10's movements from being reflected in the character's movements in the content during the preparatory movement, the display screen of the virtual space representing the content remains unchanged, and the user 10 can change their position and posture in the real world while maintaining the display screen before the preparatory movement. If only a portion of the predicted action has been executed (S1016), the process returns to sequence S1011, and a preparatory action is determined so that the next portion of the predicted action will be performed within the user play area. If the entire predicted action is executed (S1016) and content play continues, the process returns to sequence S1004. If content play ends, the entire sequence ends (S1017). Incidentally, if the HMD 100 does not immediately detect the start of the preparatory movement notified by the guide (S1012), and if the predicted action is being performed within the play area (S1018), the HMD 100 notifies the user 10 to urge the user 10 to perform the preparatory movement (S1019). However, if the predicted action is no longer being performed within the play area (S1018), the HMD 100 notifies the user 10 that play is no longer possible, notifies the user 10 to change the play style, or stops the execution of content play (S1020), and the entire sequence ends.Furthermore, the play style may be forcibly changed, for example, from a moving (room scale) play style to a standing play style, without stopping the execution of the content.

[0051] Thus, even if the recommended play area for the content recommended in the content cannot be secured, the content can be played in the same manner as when the recommended play area is secured, widening the range of content selection, and the user can enjoy the content to the fullest without being restricted in movement.

[0052] 11A and 11B , examples of predicted behavior information indicating predicted behavior (predicted behavior) in the information processing device and information processing system according to this embodiment will be described. The predicted behavior information is stored as a table in the information server 112 or the like, and indicates how the user / player will move in relation to direction and distance in which content and in which scene. The aforementioned scenario is composed of one or more scenes shown in FIG. 11A . For example, in the scenario of FIG. 5A , a scenario of heading toward building 531 is composed of a scene in which the user takes 10 steps straight toward a three-way intersection from the position indicated by 534, a scene in which the user reaches the three-way intersection and decides which direction to head in scene 100, and a scene 90 in which the user turns right in scene 100. Furthermore, the predicted behavior information may be information assumed in advance by the content, or the history and tendencies of the user / player or all users / players may be collected and that information may also be taken into consideration. Content that makes it easy to predict the user / player's movements makes it easier to guide preparatory movements, increasing accuracy and providing more time for preparatory movements. In addition, in games such as sports and fighting games, predicted behavior may be captured based on the history and tendencies of the user / player, such as if the previous state was like this, then the next state will be like this.

[0053] 11A , in a scene at a three-way intersection, predicted behavior 1 may be predicted to go right with a 60% probability, predicted behavior 2 may be predicted to go left with a 30% probability, and predicted behavior 3 may be predicted to turn back with a 10% probability. In addition, in a scene at a four-way intersection, predicted behavior 1 may be predicted to go straight with a 70% probability, predicted behavior 2 may be predicted to go right with a 20% probability, and predicted behavior 3 may be predicted to go left with a 10% probability. 11B , in content such as a table tennis game, the predicted behavior information may be "move 1.0 m to the right" if the opponent's previous action hits a fast ball to the right side of the player's right side, or if the player's previous action hits a slow serve to the right side of the net and then the opponent's previous action hits a fast ball to the right side of the player's right side, or if the opponent's previous action hits a slow serve to the left side of the net and then the player's previous action hits a slow serve to the right side of the net and then the opponent's previous action hits a fast ball to the right side of the player's right side, as shown in FIG. In this case, the history and tendencies of the player and the opponent are collected and this information is taken into consideration when generating the predicted behavior information.

[0054] Next, a configuration example of an information processing device and an information processing system according to this embodiment will be described using a block diagram. FIG. 12 is a block diagram showing a configuration example of an information processing device and an information processing system according to this embodiment. In FIG. 12, the HMD 100 is configured appropriately using a camera 101, a distance measurement sensor 102, a positioning sensor 103, an attitude sensor 104, a microphone 106, a speaker 107, a vibrator 108, an operation input interface 1207, a display 1208, a processor 1220, a memory 1230 including a program 1231 and information data 1232, a network communication interface 1246, and a short-range wireless communication interface 1247, and each component is interconnected via a bus 1260. Furthermore, the HMD 100 wirelessly communicates content information, predicted behavior information, and the like with an external information server 112 via an external network 111 via the network communication interface 1246. Next, each component of the HMD 100 will be described in detail.

[0055] The camera 101 captures the user's surroundings by converting light incident from a lens into an electrical signal using an image sensor to obtain a camera-captured image. The HMD 100 extracts the spatial environmental state around the user 10 from the camera-captured image captured by the camera 101 and obtains user environment information indicating the environmental state around the user.

[0056] The ranging sensor 102 is a sensor that can measure the distance and angle to an object and capture the shape of the object in three dimensions. Examples of such sensors include LiDAR (Light Detection and Ranging), which irradiates an object with laser light such as infrared light, measures the reflected scattered light, and analyzes and detects the distance to a distant object and the state of the object; TOF (Time of Flight) sensors, which measure the distance by irradiating a subject with pulsed light and measuring the reflection time for each pixel; and millimeter-wave radar, which emits millimeter-wave radio waves, captures the reflected waves, and detects the distance to the object and the state of the object. The HMD 100 uses the ranging sensor 102 to measure the distance and direction to the object and acquires user environment information indicating the environmental state around the user from the shape of the object calculated based on the measurement information.

[0057] The positioning sensor 103 receives radio signals from GPS (Global Positioning System) satellites in the sky to detect the current location, and can identify the position of the HMD 100 and the current location of the user 10 wearing the HMD 100.

[0058] The orientation sensor 104 is composed of an acceleration sensor 1204, a gyro sensor 1205, a geomagnetic sensor 1206, and the like, and detects orientation states such as direction and tilt. The acceleration sensor 1204 is a sensor that detects acceleration, which is a change in speed per unit time, and can capture movement, vibration, shock, and the like. The gyro sensor 1205 is a sensor that detects angular velocity in the rotation direction, and can capture vertical, horizontal, and diagonal orientation states. The geomagnetic sensor 1206 is a sensor that detects the Earth's magnetic force, and detects the direction in which the HMD 100 is facing. These sensors enable the user's orientation, tilt, and other orientations to be acquired as user position / orientation information.

[0059] The operation input interface 1207 is an interface with input means such as gaze, hands, or a pointer, through which the user 10 sets and inputs desired information. An input operation screen such as an operation menu may be displayed on the display screen of the display 1208, and input operation information may be acquired based on the position on the input operation screen to which the gaze is directed. Alternatively, a hand or a pointer may be displayed on the input operation screen, and input operation information may be acquired by manipulating the hand or pointer. Alternatively, input operation information may be acquired by an operation device controller 1248, such as a keyboard, key buttons, or touch keys, which is an operation device separate from the HMD 100. The operation device controller 1248, which has an operation device input operation unit 1249, may be connected to the HMD 100 main body wirelessly via a short-range wireless communication interface 1250, or may be connected to the HMD 100 main body using a wired connection medium. Examples of operation input include content selection and setting, and setting a user play area.

[0060] The display 1208 is configured with a display device using a liquid crystal panel or a hologram, for example, in an immersive HMD that is worn on the user's head and displays a virtual space on a display unit, and displays virtual objects of the content.

[0061] The processor 1220 (control unit) is composed of a CPU or the like, and controls each component of the HMD 100 by executing an operating system (OS) 1233 and an application program 1234 for operation control, etc., stored in the memory 1230, thereby realizing functions of the OS, middleware, applications, etc., and other functions. In the processor 1220, a user play area setting processing unit 1221, a predicted behavior information acquisition processing unit 1222, a predicted behavior area determination processing unit 1223, a preparatory movement decision guide notification processing unit 1224, and a preparatory movement display support processing unit 1225 control the operation of each function.

[0062] The memory 1230 is configured with a nonvolatile storage device or the like, and stores various programs 1231 and information data 1232 handled by the processor 1220, etc. The programs 1231 include an operating system 1233 and various application programs 1234 for controlling operations used by the processor 1220. The information data 1232 stores user environment information 1235 indicating the user's movement environment obtained by photographing with the camera 101 and detection by the distance sensor 102, user position / posture information 1236 captured by the positioning sensor 103 and the posture sensor 104, user play area information 1237 set by the user, content information 1238, predicted behavior information 1239, preparatory movement information 1240, etc.

[0063] The microphone 106 collects the user's own speech and external sounds and converts them into audio data. The user 10 speaks a voice indicating an instruction such as an input operation, and the voice is collected by the microphone 106 to capture the instruction information such as the input operation, thereby enabling the user to conveniently execute an operation in response to the instruction information.

[0064] The speaker 107 outputs sound based on the audio data, and can audibly notify the user 10 of notification instruction information. For example, a message such as "This is a preliminary operation guide, please rotate your orientation 180 degrees" can be audibly output to notify the user 10, thereby improving usability.

[0065] The vibrator 108 generates vibrations under the control of the processor 1220, and converts notification information sent by the HMD 100 to the user 10 into vibrations. The vibrator 108 transmits vibrations to the head of the user wearing the HMD 100, and can notify the user 10 of notification information such as a message guide such as "Preliminary operation guide: rotate your orientation 180 degrees" by vibration, thereby improving usability.

[0066] The network communication interface 1246 is a communication interface that performs wireless communication with the external information server 112 via the external network 111 by base station communication or the like. The network communication interface 1246 can receive content information, predicted behavior information, and the like stored in the information server 112 from the information server 112, and conversely, can transmit requests for predicted behavior information, notifications of content play status, and the like to the information server 112. Note that the base station communication may use long-distance wireless communication such as W-CDMA (Wideband Code Division Multiple Access) or GSM (Global System for Mobile communications).

[0067] The short-range wireless communication interface 1247 is a communication interface that performs short-range wireless communication between the HMD 100, the information server 112, and the operation device controller 1248 when the information server 112 is within a range where short-range wireless communication is possible. The short-range wireless communication is performed using, for example, an electronic tag, but is not limited to this, and may also be performed using Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or a wireless LAN (IEEE802.11a, IEEE802.11b, IEEE802.11g).

[0068] Next, each processing step performed by the processor 1220 will be described.

[0069] The user play area setting processing unit 1221 performs processing to set the range in which the user can move during play as the user play area based on the user environment information.

[0070] The predicted behavior information acquisition processing unit 1222 acquires predicted behavior that will unfold in the content from the information server 112 while the content is being played.

[0071] The predicted behavior area determination processing unit 1223 determines whether the predicted behavior will be performed within the user play area.

[0072] When the predicted action is not performed within the user play area, the preparatory action determination guide notification processing unit 1224 determines a preparatory action for the user that will cause the predicted action to be performed within the user play area, and notifies the user of the preparatory action.

[0073] The preparatory movement display handling processor 1225 stops the change in content display accompanying the preparatory movement while the user is performing the preparatory movement for which a guide notification has been sent.

[0074] With the above configuration, in the HMD 100, the user play area setting processing unit 1221 sets a user play area in which the user can move during play based on user environment information. The HMD 100 acquires content information for content selected by the user, and when content play begins, the predicted behavior information acquisition processing unit 1222 acquires predicted behavior that will unfold in the content. The HMD 100 uses the predicted behavior area determination processing unit 1223 to determine whether the acquired predicted behavior will occur within the set user play area. If the predicted behavior will not occur within the user play area, the HMD 100 uses the preparatory movement determination guide notification processing unit 1224 to determine a user preparatory movement that will cause the predicted behavior to occur within the user play area, and notifies the user of the preparatory movement. The HMD 100 uses the preparatory movement display response processing unit 1225 to stop content display changes associated with the user's preparatory movement during the user's preparatory movement, and resumes content display changes associated with the user's play movement after the user's preparatory movement is completed. Furthermore, if the preparatory movement notified by the HMD 100 does not start quickly, the HMD 100 prompts the user 10 to perform the preparatory movement if the predicted action is being performed within the play area, and if the predicted action is no longer being performed within the play area, the HMD 100 issues a warning to the user 10 and stops content play or notifies the user 10 to change the play style. This allows content to be played within the play area where the user is always moving, resulting in the effects of expanding the range of content selection and allowing the user to fully enjoy content without restricting their movements.

[0075] When determining whether a user's preparatory action is a normal play action, a user's preparatory action may be determined to be a movement performed while performing a special operation, or a movement performed during a designated period. Examples of such special operations include a combination of multiple button operations performed by the operating device controller 1248 held by the user that are not used in normal content play, or a combination of a user action and a button operation. An example of the designated period is a period during which a preparatory action guide notification is displayed. Regarding the timing of the guide notification, the more complex the user's preparatory action, the earlier the guide notification may be started. If the user's preparatory action is not completed, the manner of the guide notification may change as the time until the action approaches. The guide notification may be displayed in the form of an absolute distance or position, such as "move 1.5 m diagonally to the right at a 45-degree angle." Alternatively, a distance scale, direction, angle, or other guide may be displayed. Intuitive guidance, such as "take a large step forward," may be provided, and the user's stride length and body shape may be calculated and adjusted accordingly. Furthermore, guidance may be provided to reduce the movement, such as guiding the user to keep the predicted movement to 1.0 m when the movement is 1.5 m. Furthermore, as a method of providing guidance notifications, the display may be output in a manner and timing that minimizes the impact on the content being played, and similarly, the audio may be output in a manner and timing that minimizes the impact on the audio of the content. Regarding content execution reset, if it becomes difficult to determine the user's position or orientation, the guidance notification may be stopped and the user may return to a predetermined position. As an exception handling, if the preparatory movement is not performed as specified, a warning such as "Be careful of your surroundings!" may be issued. User feedback may be provided regarding the timing and output method of notification guidance from the user during notification guidance or at the end of play, and settings may be changed in preparation for the next play.

[0076] In the above description, an immersive HMD that is worn on the user's head and displays a virtual space on a display unit has been used as a specific example of an information processing device, but the present invention is applicable to all devices that have similar functions. For example, it goes without saying that the same effects and advantages can be obtained with an optically transparent HMD that displays virtual objects on a display unit while visually recognizing real objects in front of the user.

[0077] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0078] Furthermore, the numerical values, messages, etc. appearing in the text and figures are merely examples, and the effects of the present invention will not be impaired even if different ones are used.

[0079] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the program, table, and file that implements each function may be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD, or may be stored in a device on a communication network.

[0080] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected.

[0081] As an example, the following information processing device, information processing system, and information processing method may be provided.

[0082] (1) An information processing device that displays content played by a user in a virtual space, the information processing device having a user play area acquisition means that acquires a user-defined user play area in which the user can move during play, and a content predicted behavior information acquisition means that acquires predicted behavior information of the content from an external information server while the content is being played, and if the predicted behavior acquired by the content predicted behavior information acquisition means is not performed within the user play area acquired by the user play area acquisition means, the information processing device determines a preparatory action to change the user's position or direction so that the predicted behavior will be performed within the user play area, and provides guidance and notification to the user about the determined preparatory action.

[0083] (2) An information processing device according to (1), characterized in that, when the information processing device detects the start of a preparatory movement by the user after a guide notification, the information processing device temporarily suspends the display of the content reflecting the user's preparatory movement, and after detecting the end of the user's preparatory movement, resumes the display of the content reflecting the user's play movement.

[0084] (3) The information processing device according to (2) is characterized in that, if only a part of the predicted action is performed by the preparatory action performed in accordance with the guide notification, the information processing device repeats the guide notification of the preparatory action and the execution of the preparatory action until all of the predicted action is performed.

[0085] (4) An information processing device according to (1), characterized in that after the guide notification, if the predicted action is being performed within the play area without detecting the start of the user's preparatory action, the information processing device notifies the user to perform the preparatory action, and if the predicted action is no longer being performed within the play area, the information processing device notifies the user that play is no longer possible and stops execution of the content.

[0086] (5) An information processing device according to (1), characterized in that when there is another information processing device having a user play area that overlaps with the user play area of ​​the information processing device, and the users of the two information processing devices collide due to the predicted actions of the two information processing devices, at least one of the information processing devices determines a preparatory action to avoid the collision and provides a guide notification of the determined preparatory action.

[0087] (6) An information processing system having an information processing device that displays content played by a user in a virtual space, and an information server that has information about the content and is communicatively connected to the information processing device, wherein the information processing device has a user play area acquisition means that acquires a user-defined user play area in which the user can move during play, and a content predicted behavior information acquisition means that acquires predicted behavior of the content from the information server as predicted behavior information while playing the content from the information server, and when the predicted behavior acquired by the content predicted behavior information acquisition means is not performed within the user play area acquired by the user play area acquisition means, the information processing system is characterized in that the information processing device determines a preparatory action to change the user's position or direction so that the predicted behavior will be performed within the user play area, and notifies the user of the content of the determined preparatory action.

[0088] (7) An information processing method in an information processing system having an information processing device that displays content played by a user in a virtual space, and an information server that has information about the content and is communicatively connected to the information processing device, the information processing method comprising the steps of: the information processing device acquiring a user-defined user play area in which the user can move while playing the content; the information processing device acquiring predicted behavior of the content from the information server as predicted behavior information while playing the content; if the acquired predicted behavior is not performed within the acquired user play area, the information processing device determining a preparatory action to change the user's position or direction so that the predicted behavior will be performed within the user play area; and the information processing device guiding and notifying the user of the content of the determined preparatory action.

[0089] 10, 11, 12: User, 100, 901, 902: HMD, 101: Camera, 102: Distance sensor, 103: Positioning sensor, 104: Orientation sensor, 106: Microphone, 107: Speaker, 108: Vibrator, 111: External network, 112: Information server, 120, 520, 701, 801, 911, 912: User play area, 131, 531: Building, 132, 532: First tower, 133, 533: Second tower, 136: User Character, 1204: acceleration sensor, 1205: gyro sensor, 1206: geomagnetic sensor, 1207: operation input interface, 1208: display, 1220: processor, 1230: memory, 1231: program, 1232: information data, 1246: network communication interface, 1247, 1250: short-range wireless communication interface, 1248: operation device controller, 1249: operation device input operation unit, 1260: bus

Claims

1. An information processing device that displays content in a virtual space, comprising a control unit, wherein the control unit acquires area information regarding an area in which a user will behave in relation to the content and predicted behavior information regarding behavior predicted based on the area information, and when it determines that the behavior predicted based on the predicted behavior information cannot be performed within the area, determines a preparatory movement regarding an action to change the user's position and / or orientation, and notifies the user of the determined preparatory movement.

2. An information processing device according to claim 1, wherein the control unit temporarily suspends the reflection of the user's preparatory movements in the content when it detects the start of the user's preparatory movements after notifying the user of the preparatory movements, and resumes the reflection of the user's play movements in the content after it detects the end of the user's preparatory movements.

3. An information processing device according to claim 2, wherein the control unit, when all predicted actions have not been performed, repeats the process of determining preparatory actions and notifying the content of the determined preparatory actions until all predicted actions have been performed.

4. An information processing device as described in claim 1, wherein the control unit, if the predicted action is being performed within the area after the notification and without detecting the start of the user's preparatory action, issues a prompting notification to the user to perform a preparatory action, and if the predicted action is no longer being performed within the play area, issues a warning and stops execution of the content.

5. An information processing device according to claim 1, characterized in that, when there is another information processing device whose area overlaps with the area of ​​the information processing device and the users of the two information processing devices collide due to the predicted actions of the two information processing devices, the information processing device determines a preparatory action to avoid the collision and notifies the user of the determined preparatory action.

6. An information processing system of an information processing device that displays content in a virtual space, wherein the information processing device acquires area information regarding an area in which a user will behave in relation to the content, acquires predicted behavior information regarding behavior predicted based on the area information, and when it determines that the behavior predicted based on the predicted behavior information cannot be performed within the area, determines a preparatory movement regarding an action to change the user's position and / or orientation, and notifies the user of the content of the determined preparatory movement.

7. An information processing method performed using an information processing device that displays content in a virtual space, comprising: a step in which the information processing device acquires area information regarding an area in which a user will behave in relation to the content; a step in which the information processing device acquires predicted behavior information regarding behavior predicted based on the area information; a step in which the information processing device determines, when it is determined that the behavior predicted based on the predicted behavior information cannot be performed within the area, a step in which the information processing device determines a preparatory movement regarding an action to change the position and / or orientation of the user; and a step in which the information processing device notifies the user of the content of the determined preparatory movement.

Images