Information processing device, information processing method, and program

Abstract

To allow a user to improve his / her posture without having to interrupt work or the like.SOLUTION: An information processing device provided herein is configured to determine whether a posture of a user viewing a display image has shifted from a predetermined reference posture or not and, if the current posture of the user is determined to have shifted from the reference posture, move a display position of the display image viewed by the user in a second direction opposite a first direction in which the current posture of the user has shifted from the reference posture.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing technique for displaying images on a head-mounted or glasses-type display device. [Background technology]

[0002] In recent years, various types of display devices have appeared, such as head-mounted types worn by users on their heads and ear-mounted types like eyeglasses. Unlike other common display devices, these display devices can provide users with images that occupy the entire field of view or a large portion of the field of view. Furthermore, these display devices are equipped with various sensors, such as angular velocity sensors, acceleration sensors, and distance sensors, and many of them have the ability to detect the movement of the user's head from the information of these sensors and continuously change the displayed image in conjunction with the position and orientation of the head. This allows users to experience a high level of immersion, as if the images displayed on the display device were actually present in the real world.

[0003] One application that takes advantage of the characteristics of such display devices is to provide users with a virtual work environment. A virtual work environment is a virtual environment in which various virtual devices, such as machines, products, and tools, are displayed on a display device worn by a user, and the virtual devices can be moved, changed, and operated based on the user's movements and inputs. However, just like in normal real work, working in a virtual work environment for a long period of time can cause poor posture without the user even realizing it.

[0004] Patent Document 1 discloses a technology for correcting the posture of a user using a mobile device. This technology estimates the user's posture based on information from an acceleration sensor, angular velocity sensor, etc., installed in the mobile device. If it is determined that the user's posture has deteriorated, the technology warns the user by displaying a warning message on the mobile device screen or sounding a warning sound. The warning given to the user is so loud that it would cause inconvenience to the user and prevent the user from continuing to use the mobile device. For example, the warning message may be displayed in a size that occupies most of the image display area, or the warning sound may be sounded at a volume that is noticeable. The mobile device then stops the warning when it detects that the user has improved their posture and taken a deep breath in accordance with the instructions included in the warning. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-171444 Summary of the Invention [Problem to be solved by the invention]

[0006] In the case of the technology disclosed in Patent Document 1, when a user is warned that their posture is deteriorating, they cannot ignore the warning and must stop what they are doing and follow the instructions on the mobile device to improve their posture, which can cause the user to lose concentration while working.

[0007] Therefore, an object of the present invention is to enable a user to improve their posture without interrupting work or the like. [Means for solving the problem]

[0008] The information processing device of the present invention is characterized by having a determination means for determining whether the posture of a user viewing a display image has moved from a predetermined reference posture, and a display movement means for, when the determination means determines that the user's current posture has moved from the reference posture, moving the display position of the display image viewed by the user in a second direction opposite to the first direction in which the user's current posture has moved from the reference posture. [Effects of the Invention]

[0009] According to the present invention, the user's posture can be improved without interrupting work or the like. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of a basic configuration of an information processing apparatus. [Figure 2] FIG. 1 illustrates an example of a virtual work environment. [Figure 3] FIG. 10 is a diagram illustrating an example of a guidance screen. [Figure 4] FIG. 10 is a diagram showing an example of a posture (reference posture) that places less strain on the body. [Figure 5] FIG. 1 is a diagram showing an example of a posture (bad posture) that places a large burden on the body. [Figure 6] 10A and 10B are diagrams illustrating examples of movement of an image on a virtual monitor for each posture. [Figure 7] 1 is a diagram illustrating an example of a configuration including an information processing apparatus according to a first embodiment. [Figure 8] 4 is a flowchart of information processing according to the first embodiment. [Figure 9] 4 is a flowchart of a posture determination process according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a configuration including an information processing apparatus according to a second embodiment. [Figure 11] 10 is a flowchart of information processing according to the second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a configuration including an information processing apparatus according to a third embodiment. [Figure 13] FIG. 10 is an explanatory diagram of a movable range of the line of sight. [Figure 14] 10 is a flowchart of information processing according to the third embodiment. [Figure 15] 10 is a flowchart of a posture determination process according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). In the following embodiments, the same components will be described with the same reference numerals.

[0012] Here, an example will be described in which the information processing device according to this embodiment generates an image to be displayed on a display device such as a head-mounted type worn on the user's head or a glasses-type worn on the ears like eyeglasses. In the following embodiment, a head-mounted display (hereinafter referred to as an HMD) will be used as an example of a display device worn by a user, and a virtual work environment will be provided to the user using the HMD. In this embodiment, an example of a virtual work environment will be described in which a virtual monitor is displayed on the HMD, and results corresponding to input from an actual mouse or keyboard are displayed on the virtual monitor. Alternatively, a virtual keyboard may be displayed on the HMD along with a virtual keyboard, and input from the virtual keyboard may be accepted in accordance with the user's hand movements.

[0013] FIG. 1 is a block diagram showing an example of a basic configuration of this embodiment, including a sensor unit 101, an information processing unit 100, and a display unit 107. The sensor unit 101 is mounted on the HMD and detects changes in the position of the HMD at regular intervals. In this embodiment, since the HMD is worn on the user's head, the sensor unit 101 detects changes in the head position of the user wearing the HMD at regular intervals. Note that the regular interval when the sensor unit 101 detects changes in the user's head position is assumed to be short enough that the user does not notice any delay in detection. The sensor unit 101 outputs sensor data that detects changes in the user's head position to the information processing unit 100. The information processing unit 100 includes a position detection unit 102, an attitude storage unit 103, a determination unit 104, a display movement unit 105, and an image generation unit . The display unit 107 is mounted on the HMD and displays the image generated by the image generation unit 106 of the information processing unit 100 .

[0014] The position detection unit 102 of the information processing unit 100 sequentially detects the user's head position based on changes in the user's head position detected at regular intervals by the sensor unit 101. In this embodiment, the user's head position is used as information representing the user's posture. The posture storage unit 103 stores information about the user's head position detected by the position detection unit 102 when the user assumes a predetermined posture as position information of a reference posture (head position information of the reference posture) used when determining the user's posture. As will be described in detail later, in this embodiment, the reference posture is a posture that places minimal strain on the user's body while working in the virtual work environment.

[0015] The determination unit 104 determines whether the user's posture has changed from the reference posture based on the head position information of the reference posture stored in the posture storage unit 103 and the user's head position information detected by the position detection unit 102. In this embodiment, the determination unit 104 calculates a movement amount indicating how much the user's head position detected by the position detection unit 102 has moved relative to the head position of the reference posture, and determines whether the user's posture has deteriorated based on the movement amount. In this embodiment, a posture deterioration refers to a state in which the user's posture places a heavy burden on the body. As will be described in detail later, the determination unit 104 determines that the user's posture has deteriorated when the movement amount of the user's head position relative to the head position of the reference posture is equal to or greater than a predetermined threshold amount and this state continues for a predetermined threshold time or longer. The determination unit 104 then sends the user's posture determination result, the user's head position information detected by the position detection unit 102, and the head position information of the reference posture read from the posture storage unit 103 to the display movement unit 105.

[0016] Based on the user's posture determination result, the user's head position information, and the head position information of the reference posture, the display movement unit 105 generates display movement information for moving the position of the display image on the display unit 107 and sends the generated information to the image generation unit 106. In this embodiment, when the display movement unit 105 receives a posture determination result indicating that the user's posture has deteriorated from the determination unit 104, the display movement unit 105 generates display movement information for moving the display position of the virtual monitor generated by the image generation unit 106 and displayed on the display unit 107. That is, when the display movement unit 105 receives a posture determination result indicating that the user's posture has deteriorated, the display movement unit 105 generates display movement information for moving the display position of the virtual monitor in a second direction opposite to a first direction, which is the posture movement direction when the user's posture has changed from the reference posture. The display movement unit 105 then sends the user's head position information sequentially detected by the position detection unit 102 to the image generation unit 106, and when the display movement unit 105 generates display movement information due to the user's posture having deteriorated, the display movement unit 105 also sends the display movement information to the image generation unit 106.

[0017] The image generation unit 106 generates an image to be displayed on the screen of the display unit 107 based on the user's head position successively detected by the position detection unit 102. For example, if the user's head position changes, the image generation unit 106 generates an image to be displayed on the display unit 107 according to the orientation of the user's head resulting from the change in the user's head position. In addition, in this embodiment, if the determination unit 104 determines that the user's posture has deteriorated and display movement information is sent from the display movement unit 105, the image generation unit 106 sends to the display unit 107 an image of the virtual monitor whose display position has been moved in accordance with the display movement information. In this embodiment, if it is determined that the user's posture has deteriorated, the display movement unit 105 sends display movement information for moving the virtual monitor in a direction opposite to the direction of movement of the user's posture when the user's posture deteriorated. Therefore, the virtual monitor displayed on the display unit 107 moves in a direction opposite to the direction of movement of the user's posture when the user's posture deteriorated.

[0018] In this embodiment, the HMD is equipped with at least a sensor unit 101 and a display unit 107, and the information processing unit 100 is configured by an external information processing device (arithmetic device) connected to the HMD. Of course, the information processing unit 100 may be included in the HMD rather than an external information processing device. In addition, one or more of the position detection unit 102, posture storage unit 103, determination unit 104, display movement unit 105, and image generation unit 106 in the information processing unit 100 may be included in the HMD. Furthermore, although each component in the information processing unit 100 is realized by, for example, a CPU executing an information processing program according to this embodiment, each component may also be realized by an actual circuit configuration.

[0019] 2 is a diagram schematically illustrating how a virtual work environment is provided to a user in this embodiment by displaying a virtual monitor on the display unit 107. In FIG. 2, an example is shown in which a user 200 wearing an HMD 203 is performing virtual work while sitting in a real chair 210, for example. In the example of FIG. 2, the sensor unit 101 and display unit 107 of FIG. 1 are mounted on an HMD 203, and the information processing unit 100 is realized by a PC (personal computer) 204 executing an information processing program. The information processing unit 100 configured in the PC 204 generates display images of virtual monitors 201 and 202 that do not exist in real space and transmits them to the HMD 203. The display unit 107 of the HMD 203 displays the virtual monitors 201 and 202, allowing a user 200 wearing the HMD 203 to view the virtual monitors 201 and 202. In this case, the user 200 can operate the PC 204 while viewing the virtual monitors 201 and 202, similar to operating the PC 204 while viewing a physical monitor. Note that while FIG. 2 illustrates an example in which the information processing unit 100 is configured by the PC 204, the information processing unit 100 may be included in the HMD 203.

[0020] Here, in the virtual work environment exemplified in this embodiment, just as during normal work in a real space, if a user continues working for a long period of time, their posture may deteriorate without them realizing it. Therefore, the information processing unit 100 of this embodiment stores the head position of a reference posture when the user has a correct posture, and determines whether the user's posture has deteriorated based on the head position of the reference posture and the user's current head position. If it is determined that the user's posture has deteriorated, the information processing unit 100 moves the display position of the virtual monitor in the direction opposite to the direction of posture movement when the user's posture deteriorated. This allows the user working in the virtual work environment to recognize that their posture has deteriorated, and they can return their posture to the correct posture (improve their posture) without interrupting the virtual work.

[0021] Below, we will explain an overview of the information processing unit 100 of this embodiment, from acquiring and storing the user's correct reference posture, to determining whether the user's posture has deteriorated, and further moving the display position of the virtual monitor depending on the results of the posture deterioration determination. First, the image generation unit 106 of the information processing unit 100 generates an image for a guidance screen for prompting the user to take a reference posture, and transmits the image to the display unit 107. As a result, the guidance screen is displayed on the display unit 107.

[0022] FIG. 3 shows an example of a guidance screen 300 for prompting the user to adopt a reference posture. As mentioned above, the reference posture is a posture that places minimal strain on the user's body while working in a virtual work environment. In the case of a virtual work environment such as that illustrated in FIG. 2, a posture that places minimal strain on the user's body is assumed to be one in which, for example, the heels are in contact with the ground, the knees are at an angle of approximately 90 degrees, the center of gravity of the upper body and the head are positioned approximately on the vertical line of the ischial tuberosities, and the chin is tucked in. In addition, in the virtual work environment illustrated in FIG. 2, it is desirable that the height of the virtual monitor be such that the top of the monitor is at eye level.

[0023] The guidance screen 300 in FIG. 3 includes, for example, a text message describing a posture that places less strain on the user's body, and a start button 301. The start button 301 is a virtual button that the user presses when the user determines that the posture places less strain on the user's body. That is, the user determines their posture according to the message displayed on the guidance screen 300, and presses the start button 301 when they determine that they have placed less strain on the user's body. When the start button 301 is pressed, the information processing unit 100 acquires, as head position information of a reference posture, head position information of the user detected by the position detection unit 102 from sensor data detected by the sensor unit 101 at the time the button was pressed. Then, the information processing unit 100 stores the head position information of the reference posture in the posture storage unit 103 and then starts processing for the virtual task.

[0024] FIG. 4(a) is a simplified side view of the ideal reference posture that user 200 should assume in the virtual work environment illustrated in FIG. 2. FIG. 4(b) is a simplified rear view of user 200 in the reference posture. A dotted line 401 in FIGS. 4(a) and 4(b) represents a vertical line extending from the ischial bones of user 200, and it is desirable that the head of user 200 in the reference posture be positioned on this dotted line. A horizontal dotted line 402 in FIGS. 4(a) and 4(b) represents the height of the head of user 200 in the reference posture.

[0025] 5(a) to 5(c) are diagrams showing examples in which the posture of user 200 has deteriorated during virtual work. As described above, a state in which posture has deteriorated means that user 200's posture places a heavy burden on the body. Note that dotted lines 401 and 402 in FIGS. 5(a) to 5(c) are the same lines as those shown in FIGS. 4(a) and 4(b).

[0026] 5(a) is a side view of user 200 in a so-called hunched posture. At this time, the head position of user 200 moves in a first direction relative to the head position when the user is in the reference posture. That is, in the case of a hunched posture, the first direction in which the head position of user 200 moves is forward, closer to the front of the body than the head position indicated by dotted line 401 in the vertical direction when in the basic posture, and downward, lower than the head position indicated by dotted line 402 in the horizontal direction. In this posture, the weight of the head is supported by the neck, which places a heavy burden on the neck.

[0027] 5(b) is a side view of user 200 sitting shallowly in chair 210. The first direction in which the head position of user 200 moves at this time is rearward, toward the back of the body, from the head position indicated by vertical dotted line 401 in the reference posture, and downward, lower than the head position indicated by horizontal dotted line 402. In this posture, the center of gravity of the upper body and the head are shifted rearward from the vertical line of the sitting bones (dotted line 401), which places a heavy burden on the lower back.

[0028] 5(c) is a view from the back of user 200, showing the state in which user 200 is resting his chin on his right hand. The first direction in which the head position of user 200 moves in this case is to the right, toward the right arm, of the head position indicated by dotted line 401 in the vertical direction in the reference posture, and downward, lower than the head position indicated by dotted line 402 in the horizontal direction. In this posture, the spine is curved to the right, and the center of gravity of the upper body is shifted to the right of the vertical line of the ischial tuberosities (dotted line 401), which places a heavy burden on the lower back.

[0029] As can be seen from the examples in Figures 5(a) to 5(c), a state in which the posture of user 200 has deteriorated is a state in which the position of the user's head has moved from the head position indicated by the reference posture. Note that the examples of the deterioration of the user's posture shown in Figures 5(a) to 5(c) are merely examples, and postures that are determined to be a deterioration of the user's posture are not limited to these examples.

[0030] Therefore, the determination unit 104 of the information processing unit 100 of this embodiment compares the current head position information detected by the position detection unit 102 from the sensor data detected by the sensor unit 101 with the head position information of the reference posture stored in the posture memory unit 103. Furthermore, the determination unit 104 determines that the user's posture has deteriorated if the amount of movement of the current head position information relative to the head position information of the reference posture is equal to or greater than a predetermined threshold amount for a predetermined threshold time or longer. The determination unit 104 then sends the posture determination result, the head position information of the reference posture, and the current head position information to the display movement unit 105.

[0031] When the display movement unit 105 receives from the determination unit 104 the determination result that the user's posture has deteriorated, as well as information about the head position in the reference posture and the current head position, the display movement unit 105 generates display movement information indicating the direction and amount of movement for moving the display position of the virtual monitor. In this embodiment, the display movement unit 105 determines the direction of movement for the display position of the virtual monitor to be the opposite direction to the direction of movement from the head position in the reference posture to the current head position, and generates display movement information for moving the display position of the virtual monitor in that direction of movement. Note that the amount of movement for the display position of the virtual monitor may be the same as the amount of movement from the head position in the reference posture to the current head position, or may be an amount of movement that allows the user to notice the movement of the virtual monitor. The amount of movement that allows the user to notice the movement of the virtual monitor may be a predetermined amount of movement or may be an amount of movement arbitrarily set by the user.

[0032] When the determination unit 104 determines that the user's posture has deteriorated and display movement information is sent from the display movement unit 105, the image generation unit 106 moves the display position of the virtual monitor displayed on the display unit 107 based on the display movement information.

[0033] FIG. 6(a) is a diagram showing an example of movement of the display position of a virtual monitor in the virtual work environment of FIG. 2, corresponding to the posture of user 200 illustrated in FIG. 5(a). In the posture illustrated in FIG. 5(a), the first direction in which user 200's head position moves is forward from the head position indicated by dotted line 401 in the reference posture and downward from the head position indicated by dotted line 402. In addition, in the posture of FIG. 5(a), the amount of movement forward and downward from the head position indicated by dotted line 401 in the reference posture is assumed to be equal to or greater than a predetermined threshold amount. In this case, display movement unit 105 generates display movement information for moving the display positions of virtual monitors 201 and 202 in a second direction that is opposite to the forward and downward movement directions (first direction) of the head position, and sends the information to image generation unit 106. As a result, the display position of the virtual monitor generated by image generation unit 106 and displayed on display unit 107 moves from the position of virtual monitors 201 and 202 in the reference posture to the position of virtual monitors 601 and 602, as shown in FIG. 6(a). At this time, user 200 can notice that the display position of the virtual monitor has moved in a direction closer to him / her and upward relative to the position in the reference posture. In other words, user 200 can recognize that his / her posture has deteriorated due to his / her hunched back, and can return his / her posture to the reference posture by, for example, straightening his / her back.

[0034] FIG. 6(b) is a diagram showing an example of movement of the display position of the virtual monitor corresponding to the posture of user 200 illustrated in FIG. 5(b) in the virtual work environment of FIG. 2. In the case of the user posture illustrated in FIG. 5(b), the first direction in which the head position of user 200 moves is behind the head position indicated by dotted line 401 in the reference posture and below the head position indicated by dotted line 402. However, in the case of the posture illustrated in FIG. 5(b), the amount of movement downward from the head position indicated by dotted line 402 in the reference posture is equal to or greater than a predetermined threshold amount, but the amount of movement backward from the head position indicated by dotted line 401 is less than the predetermined threshold amount. In this case, display movement unit 105 generates display movement information for moving the display positions of virtual monitors 201 and 202 in a direction (second direction) opposite to the downward movement direction of the head position (first direction), and sends the information to image generation unit 106. As a result, the display positions of the virtual monitors generated by image generation unit 106 and displayed on display unit 107 move from the positions of virtual monitors 201 and 202 in the reference posture to the positions of virtual monitors 611 and 612, as shown in FIG. 6(b). At this time, user 200 can notice that the display positions of the virtual monitors have moved upward relative to the positions in the reference posture. In other words, user 200 can recognize that his / her posture has deteriorated because he / she is sitting shallowly in chair 210, and can return his / her posture to the reference posture by, for example, resitting in chair 210 so that he / she can easily see the virtual monitor.

[0035] FIG. 6(c) is a diagram showing an example of movement of the display position of the virtual monitor corresponding to the posture of user 200 illustrated in FIG. 5(c) in the virtual work environment of FIG. 2. In the case of the user posture illustrated in FIG. 5(c), the first direction in which the head position of user 200 moves is to the right of the head position indicated by dotted line 401 in the reference posture and downward of the head position indicated by dotted line 402. However, in the case of the posture illustrated in FIG. 5(c), the amount of movement to the right of the head position indicated by dotted line 401 in the reference posture is equal to or greater than a predetermined threshold amount, but the amount of movement downward of the head position indicated by dotted line 402 is less than the predetermined threshold amount. In this case, display movement unit 105 generates display movement information for moving the display positions of virtual monitors 201 and 202 in a direction (second direction) opposite to the movement direction (first direction) to the right of the head position, and sends the information to image generation unit 106. As a result, the display positions of the virtual monitors generated by the image generation unit 106 and displayed on the display unit 107 move from the positions of the virtual monitors 201 and 202 in the reference posture to the positions of the virtual monitors 621 and 622, as shown in FIG. 6(c). At this time, the user 200 can notice that the display positions of the virtual monitors have moved to the left relative to the positions in the reference posture. That is, the user 200 can recognize that his or her posture has deteriorated because he or she is resting his or her chin on his or her hand, and can, for example, stop resting his or her chin on his or her hand and return to the reference posture with a straight spine. Note that in FIG. 6(c), the amount of movement downward from the head position indicated by the dotted line 402 is less than a predetermined threshold amount, so the display positions of the virtual monitors are moved only to the left. On the other hand, for example, if the amount of downward movement when posture deteriorates is also equal to or greater than a predetermined threshold amount, the virtual monitors 621 and 622 may be displayed tilted in the opposite direction to the tilt of the user 200's head (displaying the virtual monitors 621 and 622 tilted to the left).

[0036] Next, we will explain the detailed configuration and processing of the information processing unit 100 according to this embodiment, which acquires and stores the reference posture as described above, determines whether the user's posture has deteriorated, and moves the display position of the virtual monitor in accordance with the results of the posture deterioration determination. First Embodiment FIG. 7 is a block diagram showing a detailed configuration of the information processing unit 100 according to the first embodiment, together with the configurations of the sensor unit 101 and the display unit 107. As shown in FIG. In the first embodiment, the sensor unit 101 includes an acceleration sensor 701 and an angular velocity sensor 702 .

[0037] The acceleration sensor 701 is mounted on the HMD 203 described above and is capable of detecting acceleration when the HMD 203 moves in each of the forward / backward, left / right, and up / down directions when an arbitrary position is set as the center position. When the user 200 wearing the HMD 203 moves his / her head, the acceleration sensor 701 detects changes in acceleration of the HMD 203 in each direction. Based on the sensor data output from the acceleration sensor 701, the information processing unit 100 calculates the amount of movement (displacement) of the user's head position at the current time relative to the head position in the reference posture. The angular velocity sensor 702 is mounted on the HMD 203, similar to the acceleration sensor 701, and is a sensor capable of detecting the amount of change in angle in each of the roll, pitch, and yaw directions of the HMD 203 when an arbitrary position is set as the center position. When a user wearing the HMD 203 rotates or tilts their head, the angular velocity sensor 702 detects the amount of change in angle in each direction of the HMD 203. Based on the sensor data of this angular velocity sensor 702, the information processing unit 100 calculates the movement (amount of movement in the tilt direction) of the current head position of the user relative to the head position in the reference posture.

[0038] In the first embodiment, the information processing unit 100 includes a transmission unit 716 in addition to the above-mentioned posture storage unit 103, determination unit 104, display movement unit 105, and image generation unit 106. A CPU 730 is connected to the information processing unit 100, sensor unit 101, and display unit 107, and an operation unit 740 is connected to the CPU 730. The CPU 730 may be included in the information processing unit 100.

[0039] Position detection unit 102 acquires sensor data from acceleration sensor 701 and angular velocity sensor 702, and detects the current position and tilt of the user's head based on the acquired sensor data. By detecting the position and tilt of the user's head in this way, image generation unit 106 at the subsequent stage can generate an image linked to the movement of the user's head, and display unit 107 can display an image linked to the movement of the user's head.

[0040] Posture storage unit 103 stores, as head position information for the reference posture, information on the user's head position when the user assumes a reference posture according to guidance screen 300 shown in Fig. 3 and then presses start button 301. Posture storage unit 103 is a readable and writable storage device, and may be built into information processing unit 100 or may be detachable.

[0041] Determination unit 104 acquires user head position information from position detection unit 102 and head position information in a reference posture from posture memory unit 103, and determines whether the user's posture has deteriorated based on these. Determination unit 104 determines that the user's posture has deteriorated if the user's current head position has moved by more than a threshold amount relative to the head position in the reference posture and this state continues for more than a predetermined threshold time. Determination unit 104 then sends the posture determination result, the head position information in the reference posture, and the current head position information to display movement unit 105.

[0042] When display movement unit 105 receives a determination result from determination unit 104 that the user's posture has deteriorated, it generates display movement information indicating the movement direction and movement amount of the virtual monitor based on head position information in the reference posture and information on the user's current head position. In this embodiment, display movement unit 105 determines the movement direction and movement amount in a second direction opposite to a first direction, which is the movement direction from the head position in the reference posture to the head position at the current time, generates display movement information indicating the movement direction and movement amount, and transmits it to image generation unit 106. Display movement unit 105 also sends information on the user's current head position to image generation unit 106.

[0043] The image generation unit 106 generates an image to be displayed on the display unit 107 based on the current head position information of the user. For example, if it is recognized from the current head position information of the user that the user is facing to the right, the image generation unit 106 generates an image to be displayed on the HMD 203 when the user faces to the right. The image data generated by the image generation unit 106 is transmitted from the transmission unit 716 to the display unit 107. Furthermore, when the determination unit 104 determines that the user's posture has deteriorated and display movement information is sent from the display movement unit 105, the image generation unit 106 generates an image in which the display position of the virtual monitor has been moved based on the display movement information. Then, the image data in which the display position of the virtual monitor has been moved is sent from the transmission unit 716 to the display unit 107.

[0044] The receiving unit 721 of the display unit 107 receives the image data sent from the transmitting unit 716 of the information processing unit 100 and sends it to the DA conversion unit 722 . The DA conversion unit 722 performs DA (Digital-to-Analog) conversion to convert the image data acquired from the receiving unit 721 into an analog electrical signal. The DA conversion unit 722 then sends the analog-converted electrical signal for each pixel to the display device 723. The display 723 has a plurality of pixels, for example, organic light-emitting diodes (OLEDs), arranged two-dimensionally, and displays an image by illuminating a light-emitting material corresponding to each pixel. That is, the display 723 acquires an electrical signal for each pixel sent from the DA converter 722 and determines the light-emitting intensity of the light-emitting material based on the intensity of the electrical signal corresponding to each pixel. The light of the image displayed on the display 723 then passes through an optical device such as a prism or half mirror and further through a lens before entering the user's eyes. The display 723 may be a liquid crystal display or the like.

[0045] For example, if the determining unit 104 determines that the user's posture has deteriorated and the image generating unit 106 generates an image in which the display position of the virtual monitor has been moved, the user will see the image in which the display position of the virtual monitor has been moved. This allows the user to recognize that their posture has deteriorated and to correct their posture accordingly.

[0046] The CPU 730 is connected via an internal bus to each component included in the sensor unit 101, the information processing unit 100, and the display unit 107. The CPU 730 controls the processing and operations performed by each component included in the sensor unit 101, the information processing unit 100, and the display unit 107. The operation unit 740 is connected to the CPU 730 via an internal bus. The operation unit 740 is made up of various operation members as an input unit that accepts user operations. The operation unit 740 is assumed to include various buttons on independent controllers on the left and right, hand gestures by the user, and eye gaze input. The operation unit 740 also includes what is displayed on the display unit 107 as a GUI (Graphical User Interface).

[0047] 8 is a flowchart showing the flow of information processing in the configuration of this embodiment, from acquiring and storing a reference posture to determining whether the user's posture has deteriorated and moving the display position of the virtual monitor in accordance with the determined posture deterioration. The processing of the flowchart shown in FIG. 8 is started when the CPU 730 detects a user operation that instructs execution of the information processing of this embodiment via the operation unit 740. The processing of each step of the flowchart in FIG. 8 is realized by the CPU 730 controlling each component of the sensor unit 101, the information processing unit 100, and the display unit 107.

[0048] First, in the process of step S802, the CPU 730 controls the image generation unit 106 of the information processing unit 100 to generate image data for a guidance screen 300 for prompting the user to take a reference posture, as shown in Fig. 3. The image data for the guidance screen 300 is transmitted from the transmission unit 716 to the display unit 107, and the guidance screen 300 is thereby displayed on the display 723 of the display unit 107.

[0049] Next, in the process of step S803, when start button 301 in guidance screen 300 is pressed via user operation on operation unit 740, CPU 730 determines that the user has assumed the reference posture. Then, CPU 730 stores the user's head position information detected by position detection unit 102 from sensor data detected by acceleration sensor 701 and angular velocity sensor 702 when start button 301 was pressed in posture storage unit 103 as head position information of the reference posture.

[0050] Next, in step S804, the determination unit 104 reads out the head position information of the reference posture stored in the posture storage unit 103. Next, in step S805, the position detection unit 102 acquires sensor data currently detected by the acceleration sensor 701 and angular velocity sensor 702 of the sensor unit 101. Furthermore, in step S806, the position detection unit 102 detects the current head position of the user based on the acquired sensor data, and sends the detected head position information to the determination unit 104.

[0051] Next, in step S807, the determination unit 104 determines the user's posture based on the current head position information of the user detected by the position detection unit 102 and the head position information of the reference posture read from the posture storage unit 103. Furthermore, in the process of step S808, the determination unit 104 determines whether the user's posture has worsened based on the user's posture determination result of step S807. If the determination unit 104 obtains a determination result that the user's posture has worsened, the process of the information processing unit 100 proceeds to step S809 and thereafter. On the other hand, if the determination result that the user's posture has worsened is not obtained, the process of the information processing unit 100 proceeds to step S816.

[0052] In step S809, display movement unit 105 calculates the movement direction and movement amount of the virtual monitor based on the result of the posture determination sent from determination unit 104, the head position information for the reference posture, and the head position information at the current time. Then, display movement unit 105 sends display movement information indicating the movement direction and movement amount of the virtual monitor to image generation unit 106. Next, in step S810, the image generation unit 106 generates an image of the virtual monitor whose display position has been moved based on the display movement information sent from the display movement unit 105.

[0053] Next, in step S811, the position detection unit 102 acquires current sensor data from the acceleration sensor 701 and angular velocity sensor 702 of the sensor unit 101. Then, in step S812, the position detection unit 102 detects the current head position of the user based on the acquired sensor data. Then, the position detection unit 102 sends the detected head position information to the determination unit 104.

[0054] Next, in the processing of step S813, the determination unit 104 determines the user's posture based on the current head position information acquired from the position detection unit 102 and the head position information of the reference posture read from the posture memory unit 103, as in the above-mentioned step S807. Furthermore, in the process of step S814, the determination unit 104 determines whether the user's posture has improved based on the user's posture determination result in step S813. That is, in step S814, the determination unit 104 determines whether the current head position of the user has returned to the head position of the reference posture. If the determination unit 104 determines that the current head position of the user has returned to the head position of the reference posture, that is, if it determines that the user's posture has improved, the process of the information processing unit 100 proceeds to step S815 and subsequent steps. On the other hand, if the user's posture has not improved, the process of the information processing unit 100 returns to step S809. Thereafter, the information processing unit 100 repeatedly executes the processes from step S809 to step S814 until the user's posture improves.

[0055] In step S815, the image generation unit 106 of the information processing unit 100 gradually returns the display position of the virtual monitor to the position corresponding to the original reference attitude. 9. For example, the CPU 730 determines to end the processing if an end instruction is input from the user via the operation unit 740. On the other hand, if an end instruction is not input from the user via the operation unit 740, the CPU 730 returns the processing to step S805 and controls the sensor unit 101, the information processing unit 100, and the display unit 107 to perform the subsequent processing.

[0056] As described above, the information processing unit 100 determines the user's posture based on the user's current head position information, and if the user's posture has deteriorated, it can encourage the user to improve their posture by moving the display position of the virtual monitor.

[0057] Fig. 9 is a detailed flowchart of the user posture determination process executed by the determination unit 104 in the process of step S807 in Fig. 8. The process of the flowchart shown in Fig. 9 starts when the determination unit 104 receives an instruction from the CPU 730 to determine the user posture. First, in the process of step S902, the determination unit 104 calculates the displacement of the current head position relative to the head position in the reference posture, based on the current head position information detected by the position detection unit 102 and the head position information in the reference posture read from the posture storage unit 103. In other words, the determination unit 104 calculates how much the current head position of the user has moved from the head position in the reference posture.

[0058] Next, in step S903, determination unit 104 determines whether the amount of movement of the current head position relative to the head position in the reference posture is equal to or greater than a preset threshold amount. The threshold amount used to determine the amount of movement is not limited to a preset value and can be set arbitrarily by the user. If determination unit 104 determines that the amount of movement of the current head position relative to the head position in the reference posture is less than the threshold amount, the process proceeds to step S905. On the other hand, if determination unit 104 determines that the amount of movement is equal to or greater than the threshold amount, the process proceeds to step S904.

[0059] If the process proceeds to step S904, the determination unit 104 determines whether the duration of the state in which the amount of movement of the head position at the current time is equal to or greater than the threshold amount is equal to or greater than a preset threshold time. Note that the threshold time used to determine the duration is not limited to a preset value and can be arbitrarily set by the user. If the process determines that the duration is less than the threshold time, the process by the determination unit 104 proceeds to step S905. On the other hand, if the process determines that the duration is equal to or greater than the threshold time, the process by the determination unit 104 proceeds to step S906.

[0060] If the process proceeds to step S906, the determination unit 104 determines that the user's posture has deteriorated. That is, the determination unit 104 determines that the user's posture has deteriorated if the amount of movement of the current head position is equal to or greater than the threshold amount and continues for equal to or greater than the threshold time. On the other hand, if the process proceeds to step S905, the determination unit 104 determines that the user's posture has not deteriorated. That is, the determination unit 104 determines that the user's posture has not deteriorated if the amount of movement of the head position at the current time is less than the threshold amount, or if the amount of movement of the head position at the current time is equal to or greater than the threshold amount but the duration of that state is less than the threshold time.

[0061] If determination unit 104 determines in step S905 that the user's posture has not deteriorated, it sends the head position information in the reference posture and the head position information at the current time to display movement unit 105, and then ends the processing of the flowchart in Fig. 9. If determination unit 104 determines in step S906 that the user's posture has deteriorated, it sends the display movement information generated by display movement unit 105, the head position information in the reference posture, and the head position information at the current time to display movement unit 105, and ends the processing in Fig. 9. According to the information processing unit 100 of the first embodiment, by performing the information processing as described above, it is possible to improve the user's posture when using a virtual work environment without interrupting the user's work.

[0062] <Second embodiment> Fig. 10 is a block diagram showing a detailed configuration of the information processing unit 100 according to the second embodiment, together with the configurations of the sensor unit 101 and the display unit 107. In the configuration in Fig. 10, the same components as those in Fig. 7 described above are denoted by the same reference numerals as those in Fig. 7, and description of those components will be omitted as appropriate.

[0063] In the second embodiment, an imaging unit 1000 is further added to the configuration of FIG. 7 described above. In the second embodiment, the imaging unit 1000 is, for example, a camera mounted on the HMD 203. Note that the HMD 203 is typically equipped with multiple cameras, and the imaging unit 1000 is one of these cameras that captures real space. The HMD 203 of this embodiment is a so-called video see-through HMD that displays an image of real space captured by the imaging unit 1000 on the display unit 107. In the case of a video see-through HMD 203, the display unit 107 is equipped with a Fresnel lens or a pancake lens that overlaps with the display 723. In the case of a video see-through HMD 203, the user can view the image displayed on the display 723 through these lenses. That is, in the second embodiment, the user can view a display image in which an image captured by the imaging unit 1000 of real space is superimposed with an image generated by the image generation unit 106 of the information processing unit 100 on the display unit 107 of the HMD 203.

[0064] The imaging unit 1000 includes an imaging sensor unit 1001 , an AD conversion unit 1002 , and a transmission unit 1003 . The image sensor unit 1001 is configured with a CMOS (Complementary Metal Oxide Semiconductor) image sensor or the like, and has multiple pixels arranged two-dimensionally. The image sensor unit 1001 receives light incident through a lens (not shown in Fig. 10), converts the intensity of the incident light into an analog electrical signal, and sends it to the AD conversion unit 1002. The AD conversion unit 1002 performs AD (Analog-Digital) conversion processing to convert the analog electrical signals sent from the image sensor unit 1001 into image data composed of digital signals. The image data generated by the AD conversion processing of the AD conversion unit 1002 is sent to the transmission unit 1003. The transmitting unit 1003 transmits the image data sent from the AD converting unit 1002 , that is, the image data obtained by capturing an image of the real space by the imaging unit 1000 , to the information processing unit 100 .

[0065] The information processing unit 100 of the second embodiment includes a receiving unit 1011 for receiving image data from the imaging unit 1000. The receiving unit 1011 of the information processing unit 100 receives image data captured by the imaging unit 1000 of real space. The image data is then sent to the image generation unit 106 via the downstream position detection unit 102 and subsequent components. The image generation unit 106 generates image data in which an image of a virtual monitor similar to that described in the first embodiment is superimposed on an image captured of real space, and sends the image data to the transmission unit 716. As a result, the display unit 107 displays an image in which the image of the virtual monitor is superimposed on the image captured of real space.

[0066] Fig. 11 is a flowchart showing the flow of information processing from capturing an image of a real space to displaying the image on the display unit 107 in the configuration according to the second embodiment shown in Fig. 10. The processing of the flowchart shown in Fig. 11 is started when the CPU 730 detects a user operation via the operation unit 740 to instruct execution of information processing of this embodiment. The processing of each step in the flowchart of Fig. 11 is realized by the CPU 730 controlling each component of the imaging unit 1000, the sensor unit 101, the information processing unit 100, and the display unit 107. In the flowchart of Fig. 11, processing steps that are the same as those in the flowchart of Fig. 8 described above are given the same reference numerals as in Fig. 8, and descriptions of those processing steps will be omitted as appropriate.

[0067] First, in the process of step S1102, the CPU 730 controls the image capturing unit 1000 to start capturing images of the real space. Then, in the process of step S1103, the CPU 730 controls the transmission unit 1003 to transmit image data captured by the image capturing unit 1000 to the information processing unit 100. As a result, in step S1104, the image generation unit 106 of the information processing unit 100 generates image data in which the image of the virtual monitor is superimposed on the image received from the imaging unit 1000, and sends the generated image data to the transmission unit 716. That is, the image data in which the image of the virtual monitor is superimposed on the image of the captured real space is sent to the display unit 107. As a result, the display 723 of the display unit 107 displays an image in which the image of the virtual monitor is superimposed on the image of the captured real space.

[0068] Next, in step S1105, the CPU 730 determines whether to start the function according to the first embodiment, that is, the same processing as that shown in the flowchart of Fig. 8. For example, if the CPU 730 detects a user operation via the operation unit 740 instructing the execution of information processing similar to that of the first embodiment, the CPU 730 advances the processing of the information processing unit 100 to step S802 and subsequent steps. On the other hand, if a user operation instructing the execution of information processing similar to that of the first embodiment is not detected, the processing of the imaging unit 1000 returns to S1103. When the process proceeds to step S802 or later, the information processing unit 100 performs the processes from step S802 to step S816 in FIG. 8 described above. As a result, in the second embodiment, as in the first embodiment, it is possible to improve the user's posture when using a virtual work environment without interrupting the user's work.

[0069] <Third embodiment> Fig. 12 is a block diagram showing a detailed configuration of the information processing unit 100 according to the third embodiment, together with the configurations of the sensor unit 101 and the display unit 107. In the configuration in Fig. 12, the same components as those in Fig. 7 described above are given the same reference numerals as in Fig. 7, and descriptions of those components will be omitted as appropriate. In the third embodiment, compared to the configuration of Figure 7 relating to the first embodiment described above, an eye gaze sensor 1201 is further added to the sensor unit 101, and an eye gaze detection unit 1211 is further added to the information processing unit 100.

[0070] The gaze sensor 1201 is mounted on the HMD 203 together with the acceleration sensor 701 and angular velocity sensor 702 described above, and detects the user's gaze. For example, the gaze sensor 1201 is disposed in a position near the eyes of the user wearing the HMD 203, and includes a visible light camera and an infrared camera. The gaze sensor 1201 transmits image data of the eyes of the user wearing the HMD 203 to the gaze detection unit 1211 of the information processing unit 100.

[0071] The gaze detection unit 1211 detects the user's gaze based on sensor data transmitted from the gaze sensor 1201. Specifically, the gaze detection unit 1211 detects the user's gaze based on the positional relationship of feature points such as the outer edge of the iris, the outer corner of the eye, and the pupil in an image of the user's eye captured by the gaze sensor 1201. Furthermore, the gaze detection unit 1211 calculates a gaze movable range that represents the range in which the user's gaze moves when looking at the virtual monitor in the reference posture, based on the head position in the reference posture acquired when the guidance screen 300 in FIG. 3 is displayed and the display position of the virtual monitor in that reference posture. In addition, in the case of the third embodiment, the posture storage unit 103 also stores information on the gaze movable range in the reference posture calculated by the gaze detection unit 1211 (hereinafter referred to as the reference gaze movable range) together with the head position information in the reference posture described above.

[0072] In the third embodiment, the information processing unit 100 determines the user's posture by combining the user's gaze information detected using sensor data from the gaze sensor 1201 with the user's current head position information detected by the position detection unit 102. Hereinafter, a description will be given of how to determine the user's posture by combining the user's gaze information detected using the sensor data of the gaze sensor 1201 and the current head position information of the user detected by the position detection unit 102.

[0073] Fig. 13(a) is a diagram showing a movable range (i.e., a reference gaze movable range) representing the range in which the gaze of the user 200 moves when the user 200 is in a standard posture and is looking at the virtual monitors 201 and 202 through the HMD 203. On the other hand, Fig. 13(b) is a diagram showing a gaze movable range representing the range in which the gaze of the user 200 moves when the user 200 is in a poor posture and is looking at the virtual monitors 201 and 202 through the HMD 203. Normally, when user 200 uses a monitor, it is desirable that the top of the monitor and the line of sight are at the same height. In other words, when user 200 uses a monitor, it is desirable that the user's line of sight movable range is within a range slightly below the horizontal. The reference line of sight movable range 1301 shown in Fig. 13(a) indicates the line of sight movable range when the top of virtual monitors 201, 202 and the line of sight are at approximately the same height.

[0074] However, when the head position of user 200 moves from the head position in the reference posture, the gaze movable range of the user at that time changes to a range different from the reference gaze movable range 1301. The gaze movable range 1311 shown in Fig. 13(b) shows an example of the range when the head position of user 200 moves due to the user's posture becoming hunched, for example, as shown in Fig. 5(a) described above. That is, in the example of Fig. 13(b), the gaze movable range 1311 expands upward by the amount that the head position of user 200 has moved downward from the head position in the reference posture, and the angle is wider by the amount that the head position has moved forward from the head position in the reference posture. Considering that the gaze movement range changes depending on the user's head position in this way, the determination unit 104 of the third embodiment determines the user's posture by combining the user's gaze information and head position information detected at the current time.

[0075] 14 is a flowchart showing the flow of information processing in the configuration according to the third embodiment shown in FIG. 12, from determining the user's posture using the user's current line of sight information to displaying an image on the display unit 107. The processing of the flowchart shown in FIG. 14 is started when the CPU 730 detects a user operation via the operation unit 740 to instruct execution of information processing according to this embodiment. The processing of each step in the flowchart of FIG. 14 is realized by the CPU 730 controlling the components of the sensor unit 101, the information processing unit 100, and the display unit 107. In the flowchart of FIG. 14, processing steps that are the same as those in the flowchart of FIG. 8 described above are given the same reference numerals as those in FIG. 8, and descriptions of those processing steps will be omitted as appropriate.

[0076] First, in step S802, the CPU 730 displays the guidance screen shown in FIG. Next, in the process of step S1403, the CPU 730 controls the gaze sensor 1201 to capture an image of the user's eyes and transmits the image data captured by the gaze sensor 1201 as sensor data to the gaze detection unit 1211 of the information processing unit 100. The gaze detection unit 1211 detects the user's gaze based on the sensor data from the gaze sensor 1201. Furthermore, the gaze detection unit 1211 calculates a reference gaze movable range based on the user's gaze information, head position information of the reference posture acquired in the same manner as in the first embodiment, and the display position of the virtual monitor in the reference posture. Then, the gaze detection unit 1211 stores the head position information of the reference posture and information about the reference gaze movable range in the posture storage unit 103.

[0077] Next, in step S1404, the determination unit 104 reads out the head position information of the reference posture and the information of the reference gaze movable range from the posture storage unit 103. Next, in step S1405, the position detection unit 102 acquires sensor data currently detected by the acceleration sensor 701 and angular velocity sensor 702. The gaze detection unit 1211 also acquires sensor data currently detected by the gaze sensor 1201.

[0078] Next, in step S1406, the position detection unit 102 detects the current head position of the user based on the sensor data acquired from the acceleration sensor 701 and the angular velocity sensor 702. Furthermore, the gaze detection unit 1211 detects the current gaze information of the user based on the sensor data acquired from the gaze sensor 1201. Then, the position detection unit 102 sends the current head position information and gaze information to the determination unit 104.

[0079] Next, in step S1407, determination unit 104 determines the user's posture based on the current head position information and gaze information of the user and the head position information and reference gaze movable range information of the reference posture read from posture storage unit 103. Details of posture determination processing that combines the head position information, gaze information, and reference gaze movable range will be described later.

[0080] Furthermore, in the process of step S1408, the determination unit 104 determines whether the user's posture has worsened based on the user's posture determination result of step S1407. If the determination unit 104 obtains a determination result that the user's posture has worsened, the process of the information processing unit 100 proceeds to step S1409 and thereafter. On the other hand, if the determination result that the user's posture has worsened is not obtained, the process of the information processing unit 100 proceeds to step S816.

[0081] In step S1409, display movement unit 105 calculates the direction and amount of movement of the virtual monitor based on the posture determination result sent from determination unit 104, the head position information and reference gaze movable range information for the reference posture, and the current head position information and gaze information. Then, display movement unit 105 sends display movement information indicating the direction and amount of movement of the virtual monitor to image generation unit 106. Next, in step S1410, the image generation unit 106 generates an image of the virtual monitor whose display position has been moved based on the display movement information sent from the display movement unit 105.

[0082] Next, in step S1411, the position detection unit 102 reacquires current sensor data from the acceleration sensor 701 and the angular velocity sensor 702. The gaze detection unit 1211 also reacquires current sensor data from the gaze sensor 1201.

[0083] Next, in the process of step S1412, the position detection unit 102 detects the current head position of the user based on the sensor data again acquired from the acceleration sensor 701 and the angular velocity sensor 702. Furthermore, the gaze detection unit 1211 detects the current gaze information of the user based on the sensor data again acquired from the gaze sensor 1201. Then, the position detection unit 102 sends the detected head position information and gaze information to the determination unit 104.

[0084] Next, in the processing of step S1413, the determination unit 104 determines the user's posture based on the user's current head position information and gaze information, and the head position information and reference gaze movable range information of the reference posture, as in the above-mentioned step S1407. Furthermore, in the process of step S1414, the determination unit 104 determines whether the user's posture has improved based on the user's posture determination result in step S1413. That is, in step S1414, the determination unit 104 determines whether the current user's head position has returned to the head position of the reference posture. If the determination unit 104 determines that the current user's head position has returned to the head position of the reference posture, that is, if it determines that the user's posture has improved, the process of the information processing unit 100 proceeds to step S815 and subsequent steps. On the other hand, if the user's posture has not improved, the process of the information processing unit 100 returns to step S1409. Thereafter, the information processing unit 100 repeatedly executes the processes from step S1409 to step S1414 until the user's posture improves. Note that the processes from step S815 onwards are the same as those in the flowchart of FIG. 8, and therefore description thereof will be omitted.

[0085] Fig. 15 is a detailed flowchart of a user posture determination process executed by the determination unit 104 according to the third embodiment as the process of step S1407 in Fig. 14. The process of the flowchart shown in Fig. 15 starts when the determination unit 104 receives an instruction to determine the user posture from the CPU 730. In the flowchart of Fig. 15, the same processing steps as those in the flowchart of Fig. 9 described above are assigned the same reference numerals as those in Fig. 9, and descriptions of those processing steps will be omitted as appropriate.

[0086] In the third embodiment, the process proceeds from step S902 to step S903 and then to step S904, and if it is determined that the current head position movement amount is equal to or greater than the threshold amount and has continued for more than the threshold time, the process of the determination unit 104 proceeds to step 1505. In step S1505, determination unit 104 determines whether the gaze movement range for the most recent duration equal to or greater than the threshold time exceeds the reference gaze movement range. If it is determined that the gaze movement range for the most recent duration equal to or greater than the threshold time exceeds the reference gaze movement range, the process by determination unit 104 proceeds to step S906. On the other hand, if it is determined that the gaze movement range is within the reference gaze movement range, the process by determination unit 104 proceeds to step S905.

[0087] Then, when the determination unit 104 obtains the determination result in step S905 or step S906, it sends the determination result to the display movement unit 105, and then ends the processing of the flowchart in FIG. By the processing described above, the information processing unit 100 of the third embodiment, like the previously described embodiments, can improve the user's posture when using a virtual work environment without interrupting the user's work.

[0088] In the above-described embodiments, examples have been given in which the user's posture is detected based on sensor data detected by the sensor unit 101 and a deterioration in the user's posture is determined, but the user's posture may be detected using other methods. For example, an image of the user performing virtual work may be captured using an external camera or the like, and a change in the user's posture may be detected from the captured image to determine a deterioration in the user's posture. Furthermore, in the above-described examples, an example has been given in which a virtual work environment is provided to the user, but the embodiments are not limited to virtual work environments and may be applied to various other virtual environments that use head-mounted or glasses-type display devices.

[0089] The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more of the functions. The above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0090] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a determination means for determining whether the posture of a user viewing a display image has moved from a predetermined reference posture; a display moving means for moving a display position of the display image viewed by the user in a second direction opposite to a first direction in which the current user's posture has moved from the reference posture when the determination means determines that the current user's posture has moved from the reference posture; An information processing device comprising: (Configuration 2) The information processing device according to configuration 1, wherein the determination means determines that the user's posture has moved from the reference posture when the state in which the current user's posture has moved from the reference posture continues for a predetermined threshold time or more. (Configuration 3) The information processing device according to configuration 1, characterized in that the determination means determines that the user's posture has moved from the reference posture when the amount of movement of the current user's posture from the reference posture is equal to or greater than a predetermined threshold amount. (Configuration 4) The information processing device described in Configuration 3 is characterized in that the determination means determines that the user's posture has moved from the reference posture when a state in which the amount of movement of the current user's posture from the reference posture is equal to or greater than the threshold amount continues for a predetermined threshold time or more. (Configuration 5) The information processing device according to any one of configurations 1 to 4, characterized in that, after moving the display position of the display image, when the current user posture becomes the reference posture, the display moving means gradually moves the display position of the display image to a display position corresponding to the reference posture. (Configuration 6) the display image is an image displayed on a head-mounted or glasses-type display device, The display device further includes a detection unit for detecting a head position of the user wearing the display device, The information processing device described in any one of configurations 1 to 5, characterized in that the determination means determines whether the user's current posture has moved from the reference posture based on the user's head position at the current time detected by the detection means and the head position corresponding to the reference posture. (Configuration 7) 7. The information processing device according to configuration 6, further comprising a storage means for storing the head position of the user detected by the detection means when the user is in the reference posture as the head position corresponding to the reference posture. (Configuration 8) the displayed image is an image on a virtual monitor; The information processing device according to configuration 6 or 7, wherein the display moving means moves the display position of the virtual monitor when the user's head position is in the reference posture in the second direction opposite to the first direction in which the user's head position at the current time has moved from the head position corresponding to the reference posture. (Configuration 9) an image generating means for generating the display image based on the current head position of the user; The information processing device according to any one of configurations 6 to 8, wherein when the determination means determines that the current posture of the user has moved from the reference posture, the display movement means moves the display position of the display image generated by the image generation means. (Configuration 10) 10. The information processing device according to configuration 9, wherein the image generating means generates an image in which the display image is superimposed on an image captured of real space. (Configuration 11) The information processing device according to any one of configurations 6 to 10, wherein the detection means detects the head position of the user based on sensor data detected by at least one of an acceleration sensor and an angular velocity sensor mounted on the display device. (Configuration 12) a gaze detection means for acquiring gaze information of the user; The information processing device described in any one of configurations 6 to 11, characterized in that the determination means determines whether the user's posture has moved from the reference posture based on the current head position of the user detected by the detection means and the gaze information acquired by the gaze detection means. (Configuration 13) The information processing device described in configuration 12, characterized in that the determination means determines whether the user's current posture has moved from the reference posture based on the user's head position at the current time, a gaze movement range representing the range in which the user's gaze moves when looking at the displayed image in the reference posture, and a gaze movement range corresponding to the user's gaze information at the current time detected by the gaze detection means. (Method 1) a determining step of determining whether the posture of the user viewing the displayed image has moved from a predetermined reference posture; a display moving step of moving a display position of the display image viewed by the user in a second direction opposite to a first direction in which the current user's posture has moved from the reference posture, when it is determined in the determining step that the current user's posture has moved from the reference posture; An information processing method comprising: (Program 1) 14. A program that causes a computer to function as the information processing device according to any one of configurations 1 to 13. [Explanation of symbols]

[0091] 100: Information processing unit, 101: Sensor unit, 102: Posture storage unit, 103: Determination unit, 104: Display movement unit, 105: Display unit

Claims

1. a determination means for determining whether the posture of a user viewing a display image has moved from a predetermined reference posture; a display moving means for moving a display position of the display image viewed by the user in a second direction opposite to a first direction in which the current user's posture has moved from the reference posture when the determination means determines that the current user's posture has moved from the reference posture; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the determination means determines that the user's posture has moved from the reference posture if the state in which the current user's posture has moved from the reference posture continues for a predetermined threshold time or more.

3. 2. The information processing device according to claim 1, wherein the determination means determines that the user's posture has moved from the reference posture if the amount of movement of the current user's posture from the reference posture is equal to or greater than a predetermined threshold amount.

4. The information processing device according to claim 3, characterized in that the determination means determines that the user's posture has moved from the reference posture if the amount of movement of the current user's posture from the reference posture is equal to or greater than the threshold amount for a predetermined threshold time or more.

5. 2. The information processing device according to claim 1, wherein, after moving the display position of the display image, if the current user posture becomes the reference posture, the display movement means gradually moves the display position of the display image to a display position corresponding to the reference posture.

6. the display image is an image displayed on a head-mounted or glasses-type display device, The display device further includes a detection unit for detecting a head position of the user wearing the display device, 2. The information processing device according to claim 1, wherein the determination means determines whether the user's current posture has moved from the reference posture based on the user's head position at the current time detected by the detection means and the head position corresponding to the reference posture.

7. 7. The information processing device according to claim 6, further comprising a storage means for storing the head position of the user detected by the detection means when the user is in the reference posture as the head position corresponding to the reference posture.

8. the displayed image is an image on a virtual monitor; 7. The information processing device according to claim 6, wherein the display moving means moves the display position of the virtual monitor when the user's head position is in the reference posture in the second direction opposite to the first direction in which the user's head position at the current time has moved from the head position corresponding to the reference posture.

9. an image generating means for generating the display image based on the current head position of the user; 7. The information processing device according to claim 6, wherein when the determination means determines that the user's current posture has moved from the reference posture, the display movement means moves the display position of the display image generated by the image generation means.

10. 10. The information processing apparatus according to claim 9, wherein the image generating means generates an image in which the display image is superimposed on an image captured in real space.

11. 7. The information processing apparatus according to claim 6, wherein the detection means detects the head position of the user based on sensor data detected by at least one of an acceleration sensor and an angular velocity sensor mounted on the display device.

12. a gaze detection means for acquiring gaze information of the user; The information processing device according to claim 6, characterized in that the determination means determines whether the user's posture has moved from the reference posture based on the current head position of the user detected by the detection means and the gaze information acquired by the gaze detection means.

13. The information processing device described in claim 12, characterized in that the determination means determines whether the user's current posture has moved from the reference posture based on the user's head position at the current time, a gaze movement range representing the range in which the user's gaze moves when looking at the displayed image in the reference posture, and a gaze movement range corresponding to the user's gaze information at the current time detected by the gaze detection means.

14. a determining step of determining whether the posture of the user viewing the displayed image has moved from a predetermined reference posture; a display moving step of moving a display position of the display image viewed by the user in a second direction opposite to a first direction in which the current user's posture has moved from the reference posture, when it is determined in the determining step that the current user's posture has moved from the reference posture; An information processing method comprising:

15. Computer, a determination means for determining whether the posture of a user viewing a display image has moved from a predetermined reference posture; a display moving means for moving a display position of the display image viewed by the user in a second direction opposite to a first direction in which the current user's posture has moved from the reference posture when the determination means determines that the current user's posture has moved from the reference posture; A program that causes the device to function as an information processing device having the above.

Images