Wearable display device, information processing device, and image presenting method

Abstract

This wearable display device includes a control unit and a display unit. The control unit acquires a three-dimensional movement model of body movements of a person on the basis of output values acquired from a plurality of motion sensors. The person wears a plurality of the motion sensors. The control unit generates a reproduced moving image of the three-dimensional movement model as seen from any direction. The display unit allows a wearer to visually recognize the reproduced moving image together with a surrounding scene.

Description

Wearable display device, information processing device, and image presentation method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2023-13419, filed on January 31, 2023, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to a wearable display device, an information processing device, and an image presentation method.

[0003] There is a demand for providing useful information by analyzing a user's arbitrary behavior. To analyze the arbitrary behavior, it is necessary to accurately detect the user's movement during the arbitrary behavior. To accurately detect the user's movement, for example, a motion capture method using an inertial measurement unit and multiple cameras has been proposed (see Patent Document 1).

[0004] Japanese Patent Application Laid-Open No. 2020-201183

[0005] A wearable display device according to a first aspect comprises: a control unit that acquires a three-dimensional motion model of the physical movements of a person wearing a plurality of motion sensors based on output values ​​acquired from the plurality of motion sensors, and generates a reproduced moving image of the three-dimensional motion model viewed from any direction; and a display unit that allows the wearer to view the reproduced moving image together with the scenery that exists behind the surroundings.

[0006] An information processing device according to a second aspect includes: a control unit that estimates the physical movement of a person wearing multiple motion sensors based on output values ​​obtained from the multiple motion sensors and a learning model that has learned the relationship between the output values ​​of the multiple motion sensors and the physical movement of the person wearing the multiple motion sensors, and generates a three-dimensional motion model of the physical movement; and a communication unit that transmits the three-dimensional motion model to a wearable display device.

[0007] An image presentation method according to a third aspect includes obtaining output values ​​from a plurality of motion sensors, obtaining a three-dimensional motion model of the body movement of a person wearing the plurality of motion sensors based on the output values, generating a reproduced moving image of the three-dimensional motion model viewed from any direction, and allowing the wearer to view the reproduced moving image together with the surrounding scenery.

[0008] 6 is a conceptual external view showing a specification aspect of a presentation system including a wearable display device and an information processing device according to a first embodiment. FIG. 7 is a block diagram showing a schematic configuration of the presentation system of FIG. 1. FIG. 8 is a flowchart for explaining a first image presentation process executed by a control unit of the wearable display device of FIG. 2. FIG. 9 is a flowchart for explaining a second image presentation process executed by a control unit of the wearable display device of FIG. 2. FIG. 10 is a conceptual external view showing a specification aspect of a wearable display device and a sensor device according to a second embodiment. FIG. 11 is a block diagram showing a schematic configuration of the wearable display device and the sensor device of FIG. 5. FIG. 12 is a flowchart for explaining a third image presentation process executed by a control unit of the wearable display device of FIG.

[0009] Hereinafter, embodiments of a wearable display device and an information processing device to which the present disclosure is applied will be described with reference to the drawings.

[0010] As shown in FIG. 1 , a presentation system 11 including a wearable display device 10 according to the first embodiment of the present disclosure may be configured to include a plurality of sensor devices 12 and the wearable display device 10. The presentation system 11 may further include an information processing device 13. The plurality of sensor devices 12 may be assumed to be worn by a subject (person) ts. The wearable display device 10 may be assumed to be worn on the head of a human body. When the wearer of the wearable display device 10 is the same as the subject ts, the plurality of sensor devices 12 and the wearable display device 10 may constitute the presentation system 11.

[0011] The multiple sensor devices 12 may output output values ​​corresponding to the physical movement of the subject ts. The multiple sensor devices 12 may transmit the output values ​​as signals to the wearable display device 10 or the information processing device 13. The wearable display device 10 or the information processing device 13 may generate a three-dimensional movement model of the subject ts based on the output values. The wearable display device 10 allows the generated three-dimensional movement model to be visually recognized by the wearer, the subject ts.

[0012] The plurality of sensor devices 12 may include at least a head sensor device 14 , an arm sensor device 15 , and a leg sensor device 16 .

[0013] The head sensor device 14 may be intended to be worn on the head of the subject ts. The head sensor device 14 may be worn on the head in any manner, such as earphones or a headband. Alternatively, the head sensor device 14 may be integrated with the wearable display device 10.

[0014] The arm sensor device 15 may be intended to be worn on the arm of the subject ts. The arm sensor device 15 may be intended to be worn on either the left or right arm. For example, the arm sensor device 15 may be intended to be worn on the left arm. The arm sensor device 15 may be worn anywhere on the arm. For example, the arm sensor device 15 is worn on the wrist. The arm sensor device 15 may be worn on the arm by a band, clip, or the like.

[0015] The leg sensor device 16 may be assumed to be attached to the leg of the subject ts. The leg sensor device 16 may be assumed to be attached to either the left or right leg. For example, the leg sensor device 16 may be assumed to be attached to the right leg. The leg sensor device 16 may be attached anywhere on the leg. For example, the leg sensor device 16 is attached to the ankle. The leg sensor device 16 may be attached to the arm by a band, clip, etc.

[0016] As shown in FIG. 2 , the sensor device 12 may include a communication unit 17 , a sensor unit 18 , a storage unit 19 , and a control unit 20 .

[0017] The communication unit 17 may include, for example, at least one communication module capable of communicating with the wearable display device 10 via a communication line that may be wired or wireless. The communication unit 17 may also include, for example, at least one communication module capable of communicating with the information processing device 13 via a communication line that may be wireless. The communication module is a communication module compatible with the communication line standard. The communication line standard capable of communicating with the wearable display device 10 is, for example, a short-range wireless communication standard including Bluetooth (registered trademark), infrared, and NFC (Near Field Communication). The communication line standard capable of communicating with the information processing device 13 is, for example, a mobile communication standard such as 4G (4th Generation) or 5G (5th Generation).

[0018] The sensor unit 18 includes at least a motion sensor. The motion sensor includes, for example, a triaxial inertial sensor. The triaxial inertial sensor includes, for example, a triaxial acceleration sensor and a triaxial gyro sensor. The sensor unit 18 detects output values ​​corresponding to the movement of the sensor device 12. In a configuration in which the sensor unit 18 is a triaxial inertial sensor, the output values ​​are triaxial accelerations and triaxial angular velocities in a local coordinate system determined for each sensor device 12.

[0019] The storage unit 19 may include any of a semiconductor memory, a magnetic memory, and an optical memory. Examples of the semiconductor memory include a RAM (Random Access Memory) and a ROM (Read Only Memory). Examples of the RAM include a SRAM (Static Random Access Memory) and a DRAM (Dynamic Random Access Memory). Examples of the ROM include an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 19 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 19 may store data used in the operation of the sensor device 12 and data obtained by the operation of the sensor device 12. The storage unit 19 stores, for example, system programs, application programs, embedded software, and the like.

[0020] The control unit 20 may be configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor may be a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 20 may execute processes related to the operation of the sensor device 12 while controlling each part of the sensor device 12.

[0021] The control unit 20 controls the communication unit 17 to periodically transmit the output value detected by the sensor unit 18 as a signal to the wearable display device 10 or the information processing device 13. In a configuration in which the output value is transmitted to the wearable display device 10, the control unit 20 may transmit the output value by broadcast or unicast. In a configuration in which the output value is transmitted by unicast, the control unit 20 may determine the wearable display device 10 worn by the subject ts of the sensor device 12 as the communication destination. For example, the control unit 20 may determine the wearable display device 10 as the communication destination based on reception of authentication information indicating that the wearable display device 10 has been authenticated as the communication partner of the wearable display device 10 based on operation input to an input unit of the wearable display device 10. In a configuration in which the output value is transmitted to the information processing device 13, the control unit 20 may recognize identification information of the wearable display device 10 worn by the subject ts of the sensor device 12. For example, the control unit 20 may transmit the identification information received together with the authentication information to the information processing device 13.

[0022] 1, the wearable display device 10 is a device that is worn on the head and allows the wearer to view an image from in front of the wearer's eyes. Examples of the wearable display device 10 include AR goggles, smart glasses, contact lenses, ICLs (Implantable Contact Lenses), and eye patches.

[0023] 2 , the wearable display device 10 includes a display unit 21 and a control unit 22. The wearable display device 10 may further include a communication unit 23, an input unit 24, a storage unit 25, and an imaging unit 28.

[0024] The communication unit 23 may include, for example, at least one communication module capable of communicating with the sensor device 12 via a communication line that may be wired or wireless. The communication unit 23 may also include at least one communication module capable of communicating with the information processing device 13 via a communication line that may be wireless. The communication module is a communication module compatible with the communication line standard. The communication line standard capable of communicating with the sensor device 12 is, for example, a short-range wireless communication standard such as Bluetooth (registered trademark), infrared, or NFC. The communication unit 23 may acquire an output value of the sensor unit 18 from the sensor device 12. The communication line standard capable of communicating with the information processing device 13 is, for example, a mobile communication standard such as 4G or 5G.

[0025] The input unit 24 may include at least one input interface that detects a user's operation input. The input interface may be, for example, a physical key, a capacitance key, a pointing device, a touch screen that is integrated with the display of the display unit 21, a microphone, or the like.

[0026] The display unit 21 is any display device that allows the wearer to view augmented reality. More specifically, the display unit 21 is a display device that allows the wearer to view any image along with a scene that exists behind the surface on which the augmented reality is viewed. The any image includes a reproduced moving image, which will be described later. The display unit 21 may be a transmissive display device or a non-transmissive display device.

[0027] A transmissive display device is a display device that emits arbitrary image light while transmitting a scene behind it. In a transmissive display device, the image light may be emitted by drawing an image on the display surface. Alternatively, in a transmissive display device, the image light may be emitted toward an area where the wearer's pupils are expected to be located using a prism or the like. In a configuration in which the display unit 21 is a transmissive display device, the wearer can simultaneously view the surrounding scene and the image by viewing the image light drawn on the display surface together with the scene through the display unit 21. A non-transmissive display device may have a camera provided behind the display surface. A non-transmissive display device is a display device that displays an arbitrary image superimposed on an image captured by the camera. In a configuration in which the display unit 21 is a non-transmissive display device, an image in which an image viewed as augmented reality is superimposed on an image of the scene captured by the camera that would be visible if the wearer were not wearing the display device is displayed on the display unit 21. In this manner, the display unit 21, which is a non-transmissive display device, allows the wearer to simultaneously view the surrounding scene and the image.

[0028] The memory unit 25 may include any of semiconductor memory, magnetic memory, and optical memory. The semiconductor memory is, for example, RAM, ROM, etc. The RAM is, for example, SRAM, DRAM, etc. The ROM is, for example, EEPROM, etc. The memory unit 25 may function as a main memory device, an auxiliary memory device, or a cache memory. The memory unit 25 may store data used in the operation of the wearable display device 10 and data obtained by the operation of the wearable display device 10. The memory unit 25 stores, for example, system programs, application programs, embedded software, etc.

[0029] The imaging unit 28 may be provided on the back side of the display surface of the display unit 21. The imaging unit 28 may acquire an image of the surroundings of the wearer ts.

[0030] The control unit 22 may be configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor may be a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA, an ASIC, etc. The control unit 22 may execute processes related to the operation of the wearable display device 10 while controlling each part of the wearable display device 10.

[0031] The control unit 22 acquires a three-dimensional motion model of the body motion of the subject ts based on output values ​​of the motion sensors of each of the multiple sensor devices 12. As will be described later, the control unit 22 may acquire the three-dimensional motion model by generating it based on the output values ​​acquired via the communication unit 23. Alternatively, the control unit 22 may acquire, via the communication unit 23, a three-dimensional motion model generated based on the output values ​​acquired by the information processing device 13. The three-dimensional motion model is, for example, a wireframe model, a solid model, or a surface model that represents the three-dimensional structure of the overall appearance of the moving subject ts.

[0032] To generate the three-dimensional motion model, the control unit 22 estimates a posture angle of at least one of the body parts of the subject ts using the output value and a learning model. The learning model is machine-learned so that when an output value is input, it outputs an estimated value of a posture angle of at least one of the body parts of the subject ts. In the first embodiment, the control unit 22 uses the transformer described in "Ashish Vaswani et al., "Attention Is All You Need," June 12, 2017, arXiv:1706.03762v5 (cs.CL)" as the learning model. The transformer is capable of processing time-series data. However, the learning model is not limited to the transformer. The control unit 22 may use a learning model generated by machine learning based on any machine learning algorithm.

[0033] The control unit 22 may estimate time-series posture angles of body parts throughout the entire body of the subject ts using a learning model. The control unit 22 may generate a three-dimensional movement model based on the time-series posture angles of body parts throughout the entire body of the subject ts. The three-dimensional movement model is, for example, a walking movement model. The control unit 22 may generate the three-dimensional movement model as an animation. The control unit 22 may generate the three-dimensional movement model by scaling a human-shaped model of a predetermined size according to the height of the subject ts. The height of the subject ts may be detected by an operation input to the input unit 24 and stored in the memory unit 25.

[0034] The control unit 22 generates a reproduced video in which the three-dimensional motion model is viewed from an arbitrary direction. The reproduced video may be a video in which the three-dimensional motion model is viewed from behind or from the side. The control unit 22 may determine the arbitrary direction, for example, by operating the input unit 24. Alternatively, in a configuration in which the wearable display device 10 has a gaze sensor that detects the wearer's gaze, the control unit 22 may determine the direction in which the three-dimensional motion model is viewed depending on the wearer's gaze. For example, when the wearer's gaze is forward, the control unit 22 generates a reproduced video in which the three-dimensional motion model is viewed from behind in the direction of the gaze. Alternatively, for example, when the wearer's gaze is lateral, the control unit 22 generates a reproduced video in which the three-dimensional motion model is viewed from the side in the direction of the gaze. Alternatively, in a configuration in which the wearable display device 10 has a motion sensor, the control unit 22 may determine the direction in which the three-dimensional motion model is viewed depending on the wearer's orientation.

[0035] The control unit 22 may determine differences in the posture angles of each body part in the three-dimensional motion model from the posture angles of each body part in an ideal motion identical to that of the three-dimensional motion model, and may add marks such as circles to indicate parts with differences in the reproduced video.

[0036] When the control unit 22 recognizes the wearer's destination, it may generate a reproduced moving image to guide the wearer along a route from the position of the wearable display device 10 on the ground to the destination. The control unit 22 may detect the position of the wearable display device 10 on the ground using a Global Navigation Satellite System (GNSS) possessed by the wearable display device 10. The control unit 22 may recognize the wearer's destination, for example, by reading out a destination acquired based on an operation input to the input unit 24 and stored in the memory unit 25. Furthermore, for example, the control unit 22 may generate a reproduced moving image so that the wearer is visually recognized as being located in the direction toward the destination at a fork in a road, etc.

[0037] When the control unit 22 recognizes the target arrival time at the destination, the control unit 22 may calculate the target speed of the wearer based on the distance from the position of the wearable display device 10 on the ground to the destination, the current time, and the target arrival time. The control unit 22 may recognize the target arrival time of the wearer by, for example, reading out the target arrival time acquired based on an operation input to the input unit 24 and stored in the memory unit 25.

[0038] Furthermore, when the control unit 22 recognizes the target arrival time at the destination, the control unit 22 may calculate the walking speed of the three-dimensional movement model. The control unit 22 may calculate the walking speed using any method. For example, the control unit 22 may calculate the walking speed based on the output values ​​of the head sensor device 14 and the leg sensor device 16. Alternatively, for example, the control unit 22 may calculate the walking speed based on the walking cycle in the three-dimensional movement model and the height of the subject ts.

[0039] Furthermore, the control unit 22 may calculate the difference between the target speed and the walking speed of the three-dimensional movement model, in other words, the relative speed of the target speed with respect to the walking state of the three-dimensional movement model. The control unit 22 may generate a reproduced moving image that moves according to the target speed. In other words, the control unit 22 may generate a reproduced moving image such that at least one of the visual position and size in the moving image varies according to the relative speed.

[0040] The control unit 22 controls the display unit 21 so that the generated reproduced video is visible to the wearer. For example, when a reproduced video of a three-dimensional movement model viewed from behind is displayed in front of the wearer's line of sight, the control unit 22 may control the position at which the reproduced video is displayed on the display unit 21 so that the reproduced video appears to be walking on the ground in front of the wearer's line of sight. When a reproduced video of a three-dimensional movement model viewed from the side is displayed to the side of the wearer's line of sight, the control unit 22 may control the position at which the reproduced video is displayed on the display unit 21 so that the reproduced video appears to be walking on the ground to the side of the wearer's line of sight.

[0041] The control unit 22 may perform image recognition processing on the image captured by the imaging unit 28. The control unit 22 may identify the position of the ground through the image recognition processing. The control unit 22 may control the position at which the reproduced moving image is displayed on the display unit 21 based on the identified position of the ground. In a configuration in which the display unit 21 is a transmissive display device, the control unit 22 may identify the position at which the reproduced moving image is displayed on the display unit 21 by previously associating coordinates set in the image captured by the imaging unit 28 with coordinates set within a range visible by the wearer through the display unit 21. On the other hand, in a configuration in which the display unit 21 is a non-transmissive display device, the control unit 22 may identify the position at which the reproduced moving image is displayed on the display unit 21 by previously associating coordinates set in the image captured by the imaging unit 28 with coordinates set on the display surface of the display unit 21. The control unit 22 may control the position at which the reproduced moving image is displayed so that the soles of the feet of the three-dimensional movement model in the reproduced moving image touch the ground at the position of the ground identified through the image recognition processing.

[0042] The control unit 22 may control the display unit 21 so that the reproduced moving image of the wearer walking on the ground faces in the direction toward the destination at a fork in the road on the way to the wearer's destination, which is recognized by image recognition processing.

[0043] The control unit 22 may use image recognition processing to identify the situation, including the presence of stairs or sloping ground in front of the wearer's line of sight. The situation may be, for example, whether the ground the wearer is traveling on is flat, uphill, or downhill. For example, when controlling the display of a reproduced video image of the three-dimensional movement model viewed from behind in front of the wearer's line of sight, the control unit 22 may control the position at which the reproduced video image is displayed in accordance with the identified situation.

[0044] Furthermore, the control unit 22 may control the display unit 21 so that at least one of the viewing position and size of the reproduced video of the wearer walking on the ground varies depending on the calculated relative speed. For example, when a reproduced video of the three-dimensional movement model viewed from behind is displayed in front of the wearer's line of sight, the control unit 22 may control the display position of the reproduced video so that the relative distance between the wearer and the reproduced video moves away from or towards the wearer over time depending on the relative speed. For example, when a reproduced video of the three-dimensional movement model viewed from the side is displayed to the side of the wearer's line of sight, the control unit 22 may control the display position of the reproduced video so that the relative distance between the wearer and the reproduced video moves diagonally forward or diagonally behind the wearer over time depending on the relative speed.

[0045] When the three-dimensional motion model is a model of the walking motion of the subject ts, the control unit 22 may determine an ideal walking speed for the subject ts. The control unit 22 may determine the ideal walking speed depending on the subject ts.

[0046] For example, the control unit 22 may determine the ideal walking speed based on an operation input detected by the input unit 24. The operation input for the ideal walking speed may be an input that directly specifies the speed. Alternatively, the operation input for the ideal walking speed may be an input that selects from a plurality of predetermined ideal walking patterns. Ideal walking speeds are set for the plurality of walking patterns. Therefore, the walking speed may be determined by selecting the ideal walking pattern of the subject ts from the plurality of patterns.

[0047] Alternatively, for example, the control unit 22 may determine the ideal walking speed of the subject ts based on the height of the subject ts. To calculate the ideal walking speed, the control unit 22 may read out from the storage unit 25 a table or a calculation formula indicating the relationship between the ideal walking speed and the height of the subject ts.

[0048] The control unit 22 may calculate the walking speed of the three-dimensional movement model. The control unit 22 may calculate the walking speed using any method. For example, the control unit 22 may calculate the walking speed based on the output values ​​of the head sensor device 14 and the leg sensor device 16. Furthermore, for example, the control unit 22 may calculate the walking speed based on the walking period in the three-dimensional movement model and the height of the subject ts.

[0049] The control unit 22 may calculate the difference between the ideal walking speed and the walking speed of the three-dimensional movement model, in other words, the relative speed of the ideal walking state with respect to the walking state of the three-dimensional movement model. The control unit 22 may generate a reference moving image that moves according to the relative speed. In other words, the control unit 22 may generate the reference moving image such that at least one of the visual position and size in the moving image varies according to the relative speed.

[0050] The reference video may be a reproduced video. In other words, the control unit 22 may change the viewing position and size according to the relative speed so that the relative distance between the wearer and the reproduced video changes over time. For example, when the relative speed is positive, the control unit 22 generates a reproduced video that moves away from the wearer at a speed according to the magnitude of the relative speed and gradually becomes smaller. Also, for example, when the relative speed is negative, the control unit 22 generates a reproduced video that moves toward the wearer at a speed according to the magnitude of the relative speed and becomes larger. Alternatively, the reference video may be a graphic shape, such as a circle, different from the reproduced video.

[0051] The control unit 22 may control the display unit 21 so that the generated reference video is visible to the wearer. For example, when a reference video of a three-dimensional movement model viewed from behind is displayed in front of the wearer's line of sight, the control unit 22 may control the position at which the reference video is displayed on the display unit 21 so that the reference video appears to be walking on the ground in front of the wearer's line of sight. When a reference video of a three-dimensional movement model viewed from the side is displayed to the side of the wearer's line of sight, the control unit 22 may control the position at which the reference video is displayed on the display unit 21 so that the reference video appears to be walking on the ground to the side of the wearer's line of sight.

[0052] The control unit 22 may control the position at which the reference moving image is displayed on the display unit 21 based on the position of the ground identified by the image recognition processing on the image captured by the imaging unit 28. In a configuration in which the display unit 21 is a transmissive display device, the control unit 22 may identify the position at which the reference moving image is displayed on the display unit 21 by previously associating coordinates set in the image captured by the imaging unit 28 with coordinates set within a range visible by the wearer through the display unit 21. On the other hand, in a configuration in which the display unit 21 is a non-transmissive display device, the control unit 22 may identify the position at which the reference moving image is displayed on the display unit 21 by previously associating coordinates set in the image captured by the imaging unit 28 with coordinates set on the display surface of the display unit 21. The control unit 22 may control the position at which the reference moving image is displayed so that the soles of the feet of the three-dimensional motion model in the reference moving image touch the ground at the position of the ground identified by the image recognition processing.

[0053] The control unit 22 may control the display unit 21 so that the reference video image of the wearer walking on the ground faces in the direction toward the destination at a fork in the road on the way to the wearer's destination, which is recognized by image recognition processing.

[0054] The control unit 22 may identify the situation, including the presence of stairs or sloping ground in front of the line of sight, through image recognition processing. For example, when controlling the display of a reference video image in which the three-dimensional motion model is viewed from behind in front of the wearer's line of sight, the control unit 22 may control the display position of the reference video image in accordance with the identified situation.

[0055] Furthermore, the control unit 22 may control the display unit 21 so that at least one of the viewing position and size of the reference video of the wearer walking on the ground varies depending on the calculated relative speed. For example, when a reference video of a three-dimensional movement model viewed from behind is displayed in front of the wearer's line of sight, the control unit 22 may control the display position of the reference video so that the relative distance between the wearer and the reference video moves away from or closer to the wearer over time depending on the relative speed. For example, when a reference video of a three-dimensional movement model viewed from the side is displayed to the side of the wearer's line of sight, the control unit 22 may control the display position of the reference video so that the relative distance between the wearer and the reference video moves diagonally forward or diagonally behind the wearer over time depending on the relative speed.

[0056] The control unit 22 may generate a sample video. The sample video is a video showing the same sample body movement as the reproduced video. The sample video may be a video of any type. For example, the sample video may be a video captured from a specific direction of a subject ts capable of performing ideal movement. Furthermore, for example, the sample video may be a video of a three-dimensional movement model viewed from any direction, similar to the reproduced video. The three-dimensional movement model that serves as the basis for the sample video may be generated based on output values ​​obtained by having a subject ts capable of performing ideal movement wear the sensor device 12. Furthermore, for example, the three-dimensional movement model that serves as the basis for the sample video may be generated based on the movement state of a subject ts capable of performing ideal movement obtained by motion capture. In a configuration in which the sample video is a video of a three-dimensional movement model viewed from any direction, the control unit 22 may generate the sample video so as to show the body movement viewed from the same direction as the reproduced video.

[0057] The control unit 22 may control the display unit 21 so that the generated sample moving image is visible to the wearer. For example, when a sample moving image of a three-dimensional movement model viewed from behind is displayed in front of the wearer's line of sight, the control unit 22 may control the position at which the sample image is displayed on the display unit 21 so that the sample moving image appears to be walking on the ground in front of the wearer's line of sight. When a sample moving image of a three-dimensional movement model viewed from the side is displayed to the side of the wearer's line of sight, the control unit 22 may control the position at which the sample image is displayed on the display unit 21 so that the sample moving image appears to be walking on the ground to the side of the wearer's line of sight.

[0058] The control unit 22 may control the position at which the sample moving image is displayed on the display unit 21 based on the position of the ground identified by the image recognition processing on the image captured by the imaging unit 28. In a configuration in which the display unit 21 is a transmissive display device, the control unit 22 may identify the position at which the sample moving image is displayed on the display unit 21 by previously associating coordinates set in the image captured by the imaging unit 28 with coordinates set within a range visible by the wearer via the display unit 21. On the other hand, in a configuration in which the display unit 21 is a non-transmissive display device, the control unit 22 may identify the position at which the sample moving image is displayed on the display unit 21 by previously associating coordinates set in the image captured by the imaging unit 28 with coordinates set on the display surface of the display unit 21. The control unit 22 may control the position at which the sample moving image is displayed so that the soles of the feet of the three-dimensional movement model in the sample moving image touch the ground at the position of the ground identified by the image recognition processing.

[0059] The control unit 22 may control the display unit 21 so that at a fork in the road on the way to the wearer's destination, as recognized by image recognition processing, the sample moving image of the wearer walking on the ground faces in the direction toward the destination.

[0060] The control unit 22 may identify the situation, including the presence of stairs or sloping ground in front of the line of sight, through image recognition processing. For example, when controlling the display of a sample moving image of a three-dimensional motion model viewed from behind in front of the wearer's line of sight, the control unit 22 may control the position at which the sample moving image is displayed in accordance with the identified situation.

[0061] The control unit 22 may generate information based on output values ​​acquired from the plurality of sensor devices 12. The information generated based on the output values ​​includes, for example, walking speed, stride length, left and right stance times, left and right sway information, and forward body tilt information. The control unit 22 may control the display unit 21 to display at least one of the output values ​​acquired from the plurality of sensor devices 12 and information generated based on the output values. Furthermore, the control unit 22 may determine, based on an input to the input unit 24, whether or not to display at least one of the output values ​​acquired from the plurality of sensor devices 12 and information generated based on the output values ​​on the display unit 21.

[0062] The information processing device 13 includes a communication unit 26 and a control unit 27 .

[0063] The communication unit 26 may include at least one communication module capable of communicating with the wearable display device 10 and the sensor device 12 via a communication line that includes wireless communication. The communication module is a communication module compatible with the standard of the communication line. The standard of the communication line capable of communicating with the wearable display device 10 and the sensor device 12 is, for example, a mobile communication standard such as 4G or 5G.

[0064] The control unit 27 may be configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor may be a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA, an ASIC, etc. The control unit 27 may execute processes related to the operation of the information processing device 13 while controlling each part of the information processing device 13.

[0065] The control unit 27 may generate a three-dimensional motion model of the body movement of the subject ts based on the output values ​​of the motion sensors of each of the multiple sensor devices 12. The control unit 27 may generate the three-dimensional motion model based on the output values ​​in the same manner as the control unit 22 of the wearable display device 10. That is, the control unit 27 generates the three-dimensional motion model of the body movement by estimating the body movement of the subject ts wearing the sensor devices 12 using the output values ​​and a learning model. The control unit 27 controls the communication unit 26 to transmit the three-dimensional motion model to the wearable display device 10 corresponding to the identification information received together with the output values.

[0066] Next, the first image presentation process executed by the control unit 22 of the wearable display device 10 in the first embodiment will be described with reference to the flowchart of Fig. 3. The first image presentation process starts, for example, every time an output value is acquired from the sensor device 12.

[0067] In step S100, the control unit 22 estimates posture angles of body parts throughout the entire body of the subject ts based on output values ​​acquired from the multiple sensor devices 12. After the estimation, the process proceeds to step S101.

[0068] In step S101, the control unit 22 associates the attitude angle estimated in step S100 with the time and stores it in the storage unit 25. After storage, the process proceeds to step S102.

[0069] In step S102, the control unit 22 reads out the attitude angles associated with each time in the time series going back from the most recent time to the most recent time from the storage unit 25. After reading, the process proceeds to step S103.

[0070] In step S103, the control unit 22 generates a three-dimensional motion model of the subject ts based on the time-series posture angles read out in step S102. After generation, the process proceeds to step S104.

[0071] In step S104, the control unit 22 determines an arbitrary direction in which to view the three-dimensional motion model generated in step S103. After the determination, the process proceeds to step S105.

[0072] In step S105, the control unit 22 determines whether or not there is a difference between the posture angles of each body part in the three-dimensional motion model generated in step S103 and the posture angles of each body part in an ideal motion identical to that of the three-dimensional motion model generated in step S103. If there is a difference, the process proceeds to step S106. If there is no difference, the process proceeds to step S107.

[0073] In step S106, the control unit 22 determines the body part determined to be different in step S105, in other words, the location on the reproduced moving image where the mark is to be superimposed. After the determination, the process proceeds to step S107.

[0074] In step S107, the control unit 22 determines whether the wearer's destination is recognized. If the destination is recognized, the process proceeds to step S108. If the destination is not recognized, the process proceeds to step S113.

[0075] In step S108, the control unit 22 determines the viewing position of the reproduced moving image based on the location on the ground and the destination of the wearable display device 10. After the determination, the process proceeds to step S109.

[0076] In step S109, the control unit 22 determines whether or not the target arrival time at the destination, the accuracy of which was determined in step S107, is recognized. If the target arrival time at the destination is recognized, the process proceeds to step S110. If the target arrival time is not recognized, the process proceeds to step S113.

[0077] In step S110, the control unit 22 calculates the target speed based on the distance from the position of the wearable display device 10 on the ground to the destination, the current time, and the target arrival time determined whether the recognition was correct or not in step S109. After the calculation, the process proceeds to step S111.

[0078] In step S111, the control unit 22 calculates the walking speed of the three-dimensional movement model generated in step S103. After the calculation, the process proceeds to step S112.

[0079] In step S112, the control unit 22 calculates the relative speed of the target speed calculated in step S110 with respect to the walking speed of the three-dimensional movement model calculated in step S111. After the calculation, the process proceeds to step S113.

[0080] In step S113, the control unit 22 generates a reproduced video based on at least the three-dimensional motion model generated in step S103 and the arbitrary direction determined in step S104. Furthermore, if the control unit 22 determined a marker superimposition location in step S106, it superimposes a marker at the marker superimposition location on the generated reproduced video. Furthermore, if the control unit 22 determined a viewing position in step S108, it sets the position at which the reproduced video is to be viewed to the viewing position. Furthermore, if the control unit 22 calculated a relative velocity in step S112, it sets the viewing position of the reproduced video to change in accordance with the relative velocity in response to the time change of the reproduced video. After generation, the process proceeds to step S114.

[0081] In step S114, the control unit 22 controls the display unit 21 so as to project the reproduced moving image generated in step S113 from the display unit 21. After the projection, the first image presentation process ends.

[0082] Next, the second image presentation process executed by the control unit 22 of the wearable display device 10 in the first embodiment will be described with reference to the flowchart of Fig. 4. The second image presentation process is started, for example, every time an output value is acquired from the sensor device 12.

[0083] In step S200, the control unit 22 determines an ideal walking speed for the subject ts. After the determination, the process proceeds to step S201.

[0084] In step S201, the control unit 22 calculates the walking speed of the three-dimensional movement model generated in step S103 of the latest first image presentation process. After the calculation, the process proceeds to step S202.

[0085] In step S202, the control unit 22 calculates the relative speed of the ideal walking speed calculated in step S200 with respect to the walking speed of the three-dimensional movement model calculated in step S201. After the calculation, the process proceeds to step S203.

[0086] In step S203, the control unit 22 generates a reference moving image that is visually recognized as moving at the relative speed calculated in step S202. After generation, the process proceeds to step S204.

[0087] In step S204, the control unit 22 controls the display unit 21 so as to project the reference moving image generated in step S203 from the display unit 21. After the projection, the second image presentation process ends.

[0088] The wearable display device 10 of the first embodiment configured as described above acquires a three-dimensional motion model of the subject ts's body movement based on output values ​​acquired from multiple motion sensors, generates a reproduced motion image of the three-dimensional motion model viewed from any direction, and allows the wearer to view the reproduced motion image together with the scenery behind it. With this configuration, the wearable display device 10 allows the wearer, who is the same as the subject ts, to instantly observe the subject ts's body movement. Therefore, the wearable display device 10 allows the wearer to instantly visually perceive the subject ts's body movement, thereby contributing to improving the subject ts's body movement. Furthermore, with the above configuration, the wearable display device 10 allows the wearer to view the reproduced motion image together with the surrounding scenery, thereby reducing the risk of focusing solely on the screen and losing attention to the surroundings, as occurs, for example, when using a smartphone. In this way, the wearable display device 10 allows the wearer to appropriately recognize the detected motion.

[0089] Furthermore, the wearable display device 10 of the first embodiment generates a reference moving image that moves according to the difference between the ideal walking speed and the walking speed of the three-dimensional movement model, and allows the wearer to visually recognize the reference moving image. With this configuration, the wearable display device 10 can make the wearer recognize an appropriate walking speed. Therefore, the wearable display device 10 can encourage the wearer to walk at an appropriate walking speed.

[0090] Furthermore, in the wearable display device 10 of the first embodiment, the reference moving image is a reproduced moving image. With this configuration, the wearable display device 10 reduces the number of places to look at for perceiving walking speed and physical movement to one place, compared to a configuration in which the reference moving image and the reproduced moving image are displayed separately, thereby reducing the possibility of distraction.

[0091] Next, a wearable display device according to a second embodiment of the present disclosure will be described. The second embodiment differs from the first embodiment in that a plurality of sensor devices and a wearable display device are worn separately. The second embodiment will be described below, focusing on the differences from the first embodiment. Note that parts having the same configuration as those in the first embodiment will be assigned the same reference numerals.

[0092] As shown in Fig. 5, in the second embodiment, similarly to the first embodiment, it may be assumed that the plurality of sensor devices 12 are worn by a subject (person) ts. Unlike the first embodiment, in the first embodiment, it may be assumed that the wearable display device 100 is worn on the head of an observer os, who is different from the subject. In the second embodiment, the configuration and function of the sensor device 12 are the same as in the first embodiment. The subject ts is, for example, a patient or a person receiving care. The observer os is, for example, a doctor, nurse, or caregiver.

[0093] 6, the wearable display device 100 may be configured to include a display unit 21, a communication unit 23, an input unit 24, a storage unit 25, an imaging unit 28, and a control unit 220, as in the first embodiment. The configurations and functions of the display unit 21, the communication unit 23, the input unit 24, the storage unit 25, and the imaging unit 28 in the second embodiment are the same as in the first embodiment. In the second embodiment, the control unit 220 may be capable of additionally executing processing different from that of the control unit 22 in the first embodiment.

[0094] In the second embodiment, the control unit 220, like the first embodiment, acquires a three-dimensional motion model of the body motion of the subject ts based on the output values ​​of the motion sensors of each of the multiple sensor devices 12. In the second embodiment, the control unit 220, like the first embodiment, may generate a reproduced moving image of the three-dimensional motion model viewed from any direction.

[0095] In the second embodiment, unlike the first embodiment, the control unit 220 may have a subject observation mode as an operation mode. The operation mode may be switched by an operation input to the input unit. In the subject observation mode, the control unit 220 may control the display unit 21 to display a reproduced moving image based on images acquired by the imaging unit 28.

[0096] Specifically, when the image acquired by the imaging unit 28 includes the entire human body as a partial image, the control unit 220 may generate, as a reproduced moving image, a three-dimensional motion model viewed from a direction different from the orientation of the entire body included as a partial image in the image. The three-dimensional motion model viewed from a different direction may be the entire body or a part of the body.

[0097] The control unit 220 may use multi-class classification using an SVM (Support Vector Machine) or a discriminant model based on semi-supervised learning to determine the orientation of the entire body in the image. The control unit 220 may also determine a determination different from the orientation of the entire body based on a predetermined rule. For example, if the entire body is facing forward of the subject ts, the side of the subject ts may be set as the gaze direction for creating the three-dimensional motion model. For example, if the entire body is facing to the side of the subject ts, the front of the subject may be set as the gaze direction for creating the three-dimensional motion model. Alternatively, the gaze direction for creating the three-dimensional motion model may be determined by the gestures or voice of the observer wearing the wearable display device 100. The control unit 220 may control the display unit 21 to display the generated reproduced moving image.

[0098] The control unit 220 may determine whether to display the entire body or a part of the body based on an operation input detected by the input unit 24. The control unit 220 may also determine which part of the body to display based on a predetermined rule.

[0099] Or, specifically, when an image acquired by the imaging unit 28 includes a partial image of a part of a person's body, the control unit 220 may generate a three-dimensional motion model of the whole body as viewed from an arbitrary direction as a reproduced moving image. The control unit 220 may control the display unit 21 to display the generated reproduced moving image. The arbitrary direction may be determined by a gesture or voice of the observer wearing the wearable display device 100, or may be determined based on a predetermined rule.

[0100] Next, the third image presentation process executed by the control unit 220 of the wearable display device 100 in the second embodiment will be described with reference to the flowchart of Fig. 7. The third image presentation process is started, for example, every time an image is acquired from the imaging unit 28.

[0101] In step S300, the control unit 220 determines whether the entire body is included in the image acquired from the imaging unit 28. If the entire body is included, the process proceeds to step S301. If neither the body itself nor the entire body is included, the process proceeds to step S302.

[0102] In step S301, the control unit 220 determines the orientation of the entire body included in the image. Furthermore, the control unit 220 generates a three-dimensional motion model viewed from a direction different from the determined orientation as a reproduced video. Furthermore, the control unit 220 controls the display unit 21 to display the reproduced video. After the reproduced video is displayed, the third image presentation process ends.

[0103] In step S302, the control unit 220 generates a three-dimensional motion model of the entire body as viewed from an arbitrary direction as a reproduced moving image. Furthermore, the control unit 220 controls the reproduced moving image to be displayed on the display unit 21. After the reproduced moving image is displayed, the third image presentation process ends.

[0104] The wearable display device 100 of the second embodiment, configured as described above, also acquires a three-dimensional motion model of the subject ts's body movements based on output values ​​acquired from multiple motion sensors, generates a reproduced motion image of the three-dimensional motion model viewed from any direction, and allows the wearer to view the reproduced motion image together with the scene behind it. For example, medical professionals such as doctors and caregivers often desire to view the posture and movements of patients, people requiring assistance, etc. from the left and right sides in addition to viewing them from the front or rear. In response to such demands, the wearable display device 100 configured as described above allows a wearer os, separate from the subject ts, to instantly observe the subject ts's body movements. Therefore, the wearable display device 100 allows the wearer os to view the subject ts, whom they directly view, from a different field of view.

[0105] Moreover, the wearable display device 100 of the second embodiment further includes an imaging unit 28 that acquires images of the surroundings of the wearer os, and the display unit 21 displays a reproduced moving image based on the image acquired by the imaging unit 28. With this configuration, the wearable display device 100 displays a reproduced moving image based on the field of view directly recognized by the wearer os, and can therefore automatically recognize the orientation of the subject ts without any operational input by the wearer os.

[0106] Furthermore, in the wearable display device 100 of the second embodiment, when the image acquired by the imaging unit 28 includes the entire human body, the display unit 21 displays, as the reproduced moving image, a three-dimensional motion model viewed from a direction different from the orientation of the body included in the image. With this configuration, the wearable display device 100 can allow the wearer os to simultaneously view the subject ts in a field of view different from the view that the wearer os can directly view.

[0107] Furthermore, in the wearable display device 100 of the second embodiment, when the image acquired by the imaging unit 28 is of a part of a human body, the display unit 21 displays a three-dimensional motion model of the whole body as viewed from any direction as a reproduced moving image. With this configuration, the wearable display device 100 can allow the wearer os to view the whole body of the subject ts even in a situation where the wearer os is in close contact with the subject ts and can only directly view a part of the subject ts.

[0108] In one embodiment, (1) the wearable display device comprises: a control unit that acquires a three-dimensional motion model of the body movement of a person wearing a plurality of motion sensors based on output values ​​acquired from the plurality of motion sensors, and generates a reproduced moving image of the three-dimensional motion model viewed from any direction; and a display unit that allows the wearer to view the reproduced moving image together with the surrounding scenery.

[0109] (2) In the wearable display device of (1) above, the reproduced moving image is a moving image of the three-dimensional motion model viewed from behind or from the side.

[0110] (3) The wearable display device of (2) above further includes an imaging unit that acquires an image of the wearer's surroundings, and the control unit identifies the ground conditions from the surrounding images captured by the imaging unit and controls the display of the reproduced moving image so that the soles of the three-dimensional movement model touch the ground based on the identified ground conditions.

[0111] (4) In the wearable display device of (3) above, the control unit identifies, as the ground condition, whether the ground is flat, uphill, or downhill.

[0112] (5) In the wearable display device of (1) to (4) above, when the three-dimensional motion model is a model of the walking motion of the person, the control unit generates a reference moving image that moves according to the difference between an ideal walking speed and the walking speed of the three-dimensional motion model, and the display unit allows the wearer to view the reference moving image.

[0113] (6) In the wearable display device of (5), the reference moving image is the reproduced moving image.

[0114] (7) In the wearable display device according to (5) or (6), the control unit determines the ideal walking speed depending on the person.

[0115] (8) In any of the wearable display devices (1) to (7) above, the control unit generates a sample moving image that is a sample movement of the same body movement as the reproduced moving image, and the display unit allows the wearer to view the sample moving image.

[0116] (9) In the wearable display device of (8) above, the sample moving image is a body movement viewed from the same direction as the reproduced moving image.

[0117] (10) The wearable display device of (1) above further includes an imaging unit that acquires an image of the wearer's surroundings, and the display unit displays the reproduced moving image based on the image acquired by the imaging unit.

[0118] (11) In the wearable display device of (10) above, when the image acquired by the imaging unit includes the entire human body, the display unit displays, as the reproduced moving image, a three-dimensional motion model viewed from a direction different from the orientation of the body included in the image.

[0119] (12) In the wearable display device of (10) or (11) above, when the image acquired by the imaging unit is of a part of a human body, the display unit displays a three-dimensional motion model of the whole body as seen from any direction as the reproduced moving image.

[0120] (13) In the wearable display device according to any one of (1) to (12) above, the display unit displays at least one of output values ​​obtained from the plurality of motion sensors and information generated based on the output values.

[0121] (14) In the wearable display device of (13) above, the information generated based on the output values ​​includes walking speed, stride length, left and right stance times, left and right sway information, and forward body lean information.

[0122] Also, (15) an information processing device includes: a control unit that estimates the physical movement of a person wearing the multiple motion sensors based on output values ​​obtained from the multiple motion sensors and a learning model that has learned the relationship between the output values ​​of the multiple motion sensors and the physical movement of the person wearing the multiple motion sensors, and generates a three-dimensional motion model of the physical movement; and a communication unit that transmits the three-dimensional motion model to a wearable display device.

[0123] Also, (16) an image presentation method acquires output values ​​from a plurality of motion sensors, acquires a three-dimensional motion model of the body movement of a person wearing the plurality of motion sensors based on the output values, generates a reproduced moving image of the three-dimensional motion model viewed from any direction, and allows the wearer to view the reproduced moving image together with the surrounding scenery.

[0124] The above has described embodiments of the wearable display device 10 and the information processing device 13. However, embodiments of the present disclosure can also be embodied as a method or program for implementing the device, as well as a storage medium on which a program is recorded (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card).

[0125] Furthermore, the implementation form of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may also be in the form of a program module incorporated into an operating system. Furthermore, the program may or may not be configured so that all processing is performed solely by the CPU on the control board. The program may also be configured so that part or all of it is executed by another processing unit mounted on an expansion board or expansion unit added to the board as needed.

[0126] The drawings illustrating the embodiments of the present disclosure are schematic, and the dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0127] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art could make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0128] All of the features described in this disclosure and / or all steps of all of the disclosed methods or processes may be combined in any combination except combinations in which these features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by an alternative feature serving the same, equivalent, or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is only one example of a generic series of identical or equivalent features.

[0129] Furthermore, embodiments of the present disclosure are not limited to the specific configurations of any of the above-described embodiments, but rather extend to any novel feature or combination thereof described herein, or any novel method or process step or combination thereof described herein.

[0130] In the present disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configuration. In the present disclosure, the configurations distinguished by descriptions such as "first" and "second" can have their numbers exchanged. For example, the first image presentation process can exchange the identifiers "first" and "second" with the second image presentation process. The exchange of identifiers is performed simultaneously. The configurations remain distinguished even after the identifier exchange. Identifiers may be deleted. A configuration from which an identifier has been deleted is distinguished by a symbol. The descriptions of identifiers such as "first" and "second" in the present disclosure should not be used solely to interpret the order of the configurations or to justify the existence of an identifier with a smaller number.

[0131] REFERENCE SIGNS LIST 10, 100 Wearable display device 11 Presentation system 12 Sensor device 13 Information processing device 14 Head sensor device 15 Arm sensor device 16 Leg sensor device 17 Communication unit 18 Sensor unit 19 Memory unit 20 Control unit 21 Display unit 22, 220 Control unit 23 Communication unit 24 Input unit 25 Memory unit 26 Communication unit 27 Control unit 28 Imaging unit os Observer ts Subject

Claims

1. a control unit that acquires a three-dimensional motion model of the body motion of a person wearing a plurality of motion sensors based on output values ​​acquired from the plurality of motion sensors, and generates a reproduced moving image of the three-dimensional motion model viewed from an arbitrary direction; a display unit that allows the wearer to visually recognize the reproduced moving image together with the surrounding scenery. Wearable display device.

2. The wearable display device according to claim 1, The reproduced moving image is a moving image of the three-dimensional motion model viewed from behind or from the side. Wearable display device.

3. The wearable display device according to claim 2, Further, an imaging unit that acquires an image of the wearer's surroundings is provided, The control unit identifies a ground condition from the surrounding image captured by the imaging unit, and controls the display of the reproduced moving image so that the soles of the three-dimensional movement model touch the ground based on the identified ground condition. Wearable display device.

4. The wearable display device according to claim 3, The control unit identifies, as the ground condition, whether the ground is flat, uphill, or downhill. Wearable display device.

5. The wearable display device according to any one of claims 1 to 4, when the three-dimensional motion model is a model of the walking motion of the person, the control unit generates a reference moving image that moves according to a difference between an ideal walking speed and a walking speed of the three-dimensional motion model; The display unit allows the wearer to visually recognize the reference moving image. Wearable display device.

6. The wearable display device according to claim 5, The reference video sequence is the reproduction video sequence. Wearable display device.

7. The wearable display device according to claim 5, The control unit determines the ideal walking speed in accordance with the person. Wearable display device.

8. The wearable display device according to any one of claims 1 to 4, the control unit generates a sample video that is a sample exercise of the same body exercise as the reproduced video; The display unit allows the wearer to visually recognize the sample moving image. Wearable display device.

9. The wearable display device according to claim 8, The sample video is a body movement viewed from the same direction as the reproduced video. Wearable display device.

10. The wearable display device according to claim 1, Further, an imaging unit that acquires an image of the wearer's surroundings is provided, The display unit displays the reproduced moving image based on the image acquired by the imaging unit. Wearable display device.

11. The wearable display device according to claim 10, When the image acquired by the imaging unit includes the entire body of the person, the display unit displays, as the reproduced moving image, a three-dimensional motion model viewed from a direction different from the orientation of the body included in the image. Wearable display device.

12. The wearable display device according to claim 10 or 11. When the image acquired by the imaging unit is of a part of a human body, the display unit displays a three-dimensional motion model of the whole body as seen from an arbitrary direction as the reproduced moving image. Wearable display device.

13. The wearable display device according to any one of claims 1 to 4. The display unit displays at least one of output values ​​obtained from the plurality of motion sensors and information generated based on the output values. Wearable display device.

14. The wearable display device according to claim 13. The information generated based on the output values ​​includes walking speed, stride length, left and right stance time, left and right sway information, and forward body lean information. Wearable display device.

15. a control unit that estimates the physical movement of a person wearing the plurality of motion sensors based on output values ​​acquired from the plurality of motion sensors and a learning model that has learned the relationship between the output values ​​of the plurality of motion sensors and the physical movement of the person wearing the plurality of motion sensors, and generates a three-dimensional motion model of the physical movement; a communication unit that transmits the three-dimensional motion model to a wearable display device. Information processing device.

16. Obtain output values ​​from multiple motion sensors, obtaining a three-dimensional motion model of the body motion of the person wearing the plurality of motion sensors based on the output values; generating a reproduced motion image of the three-dimensional motion model viewed from an arbitrary direction; The reproduced moving image is visually recognized by the wearer together with the surrounding scenery. Image presentation method.