Furniture-type apparatus for estimating leg posture of seated user, ancillary device and system

Abstract

To provide furniture-type apparatus, etc. for estimating a leg posture of a seated user.SOLUTION: Furniture-type apparatus includes: one or more lower leg sensors disposed at a position below a seat surface of the furniture-type apparatus and measuring a planar depth distribution around lower legs of a seated user; and posture estimation means for estimating and outputting a posture of an upper leg and a lower leg of each of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor. The posture estimation means outputs a position of a knee and / or a direction of the upper leg as the posture of the upper leg, and a position of an ankle, a direction of the lower leg, and / or a bending angle of a knee joint as the posture of the lower leg. The posture estimation means estimates the posture of the upper leg and the lower leg of the seated user based on an ankle position vector and a lower leg direction vector estimated from the depth distribution of the lower leg sensor and a ratio of the lengths of the upper leg and the lower leg that is set in advance or can be set later.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology for a device and a system that serves as a user interface in a virtual space, and in particular to a technology for manipulating the posture of a virtual object (an avatar or a character) in a virtual reality space or a game space. [Background technology]

[0002] Virtual reality is a technology that projects a three-dimensional virtual space in the user's field of vision. While viewing the virtual space, the user can move within the virtual space by grasping and operating a stick device or a game controller. In particular, the user can obtain a sense of immersion from the field of vision by wearing an HMD (Head Mounted Display) terminal on the head. This HMD terminal may access a virtual space server via a network, or may be equipped with a virtual space server itself. The HMD terminal detects the movement of the user's head using a posture sensor, and controls the user's field of vision and position in the virtual space according to the measured value. In addition, the user's own hand movement can be detected by a depth camera that captures the user's field of vision, and the operation can be reflected in the virtual space.

[0003] The virtual space server may be a virtual space server (Dedicated Server) located on the Internet, or a virtual space server (Listen Server) located locally. A service that provides a virtual space may be, for example, a social VR platform such as VRChat (registered trademark), Metaverse (registered trademark), or Second Life (registered trademark).

[0004] Conventionally, there is a technique for reflecting the posture of a user's legs in real space in the posture of a virtual character object in virtual reality space (see, for example, Non-Patent Document 1). According to this technique, an IMU (Inertial Measurement Unit) sensor unit is attached to at least the user's torso, above the left and right knees, and below the left and right knees. Then, posture information of the user's torso and legs estimated using the IMU measurement values ​​is transmitted to a terminal running a virtual reality application. This makes it possible to reflect the user's posture in real space in the posture of the user's virtual object in the virtual reality application.

[0005] There is also a technology that uses an RGB camera and a depth sensor to detect the user's posture and control the game (see, for example, Non-Patent Document 2). This technology uses an RGB camera, a depth sensor, a multi-array microphone, and a processor that runs dedicated software to recognize the player's position, movement, voice, and face. This allows the user to intuitively play a video game using their own body. This technology also uses motion capture, which mainly reads and synthesizes the player's movements. However, unlike general motion capture, it does not require the special marker-equipped suit worn during normal motion capture, or the tracker used to detect the markers. By projecting the subject on the camera, the distance from the player to the Kinect is measured, the player's skeletal movements are detected, and reflected in real time in the character's movements in the game.

[0006] There is also a chair-type device that detects the user's posture in real space (see, for example, Patent Document 1). This technology is based on a controller chair that generates signals corresponding to the movements of the user's torso, left arm, right arm, left leg, and right leg, and includes the following parts: - Seat supported by a base A backrest portion that extends upward or diagonally upward from the rear edge side of the seat portion and is supported on a base so as to be rotatable about an axis that extends along the center line in the width direction of the seat portion. A left arm support portion that protrudes forward from the left side of the backrest and is supported on the backrest so as to be rotatable up and down around its base end portion. A right arm support part that protrudes forward from the right side of the backrest and is supported on the backrest so as to be rotatable up and down around its base end part. A left leg support portion that extends downward from the left side of the front edge of the seat at the front side of the base and is supported by the base so as to be rotatable up and down around its upper end portion. A right leg support portion that extends downward from the front right side of the seat at the front side of the base and is supported by the base so as to be rotatable up and down around its upper end portion. A first rotation angle detection unit, a second rotation angle detection unit, a third rotation angle detection unit, a fourth rotation angle detection unit and a fifth rotation angle detection unit which detect the rotation angles of the backrest unit, the left arm holding unit, the right arm holding unit, the left leg holding unit and the right leg holding unit, respectively. The chair-type device can change the posture of the virtual character by transmitting these rotation angle signals to a terminal running a virtual reality application. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 7189811 [Patent Document 2] Patent No. 6886141 [Patent Document 3] Patent No. 6778205 [Non-patent literature]

[0008] [Non-Patent Document 1] Shiftall, [online], [searched on July 12, 2023], Internet<URL: https: / / ja.shiftall.net / products / haritorax> [Non-Patent Document 2] Wikipedia "Kinect", [online], [searched on July 12, 2023], Internet<URL: https: / / en.wikipedia.org / wiki / Kinect> [Non-Patent Document 3] Wikipedia "Eular angles", [online], [searched July 12, 2023], Internet<URL: https: / / en.wikipedia.org / wiki / Euler_angles> [Non-Patent Document 4] Wikipedia "Quaternion", [online], [searched July 12, 2023], Internet<URL: https: / / en.wikipedia.org / wiki / Quaternion> [Non-Patent Document 5] Wikipedia "Axis-angle representation", [online], [searched on July 12, 2023], Internet<URL: https: / / en.wikipedia.org / wiki / Axis-angle_representation > Summary of the Invention [Problem to be solved by the invention]

[0009] According to the techniques such as those described in Patent Document 1 and Non-Patent Documents 1 and 2, the user's own posture in real space can be reflected in the posture of a virtual object and in game operations.

[0010] However, according to Non-Patent Document 1, in order to detect the posture of the user's legs, it is necessary to attach a posture detection device to a part of the user's body in advance, which is a problem. Of course, it is not possible to detect the posture of a body part to which a sensor unit is not attached. According to Non-Patent Document 2, there is also a problem that privacy may be violated due to whole-body observation of the user using an RGB camera or a depth sensor. Furthermore, according to Patent Document 3, the attitude detection device is limited to detecting a rotation angle with one degree of freedom and cannot detect attitudes with multiple degrees of freedom, and there is also the problem that the user's range of motion is limited by the holding unit.

[0011] Therefore, the present invention aims to provide a furniture-type device, an accessory device, and a system that can estimate the posture of the legs (upper and lower legs) of a seated user. In particular, according to the present invention, there is no need for the user to wear a sensor, there is little possibility of violating the user's privacy due to whole-body observation, and the user's range of motion is not restricted by a holder. [Means for solving the problem]

[0012] According to the present invention, there is provided a furniture-type device on which a user can sit, comprising: One or more lower leg sensors are arranged at a position below a seat of the furniture-type device, and are configured to measure a planar depth distribution around the lower legs of a user who is seated; a posture estimation means for estimating and outputting the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor; The present invention is characterized by having the following.

[0013] According to another embodiment of the furniture-type device of the present invention, The posture estimation means detects, as the posture, a position vector i of the user's ankle and a direction vector l of the user's lower leg based on the depth distribution. It is also preferable.

[0014] According to another embodiment of the furniture-type device of the present invention, The posture estimation means estimates a knee position vector k as a posture from a position vector i of the user's ankle and a direction vector l of the lower leg, using a predetermined parameter t. It is also preferable.

[0015] According to another embodiment of the furniture-type device of the present invention, The posture estimation means estimates a posture, based on a position vector i of the user's ankle, a direction vector l of the lower leg, and a position vector h of the hip joint that is set in advance or detected by a sensor, using a predetermined parameter t. It is also preferable.

[0016] According to another embodiment of the furniture-type device of the present invention, The posture estimation means estimates the flexion angle θ of the knee joint from the direction vector l of the lower leg and the direction vector u of the upper leg as the posture. It is also preferable.

[0017] According to another embodiment of the furniture-type device of the present invention, The posture estimation means estimates the length x of the upper leg and the length y of the lower leg as the posture from the position vector i of the user's ankle, the direction vector l of the lower leg, and the position vector h of the hip joint that is set in advance or detected by a sensor, using a predetermined parameter t. It is also preferable.

[0018] According to another embodiment of the furniture-type device of the present invention, The posture estimation means determines a predetermined parameter t based on a ratio α between the length x of the upper leg and the length y of the lower leg. It is also preferable.

[0019] According to another embodiment of the furniture-type device of the present invention, The posture estimation means estimates the height and / or eye height of the user from the upper leg length x and the lower leg length y using statistical information on human body dimensions. It is also preferable.

[0020] According to another embodiment of the furniture-type device of the present invention, The posture estimation means has a machine learning engine trained with a predetermined parameter t as a learning model. It is also preferable.

[0021] According to another embodiment of the furniture-type device of the present invention, The posture estimation means corrects the posture of the seated user to a standing posture and outputs the corrected posture. It is also preferable.

[0022] According to another embodiment of the furniture-type device of the present invention, Further comprising one or more seat posture sensors for detecting the posture of the seat itself; The posture estimation means corrects and outputs the posture of the user based on the posture of the seat itself. It is also preferable.

[0023] According to another embodiment of the furniture-type device of the present invention, The seat further includes one or more seat pressure sensors arranged on the seat portion to measure a pressure distribution, The posture estimation means estimates a position vector h of the hip joint based on the pressure distribution. It is also preferable.

[0024] According to another embodiment of the furniture-type device of the present invention, The posture estimation means outputs a movement amount in the virtual space of a virtual object operated by the user based on a change in the posture of the user. It is also preferable.

[0025] According to another embodiment of the furniture-type device of the present invention, Chair-shaped, cushion-shaped, mat-shaped, or wheelchair-shaped It is also preferable.

[0026] According to the present invention, there is provided an accessory device that can be attached to furniture on which a user can sit, comprising: One or more lower leg sensors for measuring a planar depth distribution around the lower leg of a seated user; a posture estimation means for estimating and outputting the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor; The present invention is characterized by having the following.

[0027] According to the present invention, there is provided a system including a furniture-type device on which a user can sit and an external device that communicates with the furniture-type device, the furniture-type device has one or more lower leg sensors that are arranged at a position below the seat and transmit a planar depth distribution around the lower leg of a user who is seated on the furniture-type device to a terminal; The external device has a posture estimation means for estimating the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor. It is characterized by: Effect of the Invention

[0028] According to the furniture-type device, accessory device, and system of the present invention, it is possible to estimate the posture of the legs (upper and lower legs) of a seated user. In particular, according to the present invention, it is not necessary for the user to wear a sensor, there is little possibility of violating the user's privacy due to whole-body observation, and the user's range of motion is not restricted by the holder. [Brief description of the drawings]

[0029] [Figure 1] 1 illustrates a chair-like furniture-type device according to a first embodiment of the present invention. [Diagram 2] 1 shows a chair-like furniture-type device according to a second embodiment of the present invention. [Diagram 3] FIG. 11 is an explanatory diagram showing each vector in the right leg of a user sitting on a piece of furniture-type equipment. [Figure 4] 1 is an explanatory diagram showing detection of an ankle position vector j and a lower leg direction vector l when the furniture-type device is viewed from the side. [Diagram 5] FIG. 2 is an explanatory diagram of vectors estimated by the present invention. [Figure 6] 11 is an explanatory diagram showing whether or not correction is performed when converting from a sitting posture to a standing posture. FIG. [Figure 7] Another embodiment of the furniture-type device is a cushion-like or mat-like device. [Figure 8] This is an accessory device that can be attached to furniture that can be sat on. [Figure 9]FIG. 1 is a configuration diagram of a system according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0031] FIG. 1 shows a chair-like furniture-type device according to a first embodiment of the present invention. FIG. 2 shows a chair-like furniture-type device according to the second embodiment of the present invention.

[0032] 1 and 2 show a chair-like furniture-type device 1 on which a user can sit. The furniture-type device 1 of the present invention has one or more lower leg sensors 11 and a posture estimation unit 12. Here, the posture estimation unit 12 is realized by a program that causes a computer mounted on the device to function. Note that the posture estimation unit 12 may be integrated into the furniture-type device 1, or may function as an external device that communicates with the furniture-type device 1.

[0033] [Lower leg sensor 11] The lower leg sensor 11 is disposed at a position below the seat surface of the first furniture-type device, and measures the planar depth distribution around the lower legs of a seated user.

[0034] According to the first embodiment of FIG. 1, one lower leg sensor 11 is installed on a relay axis from the seat of the furniture-type device 1 toward the floor surface, and is directed toward the area around the lower leg of a seated user. The device of the lower leg sensor 11 may be, for example, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) or a depth camera. This makes it possible to measure the planar depth distribution around the lower leg of a seated user. For example, even if a stereo camera or an IR (infrared) camera is used, the planar depth distribution can be measured with one device.

[0035] According to the second embodiment shown in FIG. 2, 30 distance measuring sensors are arranged in 5 rows and 6 columns on the front curtain surface below the seat of the furniture-type device 1, facing the vicinity of the lower legs of a seated user. Distance sensors measure a single distance using light, ultrasound, changes in capacitance, etc. For example, by arranging sensors in an array of 2 or more rows x 2 or more columns, it is possible to measure a planar depth distribution.

[0036] According to the present invention, it is sufficient to measure a planar depth distribution using one or more lower leg sensors 11. Here, a planar depth distribution is one in which a plurality of depth values ​​are arranged two-dimensionally, or one that is configured as a three-dimensional point cloud based on those depth values. A depth distribution in which the depth values ​​are linear (i.e., arranged one-dimensionally) is not included in the lower leg sensor of the present invention.

[0037] [Posture estimation unit 12] The posture estimation unit 12 estimates and outputs the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor 11.

[0038] FIG. 3 is an explanatory diagram showing each vector in the right leg of a user sitting on a piece of furniture-type equipment.

[0039] According to FIG. 3, in posture estimation section 12, the following five vectors are represented in a preset coordinate system. Ankle position vector j: Posture of the lower leg · Direction vector l of the lower leg: Posture of the lower leg (from the knee to the ankle) Knee position vector k: Posture of upper leg · Upper leg direction vector u: Upper leg posture (from the hip joint to the knee) Hip joint position vector h In the present invention, the ankle means the area around the ankle or heel.

[0040] Here, position means a three-dimensional real vector. These vectors may be converted into a predetermined coordinate system and output. The coordinate system may be one or more arbitrary coordinate systems set when the furniture-type device 1 is manufactured, or may be a coordinate system that is dynamically set according to the posture of the seated user. The output values ​​may be scaled or rotated, an offset may be added, or the range of values ​​may be limited.

[0041] FIG. 4 is an explanatory diagram showing detection of an ankle position vector j and a lower leg direction vector l when the furniture-type device is viewed from the side.

[0042] The posture estimation unit 12 detects, as the posture, a position vector i of the user's ankle and a direction vector l of the user's lower leg based on the depth distribution. 4, five depths are measured when the front curtain surface below the seat surface of the furniture-type device 1 is viewed from the side. Here, the difference in depth between the upper two depths and the lower three depths is large. This can be understood as the upper two depths representing the distance to the user's lower legs. The "ankle position vector j" can be regarded as the lowest depth position of the two upper depths. Also, the "lower leg direction vector l" can be calculated from the difference between the positions of the two upper depths. In this way, the ankle position vector i and the lower leg direction vector l can be analytically detected based on the depth distribution detected by the lower leg sensor 11, by taking the difference between adjacent depth values, or by extracting feature points based on a threshold or gradient.

[0043] For simplicity, Fig. 4 shows the case of one leg in two dimensions, but of course it can be expanded to both legs in three dimensions. For example, when a high-resolution depth distribution is obtained by LiDAR or a depth camera, it can be estimated with high accuracy.

[0044] [Detection of ankle position vector i and lower leg direction vector l using a machine learning engine] A machine learning engine may be used to estimate the ankle position vector i and the lower leg direction vector l from the depth distribution detected by the lower leg sensor 11. In this case, as the teacher data, a plurality of depth distributions are used as explanatory variables, and the ankle position vector i and the lower leg direction vector l are used as objective variables. The machine learning engine is trained in advance using this teacher data to construct a learning model. Of course, methods such as segmentation and clustering may be used, or a neural network may be used.

[0045] FIG. 5 is an explanatory diagram of vectors estimated by the present invention.

[0046] 5(a), the posture estimation unit 12 estimates a knee position vector k as a posture from a position vector i of the user's ankle and a direction vector l of the lower leg, using a predetermined parameter t. Specifically, this is expressed by the following formula. k=-tl+j (t>0)

[0047] According to FIG. 5(b), the posture estimation unit 12 estimates the posture, ie, the length x of the upper leg and the length y of the lower leg, from the position vector i of the user's ankle, the direction vector l of the lower leg, and the position vector h of the hip joint that is set in advance or detected by a sensor, using a predetermined parameter t. The length x of the user's upper leg and the length y of the user's lower leg are expressed by the following formula. x=||kh||=||-tl+jh|| y=||jk||=||tl||

[0048] Here, the hip joint position vector h is determined based on the position and orientation of the hip of the seated user as the center of rotation of the upper leg. This may be set in advance when the furniture-type device is designed. Also, the furniture-type device 1 may further include one or more seat pressure sensors for measuring pressure distribution arranged on the seat (see, for example, Patent Document 2). The posture estimation unit 12 can estimate the hip joint position vector h based on the pressure distribution measured by the seat pressure sensor. That is, it can estimate the position and orientation of the hip of the seated user. For example, the built-in pressure distribution sensor sheet is a film-like sensor sheet with a thickness of about 0.1 mm, and can detect applied pressure in real time.

[0049] Here, the posture estimation unit 12 determines a predetermined parameter t based on the ratio α between the length x of the upper leg and the length y of the lower leg. The ratio α may be preset based on statistical information of human body dimensions. It may also be approximated as α=1, or may be changeable by the user. In this way, it is expressed by the following formula. x=αy (α>0) ||-tl+jh|| 2 = α 2 ||tl|| 2 This equation is a linear equation with respect to t when α=1, and a quadratic equation when α≠1, so the following solution can be derived analytically.

number

[0050] 5(c), the posture estimation unit 12 estimates the upper leg direction vector u as the posture from the user's ankle position vector i, lower leg direction vector l, and hip joint position vector h that is set in advance or detected by a sensor, using a predetermined parameter t. This is expressed by the following equation. u=kh=-tl+jh

[0051] The orientation of the upper leg direction vector u and the lower leg direction vector l is calculated based on a predetermined coordinate system. Here, the orientation means information that represents rotation such as Euler angles, quaternions, and axis-angle representations (see, for example, Non-Patent Documents 3, 4, and 5).

[0052] As shown in Figures 5(a) to (c), according to the present invention, the lower leg sensor 11 arranged on the furniture-type device 1 can detect the position of the knee and the posture of the upper leg by observing only a part of the user's lower leg.

[0053] <About local coordinate systems> The orientation of a body part can be calculated by setting a reference coordinate system and a local coordinate system of that body part (see, for example, Non-Patent Document 3). For example, the local coordinate system of the upper leg is set with the seat surface or the lower back as the reference coordinate system. x-axis: upper leg direction vector u y-axis: A vector that is perpendicular to the upper leg direction vector u and parallel to the plane determined by the upper leg direction vector u and the lower leg direction vector l z-axis: A vector perpendicular to the x-axis and y-axis In addition, the local coordinate system of the lower leg is set with the previously determined local coordinate system of the upper leg as the reference coordinate system. x-axis: directional vector of lower leg y-axis: A vector that is perpendicular to the lower leg direction vector l and parallel to the plane determined by the upper leg direction vector u and the lower leg direction vector l z-axis: A vector perpendicular to the x-axis and y-axis However, since the setting of the local coordinate system of the body part differs depending on the system, the method of calculating the orientation of the body part is not limited to this.

[0054] When outputting the orientation of the upper leg or the lower leg as Euler angles, the unit of the value may be converted to degrees, radians, or the like and output. Also, taking into account the degree of freedom of the joint, only a part of the Euler angles may be output. For example, since the rotation axis of the knee joint has one degree of freedom, when outputting the orientation of the lower leg with the local coordinate system of the upper leg as the reference coordinate system, only the pitch angle of the Euler angles may be output. Also, corrections may be made to the output orientation value, such as multiplying or rotating it, adding an offset, or limiting the range of values. For example, the pitch angle of the orientation of the lower leg with the local coordinate system of the upper leg as the reference coordinate system may be corrected to output it as the flexion angle of the knee joint.

[0055] Possible positions that the furniture-type device may output are, for example, as follows. Upper leg posture: Position of upper leg rotation center (hip joint), knee position, upper leg direction, knee direction - Lower leg posture: Position of the center of rotation of the lower leg (knee), position of the ankle, direction of the lower leg, direction of the ankle Note that these posture information items have redundancy. That is, the orientations of the knees, ankles, upper legs, and lower legs can be calculated from the position of the rotation center of the upper legs, the position of the knees, and the position of the ankles. If the lengths of the upper legs and lower legs are set in advance, the positions and orientations of the knees and ankles can also be calculated from the orientations of the upper legs and the lower legs. As a result, the furniture-type device does not need to output all of the above postures, and may output only some of them.

[0056] Furthermore, as another embodiment, the posture estimation unit 12 estimates the height and / or eye height of the user from the upper leg length x and the lower leg length y using statistical information on human body dimensions. The posture estimation unit 12 may have a table in which the correlation between the height and / or eye height of the user and the upper leg length x and the lower leg length y is previously associated.

[0057] FIG. 6 is an explanatory diagram showing the presence or absence of correction in the transition from a sitting posture to a standing posture.

[0058] The posture estimation unit 12 corrects the posture of a seated user to a standing posture and outputs the corrected posture. FIG. 6(a) shows a user sitting on a chair-like piece of furniture, with the right leg and the left leg in different positions. 6(b1) and (b2) show how the knee positions and ankle positions output by the furniture-type device 1 are reflected in the posture of a virtual object in a standing posture. However, in FIG. 6(b1), since no correction process has been performed to change the user's posture from a seated position to a standing position, the right knee of the virtual object is bent in an unnatural direction. In contrast, in FIG. 6(b2), correction processing to a standing posture is performed taking into account the range of motion of the joints, making the standing posture of the virtual object more natural.

[0059] Regarding the correction process of FIG. 6(b2), the posture estimation unit 12 can estimate the flexion angle θ of the knee joint as the posture from the directional vector l of the lower leg and the directional vector u of the upper leg. At this time, a correction process is performed to set the range of motion of the knee flexion angle θ. The corrected flexion angle θ of the knee joint can be reflected in the movement of the virtual object by transmitting it to the virtual space application.

[0060] In the case of a movable chair-type furniture device as shown in FIG. 1, the posture (position and orientation) of the seat itself can be detected by installing an IMU, a rotary encoder, a tracking system for VR (see, for example, Patent Document 3), etc. This information may be used to correct the positions and orientations of the user's body parts and output them. For example, when outputting the positions of the knees and ankles, the knee position vector k and the ankle position vector j can be rotated according to the orientation of the seat itself, and the position of the seat itself can be added as an offset and output. By such correction, the posture of the user can be converted from relative coordinates based on the furniture device to absolute coordinates based on the space in which the furniture device exists, and output.

[0061] <Output of travel distance> The posture estimation unit 12 may output the amount of movement of a virtual object operated by a user in the virtual space based on a change in the posture of the seated user. For example, the output may be as follows. When the user steps while seated, the forward movement distance is output. When the user moves their leg sideways, the amount of translation or rotation to the left or right in virtual space is output. Based on the user's knee flexion angle, a boolean virtual crouch value is output as the movement amount. The virtual jump amount is output by bending and straightening the knee within the threshold time. Conventionally, these operations such as crouching and jumping in virtual reality applications have been performed by pressing buttons on a controller, but according to the present invention, the user can intuitively perform crouching and jumping operations simply by moving their legs.

[0062] FIG. 7 shows a cushion-like or mat-like furniture-type device according to another embodiment.

[0063] 7(a) shows a cushion-like piece of furniture-type equipment that can be placed on the seat of a chair. This equipment has multiple lower leg sensors 11 arranged in the thickness of the seat. 7(b) shows a mat-like furniture-type device that covers the entire chair and is placed close to the user's body. This device has multiple lower leg sensors 11 arranged on the front curtain surface of the lower leg area of ​​the user. Of course, the present invention is not limited to the embodiment shown in FIG. 7, and may be a wheelchair-shaped furniture-type device.

[0064] FIG. 8 shows an accessory device that can be attached to furniture that allows seating.

[0065] 8, a pipe chair is shown as a piece of furniture on which a user can sit. The accessory device 1 of the present invention is attached to the pipe chair. The accessory device 1 has a lower leg sensor 11 and a posture estimation unit 12, similar to the furniture-type device 1 described above.

[0066] FIG. 9 is a diagram showing the configuration of a system according to the present invention.

[0067] 9 includes a furniture-type device 1 on which a user can sit, and an external device 2 that communicates with the furniture-type device 1. Here, the external device 2 may be an HMD terminal 21 or a virtual space server 22. In this case, the furniture-type device 1 has one or more lower leg sensors 11 that are placed at a position below the seat and transmit a planar depth distribution around the lower legs of a seated user to a terminal. The external device 2 also has a posture estimation unit 12 that estimates the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor 11. The external device 2 allows the user to visually recognize the virtual space in which the user himself / herself moves as an avatar. In particular, when the HMD terminal 3 is attached to the head, the user experiences a sense of immersion in the virtual space.

[0068] A system having the furniture-type device 1 and the HMD terminal 21 can be used, for example, for the rehabilitation of a user's walking training. An image of the user walking in front of the device is displayed on the display of the HMD terminal 21 as a virtual space. As a result, even if the user is seated in a chair in the real space, the furniture-type device 1 outputs the amount of movement, allowing the user to recognize the walking state in the virtual space, which can be used for rehabilitation of the walking nerves in the brain. Furthermore, by combining the posture of the lower body detected by the furniture-type device 1 with the hand tracking function of the HMD terminal 21, the posture of the entire body of the user can be reflected in the virtual object.

[0069] As described above in detail, the furniture-type device, accessory device, and system of the present invention can estimate the posture of the legs (upper and lower legs) of a seated user. In particular, the present invention does not require the user to wear a sensor, there is little possibility of violating the user's privacy due to whole-body observation, and the user's range of motion is not restricted by the holder.

[0070] According to the various embodiments of the present invention described above, those skilled in the art can easily make various changes, modifications and omissions within the scope of the technical idea and concept of the present invention. The above description is merely an example and is not intended to be restrictive. The present invention is limited only by the scope of the claims and their equivalents. [Explanation of symbols]

[0071] 1 Furniture type equipment 11 Lower leg sensor 12 Posture estimation section 2 External device 21 HMD terminal 22 Virtual Space Server

Claims

1. A furniture-type device on which a user can sit, one or more lower leg sensors arranged at a position below a seat of the furniture-type device for measuring a planar depth distribution around the lower leg of a user who is seated; a posture estimation means for estimating and outputting the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor; A furniture-type device comprising:

2. The posture estimation means detects, as the posture, a position vector i of the user's ankle and a direction vector l of the user's lower leg based on the depth distribution.

2. The furniture-type device according to claim 1 .

3. The posture estimation means estimates a knee position vector k as a posture from a position vector i of the user's ankle and a direction vector l of the lower leg, using a predetermined parameter t.

3. The furniture-type device according to claim 2 .

4. The posture estimation means estimates a posture, based on a position vector i of the user's ankle, a direction vector l of the lower leg, and a position vector h of the hip joint that is set in advance or detected by a sensor, using a predetermined parameter t.

3. The furniture-type device according to claim 2 .

5. The posture estimation means estimates the flexion angle θ of the knee joint from the direction vector l of the lower leg and the direction vector u of the upper leg as the posture.

5. The furniture-type device according to claim 4.

6. The posture estimation means estimates the length x of the upper leg and the length y of the lower leg as the posture from the position vector i of the user's ankle, the direction vector l of the lower leg, and the position vector h of the hip joint that is set in advance or detected by a sensor, using a predetermined parameter t.

3. The furniture-type device according to claim 2 .

7. The posture estimation means determines a predetermined parameter t based on a ratio α between the length x of the upper leg and the length y of the lower leg.

7. The furniture-type device according to claim 6.

8. The posture estimation means estimates the height and / or eye height of the user from the upper leg length x and the lower leg length y using statistical information on human body dimensions.

7. The furniture-type device according to claim 6.

9. The posture estimation means has a machine learning engine trained with a predetermined parameter t as a learning model.

7. The furniture-type equipment according to claim 3, 4 or 6.

10. The posture estimation means corrects the posture of the seated user to a standing posture and outputs the corrected posture.

2. The furniture-type device according to claim 1 .

11. Further comprising one or more seat posture sensors for detecting the posture of the seat itself; The posture estimation means corrects and outputs the posture of the user based on the posture of the seat itself.

2. The furniture-type device according to claim 1 .

12. The seat further includes one or more seat pressure sensors arranged on the seat portion to measure a pressure distribution, The posture estimation means estimates a position vector h of the hip joint based on the pressure distribution.

5. The furniture-type device according to claim 4.

13. The posture estimation means outputs a movement amount in the virtual space of a virtual object operated by the user based on a change in the posture of the user.

2. The furniture-type device according to claim 1 .

14. Chair-shaped, cushion-shaped, mat-shaped, or wheelchair-shaped 2. The furniture-type device according to claim 1 .

15. An accessory device that can be attached to furniture on which a user can sit, One or more lower leg sensors for measuring a planar depth distribution around the lower leg of a seated user; a posture estimation means for estimating and outputting the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor; An accessory device comprising:

16. A system including a furniture-type device on which a user can sit and an external device that communicates with the furniture-type device, the furniture-type device has one or more lower leg sensors that are arranged at a position below the seat and transmit a planar depth distribution around the lower leg of a user who is seated on the furniture-type device to a terminal; The external device has a posture estimation means for estimating the postures of the upper and lower legs of the left and right legs of the seated user based on the depth distribution measured by the lower leg sensor. A system characterized by:

Images