Body guidance device, body guidance method, and program

Abstract

To physically guide various parts of the body toward a target.SOLUTION: A body guiding apparatus includes: a display unit that displays an image of an object to a user; a body guiding unit that physically guides a part of a user's body toward the object; a guiding force determining unit that determines a relation between an angle and a guiding force of the body guiding unit with respect to an angle formed by a direction in which a part of the user's body faces the object and a direction in which the part of the user's body currently faces; and a control unit that controls the body guiding unit, wherein the control unit controls the body guiding unit by feeding back the angle during guidance to adjust the guiding force by using the relation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a body guidance device, a body guidance method, and a program.

Background Art

[0002] By dynamically arranging a three-dimensional guiding object at an appropriate position in a three-dimensional space so as to effectively guide the user's line of sight, a technique has been developed that gives a visual line-of-sight guiding effect linked to the operation of a head-mounted display (hereinafter also referred to as "HMD") to a user immersed in a three-dimensional virtual space while wearing the HMD (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, the technique described in Patent Document 1 can guide the user's line of sight, but cannot physically guide other body parts than the line of sight.

[0005] The present disclosure has been made in view of such a situation, and an object thereof is to provide a technique capable of physically guiding various parts of the body.

Means for Solving the Problems

[0006] In order to solve the above problems, a body guidance device according to an aspect of the present invention is a device that guides a part of a user's body, and includes a display unit that displays an image of an object to the user, a body guidance unit that physically guides a part of the user's body toward the object, a guidance force determination unit that determines the relationship between the angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body currently faces, and the guidance force of the body guidance unit, and a control unit that controls the body guidance unit. The control unit controls the body guidance unit by adjusting the guidance force using the above relationship by feeding back the angle during guidance.

[0007] Another aspect of the present invention is a body guidance method for guiding a part of a user's body. This method includes a step of displaying an image of an object to the user using display means, a step of determining, using guidance force determination means, the relationship between the angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body currently faces, and the guidance force of the body guidance means, a step of controlling the body guidance means using control means, and a step of physically guiding a part of the user's body toward the object using the body guidance means. The control means controls the body guidance means by adjusting the guidance force using the above relationship by feeding back the angle during guidance.

[0008] Still another aspect of the present invention is a program. This program causes a computer to execute a method including a step of displaying an image of an object to the user using display means, a step of determining, using guidance force determination means, the relationship between the angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body currently faces, and the guidance force of the body guidance means, a step of controlling the body guidance means using control means, and a step of physically guiding a part of the user's body toward the object using the body guidance means. The control means controls the body guidance means by adjusting the guidance force using the above relationship by feeding back the angle during guidance.

[0009] In addition, any combination of the above-described components, as well as those obtained by converting the expressions of the present disclosure among a method, an apparatus, a system, a recording medium, a computer program, etc., are also effective as aspects of the present disclosure.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to physically guide various parts of the body.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0012] [First Embodiment] FIG. 1 shows a functional block diagram of a body guidance device 1 according to the first embodiment. The body guidance device 1 includes a display unit 10, an induction force determination unit 20, a control unit 30, and a body guidance unit 40.

[0013] The display unit 10 displays an image of an object to the user. The object in this specification is an object for performing operations, processes, creations, etc. while the user visually confirms it. Specific examples of the object include microorganisms and cells observed with a microscope, parts of a fine precision instrument, and the like. In such a case, the user performs necessary operations while visually confirming a microscope image of a minute object. The display unit may be any suitable display device such as an HMD, a liquid crystal display, a video projector, or the like. The image displayed on the display unit may be an XR (Cross Reality) image such as MR (Mixed Reality), VR (Virtual Reality), or AR (Augmented Reality) in addition to a real image. In particular, the display unit is usefully an HMD for displaying an MR image in which the real world and the virtual world are fused.

[0014] The induction force determination unit 20 determines the relationship between the angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body is currently facing, and the induction force of the body guidance unit 40 with respect to the angle. Here, a part of the user's body may be, for example, the head, upper body, lower body, hand, finger, abdomen, waist, foot, or the like. Further, the direction in which a part of the body faces may be the direction in which the line of sight faces, the direction in which the tip of the hand, finger, or foot points, the normal direction of the frontal plane, sagittal plane, or horizontal plane of the head, abdomen, or waist, a specific direction with respect to a joint part of the body, or the like.

[0015] Hereinafter, in this specification, the force for guiding and moving a part of the user's body toward an object is referred to as "guiding force". Generally, it is presumed that the larger the range to be moved, the greater the force required for the guiding force. Furthermore, the guiding force depends on physical characteristics such as the user's physique and muscle strength, and psychological characteristics such as the sense of movement. Therefore, the relationship between the range of guiding a part of the user's body to move and the required guiding force needs to be determined for each user. FIG. 2 schematically shows the operation of the guiding force determination unit 20 for determining such a relationship.

[0016] FIG. 20 is a schematic view of the head as a part of the user's body seen from above. The user's line of sight is selected as the direction in which the head faces, and this is represented by arrows g 0 , g 1 , g 2 , …, g 10 . Here, the arrow g 0 is the user's current line of sight (that is, the direction in which a part of the user's body is currently facing). The arrows g 1 , g 2 , …, g 10 indicate the respective lines of sight when the head is moved by gradually changing the guiding force applied to the head (here, the force for rotating the head clockwise when viewed from above around the vertical axis). This corresponds to the existence of an object corresponding to each guiding force at the tip of the lines of sight g 1 , g 2 , …, g 10 . The angles θ 1 , θ 2 , …, θ 10 are the angles formed by g 1 , g 2 , …, g 10 and g 0 (that is, the angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body is currently facing). Let the guiding forces corresponding to the lines of sight g 1 , g 2 , …, g 10 be P 1 , P 2 , …, P 10 respectively. In this example, it is assumed that the following relationship holds between the guiding force P and θ. That is, it is assumed that P is represented by a linear expression of θ. P = f(θ) = a·θ + b ··· (1) At this time, the induction force determination unit 20 uses the actually measured θ n and P n (n = 1,..., 10) to determine the coefficients a and b by applying the least squares method. Hereinafter, the relational expression f(θ) representing the relationship between θ and P as in (1) is called the "determination equation".

[0017] As described above, generally, the larger θ is, the larger P is, so the coefficient a is obtained as a positive value. That is, generally, f(θ) is a monotonically increasing function of θ.

[0018] In the above example, a linear function was assumed as the functional form of f(θ) in the determination equation. However, it is not limited to this, and f(θ) may be any suitable monotonically increasing function such as an exponential function, a polynomial function (partial domain), a sigmoid function (partial domain), or a logistic function (partial domain).

[0019] Figs. 3 and 4 schematically show the operation of the induction force determination unit 20 when the body induction device 1 has a plurality of induction directions. Figs. 3 and 4 are schematic views looking down on the user's head from above. In Figs. 3 and 4, the x-axis is taken in the direction from the left side to the right side of the user's head, the y-axis is taken in the direction from the bottom to the top of the user's head, and the z-axis is taken in the direction from the back to the front of the user's head. The subscript k indicates the type of induction direction. Specifically, in this example, k = 1 indicates a positive rotation around the y-axis (induction to turn the head to the left), k = 2 indicates a negative rotation around the y-axis (induction to turn the head to the right), k = 3 indicates a positive rotation around the x-axis (induction to tilt the head forward), and k = 4 indicates a negative rotation around the x-axis (induction to tilt the head backward). At this time, the determination equation is expressed as follows. P k = f k (θ k ) = a k ·θ k + b k (k = 1,..., 4) ··· (2) The coefficients a k and b k are the actually measured θ k n and Pk n It is determined by applying the least squares method based on (n = 1, …, 10).

[0020] In the above description, as the functional form of f(θ), a monotonically increasing function considered to be the most effective is taken as an example. However, it is not limited to this, and f(θ) may be any suitable function such as an oscillatory function or a function having an extreme value.

[0021] As described above, for each user, the relationship between the range for guiding and moving a part of the user's body and the required guiding force is determined by a determination equation. However, even if guidance is performed using this relationship, for example, if the state of the object during actual use or the physical or mental state of the user is different, accurate guidance as intended is not always achievable. That is, simply using the relationship determined by the determination equation as an initial condition may result in a difference between the determined guiding force and the actually required guiding force.

[0022] To solve this, the control unit 30 of the present embodiment controls the body guidance unit 40 using feedback as described below. FIG. 5 schematically shows the feedback control executed by the control unit 30. The guiding force determined by the guiding force determination unit 20 is input to the control unit 30 as the target guiding force. The target guiding force is input to the body guidance unit 40 to guide a part of the user's body. Thereafter, based on the difference between the currently observed angle and the target angle by the user, the guiding force corresponding to the current angle is determined using the determination equation. This guiding force corresponding to the current angle is added to the target guiding force as a feedback gain. By creating such a feedback loop, the difference between the determined guiding force and the actually required guiding force is corrected. The above feedback can be expressed as follows. P k t =f k (θ k t )+G(f k (θ k t )-f k (θ k)) (k = 1, …, 4) ··· (3) Here, G is a coefficient that determines the magnitude of the feedback gain.

[0023] As described above, the control unit 30 controls the body guidance unit 40 by adjusting the guidance force using the relationship determined by the determination equation by feedbacking the angle during guidance.

[0024] The body guidance unit 40 physically guides a part of the user's body toward the object.

[0025] As described above, according to this embodiment, it is possible to physically guide various parts of the body toward the object.

[0026] [Specific implementation example of the body guidance device] Hereinafter, the body guidance device will be specifically described by taking, as an example, a body guidance unit mounted on clothing worn by a user and implemented as an actuator driven by artificial muscles. FIGS. 6 to 10 are diagrams showing such a body guidance unit. FIG. 6 is a schematic diagram of the entire body guidance device. FIG. 7 is a photograph of the entire body guidance device including the body guidance unit. FIG. 8 is a photograph of the front of the clothing including the body guidance unit. FIG. 9 is a photograph of the back of the clothing including the body guidance unit. FIG. 10 is a photograph of a user wearing the clothing including the body guidance unit. In FIG. 10, an experimental keyboard described later is placed on the operator's knee.

[0027] The display units in FIGS. 6 and 7 are composed of an MR type HMD. The guidance force determination unit is implemented as a software program executed by a PC. The body guidance unit is attached to the clothing worn by the user and implemented as an actuator driven by 12 artificial muscles. The artificial muscles are actuated by compressed air supplied from an air compressor. The artificial muscles contract more strongly and exert a greater guidance force as the air pressure from the air compressor is higher. The control unit controls the air pressure of the air compressor.

[0028] Of the 12 artificial muscles that drive the actuator, two guide the user's upper body to twist to the left, two guide the user's upper body to twist to the right, four guide the user's upper body to tilt forward, and four guide the user's upper body to tilt backward.

[0029] In the above example, the body guidance part was a cloth-like wearable type actuator. However, it is not limited to this, and the body guidance part may have any suitable configuration such as an exoskeleton type muscle suit or a motion controller using functional electrical stimulation.

[0030] [Examples of usage scenarios of the body guidance device] As an actual usage scenario, an example of operating a tiny object at a remote location using the body guidance device in FIG. 7 will be described. Since the display part of the body guidance device in FIG. 7 is composed of an MR type HMD, an image in which the real world and the virtual world are fused is displayed to the user. FIG. 11 is an image displayed to the user wearing the body guidance device in FIG. 7. FIG. 12 is an image (microscopic image of a pig embryo at a remote location) constituting the virtual world to be displayed. That is, the user experiences as if the object (virtual world) at a remote location exists in the surrounding scenery (real world). At this time, since the body guidance device guides the user's body toward the object, the user can accurately operate the tiny object without feeling discomfort.

[0031] [Size of the object] In the above example, the image (microscopic photograph) of the object displayed on the display part was an enlarged image of a tiny object. However, it is not limited to this, and the image of the object may be approximately equal to the actual size or a reduced image of a huge object. For example, tiny objects include biological cells, parts of precision machinery, microbeads, etc. On the other hand, huge objects include construction machinery, heavy machinery, etc.

[0032] The relational expression determined by the determination equation may reflect the size of such an object. For example, in the case of an object with a minute size, since the viscous term in the fluid becomes dominant, the determination equation reflects this. On the other hand, in the case of an object with a huge size, since the inertial term becomes dominant, the determination equation reflects this.

[0033] [Embodiment of further displaying an image for guiding a line of sight to an object] As a developmental embodiment, the display unit may further display an image for guiding the user's line of sight to the object. Fig. 13 shows, in addition to the object displayed on the display unit, an image for guiding the user's line of sight to the object. One of the plurality of spheres present in the image is the object. The spheres other than the object are experimental dummies. The image indicated by the arrow-shaped icon is an image for guiding the user's line of sight to the object.

[0034] To verify the effect of this embodiment, an experiment was conducted by a subject wearing the body guidance device of Fig. 7. The method of the experiment is as follows. One target object and 100 dummy objects are displayed on the display unit. These target objects and dummy objects move randomly in the vertical, horizontal, and front-rear directions, respectively. The subject is instructed to search for the target object and press a button on the keyboard when the target object is recognized. When the button on the keyboard is pressed, the target object returns to the initial position. Hereinafter, the guidance by the body guidance device is referred to as "physical guidance", and the guidance by the image for guiding the user's line of sight is referred to as "line-of-sight guidance". The experiment is conducted under four conditions: "no physical guidance nor line-of-sight guidance", "guidance by physical guidance only", "guidance by line-of-sight guidance only", and "coexistence of physical guidance and line-of-sight guidance". For one subject, this experiment is repeated 40 times for each condition. Hereinafter, the experimental results obtained for six subjects are shown.

[0035] Figures 14 and 15 show the results of the above experiment. Figure 14 is a graph showing the variation in the number of target objects recognized by six subjects under the above four conditions. Figure 15 is a table showing the average number of target objects recognized by six subjects under the above four conditions. From these results, it can be seen that when there is physical guidance, more target objects can be recognized compared to the case where there is no guidance at all. Furthermore, it can be seen that when physical guidance and line-of-sight guidance coexist, more target objects can be recognized compared to the case where there is only physical guidance. In this experiment, it was found that when there is only line-of-sight guidance, more target objects can be recognized compared to the case where there is only physical guidance. This is considered to be because this experiment was conducted exclusively by a method using object recognition by the user's eyes.

[0036] [Efficiency improvement of work by body guidance] As an application example of this embodiment, there is one that improves the efficiency of work on an object by continuously performing a plurality of body guidances such as line of sight, trunk, and limbs. For example, when considering an operation of "approaching an object located far away, picking it up, and throwing it into a box in the virtual space", the body guidance is as follows. Body guidance (1): Direct the user towards the object. Body guidance (2): Make the user walk towards the object. Body guidance (3): Make the user pick up the object with both hands. Body guidance (4): Make the user throw the object into a box in the virtual space. Body guidance (5): Direct the user in another direction. According to this usage example, for example, it is possible to improve the efficiency of complex work procedures that become long texts when manualized, or work including delicate operations that are difficult to verbalize.

[0037] [Second Embodiment] FIG. 16 shows a flowchart of a body guidance method according to the second embodiment. This method is a method for guiding a part of a user's body using a display means, a guiding force determination means, a control means, and a body guiding means, and includes step S1, step S2, step S3, and step S4.

[0038] In step S1, the method uses the display means to display an image of an object to the user. In step S2, the method uses the guiding force determination means to determine the relationship between the angle formed by the direction in which a part of the user's body faces the object and the current direction in which the part of the body faces, and the guiding force of the body guiding means with respect to the angle. In step S3, the method uses the control means to control the body guiding means. In step S4, the method uses the body guiding means to physically guide a part of the user's body toward the object. The control means controls the body guiding means by adjusting the guiding force using the above relationship by feeding back the angle during guiding. Since the configurations and operations of the display means, the guiding force determination means, the control means, and the body guiding means are as described in the first embodiment, detailed descriptions are omitted.

[0039] According to this embodiment, various parts of the body can be physically guided toward the object.

[0040] [Third Embodiment] The third embodiment is a program. This program causes a computer to execute the method according to the aforementioned second embodiment. According to this embodiment, physical guidance of various parts of the body toward an object can be realized as computer software.

[0041] [Aspects of the Present Disclosure] The body guidance device according to an aspect of the present disclosure includes a display unit that displays an image of an object to a user, a body guidance unit that physically guides a part of the user's body toward the object, an induction force determination unit that determines the relationship between the angle formed by the orientation of a part of the user's body with respect to the object and the current orientation of the part of the body, and the induction force of the body guidance unit, and a control unit that controls the body guidance unit. The control unit controls the body guidance unit by adjusting the induction force using the above relationship by feeding back the angle during induction.

[0042] According to this aspect, various parts of the body can be physically guided toward the object.

[0043] The above relationship may be such that the induction force is represented as a monotonically increasing function of the angle.

[0044] According to this aspect, body guidance can be realized by providing a specific monotonically increasing function as a determination equation.

[0045] The image of the object displayed on the display unit may be an enlarged or reduced image of the actual object.

[0046] According to this aspect, objects of different sizes (for example, objects of a microscopic size or a huge size) that are different from the objects that exist around the body and can be touched by hand can be recognized and operated without discomfort.

[0047] The display unit may further display an image for guiding the user's line of sight to the object.

[0048] According to this aspect, in addition to the physical body guidance toward the object, visual line-of-sight guidance is performed, so that more accurate body guidance can be realized.

[0049] The body guidance unit is attached to the clothing worn by the user and includes an actuator driven by an artificial muscle, and the control unit may control the body guidance unit by controlling the output of the artificial muscle.

[0050] According to this aspect, a specific configuration of the body guidance device can be provided.

[0051] A body guidance method according to an aspect of the present disclosure includes: a step of displaying an image of an object to a user using a display means; a step of determining, using a guiding force determination means, a relationship between an angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body currently faces, and the guiding force of the body guiding means; a step of controlling the body guiding means using a control means; and a step of physically guiding a part of the user's body toward the object using the body guiding means. The control means controls the body guiding means by adjusting the guiding force using the above relationship by feeding back the angle during the guidance.

[0052] According to this aspect, various parts of the body can be physically guided toward the object.

[0053] A program according to an aspect of the present disclosure causes a computer to execute a method including: a step of displaying an image of an object to a user using a display means; a step of determining, using a guiding force determination means, a relationship between an angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body currently faces, and the guiding force of the body guiding means; a step of controlling the body guiding means using a control means; and a step of physically guiding a part of the user's body toward the object using the body guiding means. The control means controls the body guiding means by adjusting the guiding force using the above relationship by feeding back the angle during the guidance.

[0054] According to this aspect, the physical guidance of various parts of the body toward the object can be realized as computer software.

[0055] The above has been described based on embodiments. It should be understood by those skilled in the art that these embodiments are illustrative, and various modifications are possible for each of their constituent elements and combinations of each processing process, and such modifications are also within the scope of the present disclosure.

[0056] For example, in the first embodiment, the display unit, the induction force determination unit, the control unit, and the body induction unit have been described as being integrated into one body induction device. However, some or all of these components may be separately arranged at a remote location, and the overall function may be realized by these components communicating with each other. According to this modification, the degree of freedom in configuration can be increased.

[0057] The above has described the embodiments and modification examples. When understanding the technical idea abstracted from the embodiments and modification examples, the technical idea should not be interpreted in a limited manner to the content of the embodiments and modification examples. The above-described embodiments and modification examples are merely specific examples, and many design changes such as changes, additions, and deletions of constituent elements are possible. In the embodiments, regarding the content for which such design changes are possible, the notation "embodiment" is added for emphasis. However, design changes are also allowed for the content without such notation.

Explanation of Reference Numerals

[0058] 1 ··· Body induction device, 10 ··· Display unit, 20 ··· Induction force determination unit, 30 ··· Control unit, 40 ··· Body induction unit, S1 ··· Step of displaying an image of an object to the user, S2 ··· Step of determining the relationship between the angle formed by the direction in which a part of the user's body faces the object and the direction in which the part of the body is currently facing, and the induction force of the body induction means, S3 ··· Step of controlling the body induction means, S4 ··· Step of inducing a part of the user's body.

Claims

1. An apparatus for guiding a part of a user's body, comprising: a display unit that displays an image of an object to the user; a body guiding unit that physically guides a part of the user's body toward the object; a guiding force determining unit that determines the relationship between the angle formed by the direction in which a part of the user's body faces the object and the direction in which that part of the body currently faces, and the guiding force of the body guiding unit; a control unit that controls the body guiding unit; The control unit controls the body guiding unit by adjusting the guiding force using the relationship by feeding back the angle during guiding. wherein the part of the body excludes the eyeballs; When the angle formed by the direction in which a part of the user's body faces the object and the direction in which that part of the body currently faces is θ, and the guiding force is P, the relationship is determined by the determination equation P = f(θ) and is characterized in that different relational expressions are used according to the size of the object.

2. The body guiding apparatus according to claim 1, wherein the determination equation is represented as a monotonically increasing function of θ.

3. The body guiding apparatus according to claim 2, wherein the determination equation is represented by any one of a linear function, an exponential function, a polynomial function (partial domain), a sigmoid function (partial domain), or a logistic function (partial domain) of θ.

4. The determination equation reflects an inertia term when the object is of a huge size, and reflects a viscous term when the object is of a tiny size and exists in a fluid, characterizing the body guiding apparatus according to claim 1.

5. The body guiding apparatus according to claim 1, wherein the image of the object displayed on the display unit is an enlarged or reduced image of the actual object.

6. The body guiding apparatus according to claim 1, wherein the display unit further displays an image for guiding the user's line of sight to the object.

7. The body guiding apparatus according to claim 1, wherein the body guiding unit is attached to clothing worn by the user and includes an actuator driven by an artificial muscle, and the control unit controls the body guiding unit by controlling the output of the artificial muscle.

8. A method for guiding a part of a user's body, comprising: displaying an image of an object to the user using display means; ​ Using the guiding force determination means, determining the relationship between the angle formed by the direction in which a part of the user's body faces the object and the current direction in which the part of the body faces, and the guiding force of the body guiding means with respect to the angle; Using the control means to control the body guiding means; Using the body guiding means to physically guide a part of the user's body towards the object; including; The control means controls the body guiding means by adjusting the guiding force using the relationship by feeding back the angle during guiding; The part of the body excludes the eyeballs; When the angle formed by the direction in which a part of the user's body faces the object and the current direction in which the part of the body faces is θ, and the guiding force is P, the relationship is the determination equation P = f(θ) determined by; The determination equation is characterized in that different relational expressions are used according to the size of the object. A body guiding method.

9. A program for causing a computer to execute a method of guiding a part of a user's body, using display means to display an image of an object to the user; using guiding force determination means to determine the relationship between the angle formed by the direction in which a part of the user's body faces the object and the current direction in which the part of the body faces, and the guiding force of the body guiding means with respect to the angle; using control means to control the body guiding means; using the body guiding means to physically guide a part of the user's body towards the object; causing a computer to execute a method including; The control means controls the body guiding means by adjusting the guiding force using the relationship by feeding back the angle during guiding; The part of the body excludes the eyeballs; When the angle formed by the direction in which a part of the user's body faces the object and the current direction in which the part of the body faces is θ, and the guiding force is P, the relationship is the determination equation P = f(θ) determined by; The determination equation is characterized in that different relational expressions are used according to the size of the object. A program.

Images