Weight sensation presentation system
The weight sensation presentation system simulates weight perception by controlling the difference between commanded and actual object positions, addressing the lack of physical haptic devices, enabling weight perception and distinction in virtual environments.
Patent Information
- Application Number
- JP2024081197
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-05-17
- Publication Date
- 2025-11-28
AI Technical Summary
Current technologies lack the ability to allow users to perceive weight without using a physical haptic device.
A weight sensation presentation system that utilizes the difference between the commanded and actual positions of a controlled object, controlled by a control device, to simulate weight perception by treating a joint as a rotational spring, increasing the positional difference with the mass of the supported object.
Enables users to perceive and distinguish weight without a physical haptic device, applicable in virtual shopping and online rehabilitation.
Smart Images
Figure 2025174688000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a weight sensation presentation system. [Background technology]
[0002] It is expected that the use of haptics will improve the realism of VR experiences, and a haptic presentation method that does not require complex devices is desired. One such method is a technology that uses illusions called pseudo-haptics. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] Majed Samad, Elia Gatti, Anne Hermes, Hrvoje Benko, Cesare Parise. Pseudo-Haptic Weight: Changing the Perceived Weight of Virtual Objects By Manipulating Control-Display Ratio. CHI2019 Summary of the Invention [Problem to be solved by the invention]
[0004] Currently, there are no adequate proposals for technologies that allow users to perceive weight without using a physical haptic device.
[0005] The present invention has been made in light of the above circumstances, and one exemplary purpose of an embodiment of the present invention is to provide a technology that allows a user to perceive weight without using a physical haptic sense presentation device. [Means for solving the problem]
[0006] In order to solve the above problem, one embodiment of the weight sensation presentation system of the present invention is a weight sensation presentation system that allows a user to perceive the weight of an object supported by a controlled object based on the difference between the commanded position of the controlled object and the actual position of the controlled object, and is equipped with a control device that controls the actual position of the controlled object so that the difference becomes greater the greater the mass of the supported object when the supported object is supported without moving it.
[0007] Another aspect of the present invention is also a sensation presentation system. This weight sensation presentation system allows a user to perceive the weight of a supported object supported by a controlled object, the controlled object being rotatable about a joint and including a control device that controls the actual position of the controlled object based on a commanded position of the controlled object. The control device regards the joint as a rotational spring whose extension is the difference between the commanded position and the actual position as the equilibrium position of the joint.
[0008] Any combination of the above components, or mutual substitution of the components or expressions of the present invention between methods, devices, systems, etc., are also valid aspects of the present invention. [Effects of the Invention]
[0009] According to one aspect of the present invention, weight can be perceived without a physical haptic presentation device. [Brief explanation of the drawings]
[0010] [Figure 1] 1 is a diagram illustrating an overview of a weight sensation presentation system according to an embodiment. [Figure 2] 1 is a diagram illustrating a configuration of a weight sensation presentation system according to an embodiment. [Figure 3] FIG. 3 is a block diagram showing the functions and configuration of the control device of FIG. 2. [Figure 4] 3 is a diagram showing an image representing a virtual world that is displayed on the display device by the display control unit of FIG. 2. FIG. [Figure 5] 3 is a flowchart showing an example of the operation of the weight sensation presentation system of FIG. 2. [Figure 6] 10 is a graph showing the results of a perception test. [Figure 7] FIG. 10 is a diagram showing the configuration of a weight sensation presentation system according to a modified example. [Figure 8] FIG. 8 is a block diagram showing the functions and configuration of the control device of FIG. 7. [Figure 9] FIG. 10 is a diagram showing the configuration of a weight sensation presentation system according to another modified example. DETAILED DESCRIPTION OF THE INVENTION
[0011] 1(a) and 1(b) are diagrams illustrating an overview of a weight sensation presentation system according to an embodiment. Fig. 1(a) shows a user's hand 50 placed on a base with the palm facing up, and Fig. 1(b) shows a control object 52 associated with the hand 50. The control object 52 has a shape (e.g., a plate shape) that resembles the user's hand 50, although this is not limited to such a shape.
[0012] In this embodiment, the control target 52 is an object in the virtual world. That is, the hand 50 in Fig. 1(a) is the user's hand in the real world, and the control target 52 in Fig. 1(b) is an object in the virtual world associated with the hand 50 in Fig. 1(a), i.e., an object in an image displayed by the weight sensation provision system on its display device 3 (described later in Fig. 2). While nothing is placed on the user's hand 50 in the real world in Fig. 1(a), a support target 54 is placed on the control target 52 in the virtual world in Fig. 1(b).
[0013] In addition, when the control object 52 is associated with the user's hand 50, it means that when the support object 54 is not placed on the control object 52, the control object 52 moves in synchronization with the hand 50, in other words, it moves as if it were the user's hand.
[0014] The user views the virtual world. That is, the user does not view his / her own hand 50 in the real world, but rather the control target 52 associated with the hand 50, which is an object in the virtual world.
[0015] When a support object 54 is placed on a control object 52, the weight sensation presentation system displays the control object 52 at a position Pv2 that is shifted downward from a position Pv1 in the virtual world that corresponds to the position Pr of the hand 50 in the real world. This gives the user the illusion that the support object 54 is actually placed on the hand 50. In other words, the user can perceive the weight of the support object 54.
[0016] In other words, the weight sensation presentation system allows the user viewing the virtual world to perceive the weight of the support object 54 based on the difference (i.e., the difference in angle θ) between a position Pv1 in the virtual world (hereinafter also referred to as the command position Pv1) corresponding to the position Pr of the user's hand 50 in the real world, and a position Pv2 (hereinafter also referred to as the actual position Pv2) that is shifted downward from the command position Pv1 and where the control object 52 is actually displayed in the virtual world.
[0017] In particular, the weight sensation presentation system displays the controlled object 52 so that the difference between the command position Pv1 and the actual position Pv2 increases as the mass of the supported object 54 increases. The greater the difference, the greater the perceived mass.
[0018] The above is an overview of the weight sensation presentation system. Preferred embodiments will now be described with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and redundant explanations will be omitted where appropriate. Furthermore, the embodiments are merely examples and do not limit the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.
[0019] 2 is a diagram showing the configuration of a weight sensation presentation system 1 according to an embodiment. The weight sensation presentation system 1 includes a detection device 2, a display device 3, and a control device 10. The detection device 2 and the display device 3 are each connected to the control device 10 via wireless communication or wired communication. The wireless communication may be short-range wireless communication such as Wi-Fi or Bluetooth (registered trademark).
[0020] The detection device 2 detects the position of a predetermined part of the user to which the control object 52 is associated. In this example, the part is a "hand" placed on a base with the palm facing up. The detection device 2 may be configured to include, for example, one or more cameras. The detection device 2 outputs the detection result to the control device 10 at a predetermined cycle.
[0021] The control device 10 is a computer, and may be, for example, a server or a personal computer. The control device 10 may be realized by a single computer or by multiple computers working together.
[0022] As will be described in detail later, the control device 10 identifies a command position c1 of the control object 52 in the virtual world based on the hand detection result by the detection device 2, and further identifies an actual position Pv2. The control device 10 generates an image representing the virtual world in which the control object 52 is located at the actual position Pv2, and displays the image on the display device 3. For example, the control device 10 generates an image in which the support object 54 is placed on the control object 52.
[0023] In this example, the display device 3 is a wearable display device that is worn on the user's head. Although the display device 3 is goggle-type in FIG. 2, other structures such as eyeglass-type or helmet-type may also be adopted. The display device 3 may include a computer and serve as the control device 10.
[0024] The display device 3 may be a stationary, wall-mounted, or portable display device. The display device 3 may also be a terminal device having a display unit, such as a smartphone or a tablet terminal.
[0025] FIG. 3 is a block diagram showing the functions and configuration of the control device 10 of FIG. 2. Each block shown here can be realized in hardware by elements and mechanical devices, such as a computer's CPU and memory, and in software by a computer program, etc. However, the functional blocks shown here are realized by the cooperation of these elements. Those skilled in the art will understand that these functional blocks can be realized in various ways by combining hardware and software. Furthermore, the functions of multiple functional blocks may be implemented in a single computer or may be distributed and implemented across multiple computers. For example, the functions of multiple functional blocks may be realized by multiple computers working together as a system. In other words, the control device 10 may be configured by multiple computers. The same applies to FIG. 8, which will be described later.
[0026] The control device 10 includes a communication unit 11, a storage unit 12, and a control unit 20. The communication unit 11 communicates with other devices according to a predetermined communication protocol. The storage unit 12 stores data that is referenced or updated by the control unit 20. The control unit 20 executes various types of data processing.
[0027] The control unit 20 includes an acquisition unit 21, an identification unit 22, and a display control unit 23. The acquisition unit 21 acquires data from the detection device 2 via the communication unit 11.
[0028] The display control unit 23 generates an image representing the virtual world and displays it on the display device 3. The image includes objects of the control target 52 and the support target 54. In detail, the display control unit 23 displays the object of the control target 52 at the actual position Pv2 identified by the identification unit 22.
[0029] 4 is a diagram showing an image representing a virtual world that is displayed on the display device 3 by the display control unit 23. The image includes a control target 52 and a supported target 54 that is supported by the control target 52. In this example, the image representing the virtual world is an image from a first-person perspective, but it may also be an image from a third-person perspective.
[0030] Returning to Fig. 3, the identification unit 22 first identifies a position Pv1 in the virtual world corresponding to the position Pr of the user's hand in the real world, that is, a command position Pv1, from the position Pr acquired by the acquisition unit 21 from the detection device 2. The identification unit 22 may identify the command position Pv1 from the position Pr using known or future available technology.
[0031] The specifying unit 22 further specifies an actual position Pv2 where the control object 52 is actually displayed from the command position Pv1.
[0032] Here, since muscles have spring-like properties, the wrist joint can be considered as a rotational spring. Furthermore, since a torque expressed by the following equation (1) is generated around the wrist joint, it is thought that the user perceives weight in a pseudo-like manner. T=kθ (1) k: spring constant θ: rotation angle (angle between actual position and command position)
[0033] Therefore, in this embodiment, the identification unit 22 identifies, as the actual position Pv2, the balanced position when the joint is regarded as a rotational spring whose extension amount is the difference between a command position Pv1 with respect to the control object 52 and an actual position Pv2 of the control object 52. In other words, the actual position Pv2 is identified so as to satisfy the following equation (2). As is clear from equation (2), the actual position Pv2 is a position according to the mass of the supported object 54. (md+MD)g=kθ (2) m: mass of the hand M: Mass of the supported object d: Distance from the wrist to the center of gravity of the hand D: Distance from the wrist to the center of gravity of the object being supported g:Gravity acceleration
[0034] The spring constant of the rotation spring is determined in advance. The spring constant of the rotation spring may be changed depending on the weight that the user wants to perceive. The spring constant of the rotation spring may be, for example, 1.45. In this case, the user can perceive a weight of 200g to 800g at an angle that allows the spring to bend naturally toward the user, for example, 26° to 57°.
[0035] The mass m of the hand, the distance d from the wrist to the center of gravity of the hand, and the distance D from the wrist to the center of gravity of the object being supported may be values determined from the average values of an adult.
[0036] The identification unit 22 may skip the process of identifying the actual position Pv2 of the control object 52 when the support object 54 is not placed on the control object 52, in other words, when the weight of the support object 54 is zero. Alternatively, the calculation formula used by the identification unit 22 to identify the actual position Pv2 may be a calculation formula such that the command position Pv1 becomes the actual position Pv2 when the weight of the support object 54 is zero. In either case, when the support object 54 is not placed on the control object 52, in other words, when the weight of the support object 54 is zero, the command position Pv1 becomes the actual position Pv2. In this case, the control object 52 in the virtual world moves in synchronization with the user's hand 50 in the real world.
[0037] The above is the basic configuration of the weight sensation presentation system 1. Next, the operation of the weight sensation presentation system 1 will be described.
[0038] Fig. 5 is a flowchart showing an example of the operation of the weight sensation presentation system 1. For example, a user places a hand 50 on a base with the palm facing up, and visually recognizes the virtual world, particularly a control target 52 in the virtual world associated with the hand 50. In this case, the process of Fig. 5 is repeatedly executed at a predetermined cycle, for example, a cycle on the order of several ms to several tens of ms.
[0039] The acquisition unit 21 acquires data output from the detection device 2 (S10).
[0040] The identification unit 22 identifies the position Pv1 of the user's hand 50, i.e., the command position Pv1, from the data acquired by the acquisition unit 21 from the detection device 2 (S12). The identification unit 22 identifies the actual position Pv2 of the control object 52 based on the command position Pv1 and the perceived weight of the support object (S14).
[0041] The display control unit 23 displays the control object 52 supporting the supporting object 54 at the actual position Pv2 identified by the identification unit 22 (S16).
[0042] The above is the operation of the weight sensation presentation system 1. Next, a perception test conducted by the inventor will be described.
[0043] In the perception test, the subject placed their hand 50 on a base with their palm facing up, and alternated between placing two support objects 54 on a control object 52 in the virtual world associated with their hand 50, and responded which felt heavier. One of the two support objects 54 had a mass of 500 g, which was a reference mass, and the other support object 54 had a mass of 200 g, 300 g, 400 g, 500 g, 600 g, 700 g, or 800 g, which were comparison masses. Each user took the perception test 14 times, with each of the seven comparison masses being selected twice.
[0044] In this perception test, the weight sensation presentation system 1 hides the control object 52 and the support object 54 until the control object 52 reaches a predetermined height, specifically until the top surface of the control object 52 on which the support object 54 rests becomes horizontal. In other words, the weight sensation presentation system 1 displays the control object 52 and the support object 54 once the control object 52 reaches the predetermined height. The subject was instructed to slowly increase the angle of their wrist in the real world, and once the control object 52 and the support object 54 were displayed, the subject was asked to fix the angle of their wrist and hold it still for six seconds. Furthermore, if the subject moved their wrist while the control object 52 and the support object 54 were displayed, the weight sensation presentation system 1 had them repeat the perception test.
[0045] Figure 6 is a graph showing the results of the perception test. In Figure 6, the horizontal axis represents the comparison mass, and the vertical axis represents the percentage of people who answered that the reference mass was heavier. The smaller the comparison mass, the greater the percentage of people who answered that the reference mass was heavier, and the larger the comparison mass, the smaller the percentage of people who answered that the reference mass was heavier. For example, when the comparison mass was 200 g, most people answered that the reference mass was heavier than the comparison mass. Also, when the comparison mass was 800 g, most people answered that the comparison mass was heavier than the reference mass.
[0046] This shows that the weight of the supported object can be perceived by using the weight sensation provision system 1. It is also shown that the weight sensation provision system 1 can distinguish the weight.
[0047] Furthermore, in this perception test, control object 52 and support object 54 were hidden until control object 52 reached a predetermined height, specifically until the top surface of control object 52 on which support object 54 rested was horizontal, and if the subject moved their wrist while control object 52 and support object 54 were displayed, they were required to repeat the perception test. This shows that the sensation of weight is not induced by watching the movement of the palm, that is, weight can be perceived even when supporting support object 54 without moving it.
[0048] Next, the effects achieved by the embodiment will be described.
[0049] According to the weight sensation presentation system 1 of this embodiment, it is possible to make a user perceive weight and furthermore to distinguish the weight without using a physical haptic sense presentation device. Therefore, the weight sensation presentation system 1 of this embodiment is expected to be applied to, for example, allowing a purchaser to virtually experience the weight of a product when shopping online, or for online rehabilitation.
[0050] Furthermore, in conventional techniques using pseudo-haptics, the perception of weight is achieved by changing the ratio (CD ratio) of the amount of actual hand movement (Control) to the amount of hand movement (Display) in the virtual world displayed on a display device when moving a support object. Therefore, when supporting a support object without moving it, the amount of movement of the support object is zero, so the CD ratio is always zero regardless of whether the support object is light or heavy. In other words, conventional techniques using pseudo-haptics are premised on the fact that the support object is moving. In contrast, this embodiment uses a model in which the wrist joint is regarded as a rotational spring, so it can be applied even when the support object is not moving. Furthermore, weight can be perceived even when supporting a support object 54 without moving it.
[0051] The present invention has been described above based on an embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present invention. These modifications will be described below.
[0052] (First Modification) In the embodiment, the case where weight is perceived by bending and straightening the wrist joint has been described, but weight may also be perceived by bending and straightening other joints such as the fingers, elbow, and shoulder. That is, in the embodiment, the case where the control object 52 is associated with the hand has been described, but the control object 52 may also be associated with other parts such as the fingers, forearm, and the entire arm. In this case, the other joints can be regarded as rotational springs.
[0053] (Second Modification) In the embodiment and the above-described modified example, the case where the user moves only one joint has been described, but the user may move multiple joints. In this case, each of the multiple joints can be regarded as a rotational spring. In this case, the determination unit 22 can determine the actual position Pv2 from the command position Pv1 using a known or future available technique such as a Jacobian matrix.
[0054] (Third Modification) In the embodiment and the above-described modified examples, the control target 52 is an object in the virtual world associated with a predetermined part of the user, but the control target 52 may be a predetermined part of a robot in the real world. In other words, the user may be made to perceive the weight of the support target 54 supported by a predetermined part of the robot based on the difference between a command position for the predetermined part of the robot in the real world and the actual position of the predetermined part. The robot may be, for example, a robot hand or a robot arm, or a prosthetic hand.
[0055] FIG. 7 is a diagram showing the configuration of a weight sensation presentation system 1 according to a modified example. The weight sensation presentation system 1 includes a detection device 2, a robot 4, and a control device 10. The weight sensation presentation system 1 of this modified example includes a robot 4 instead of a display device 3. In this example, the robot 4 is a robot arm. An object, which is a support target 54, is placed in the hand of the robot 4, which is a control target 52. The following description will focus on the differences from the embodiment.
[0056] Fig. 8 is a block diagram showing the functions and configuration of the control device 10 of Fig. 7. The control unit 20 of the control device 10 of this modification includes a robot control unit 24 instead of the display control unit 23.
[0057] The identification unit 22 first identifies, from the position of the user's hand acquired by the acquisition unit 21 from the detection device 2, the position of the control target 52 (the hand of the robot 4) corresponding to that position, i.e., the command position. The identification unit 22 further identifies, from the command position, the actual position at which the control target 52 (the hand of the robot 4) is actually positioned. As in the embodiment, the identification unit 22 regards the joint as a rotational spring, and identifies the balanced position in that case as the actual position.
[0058] The robot control unit 24 controls the robot 4. The robot control unit 24 moves the hand of the robot 4, which is the control target 52, to the actual position identified by the identification unit 22.
[0059] This modification can achieve the same effects as the embodiment. For example, the weight sensation presentation system 1 can allow the user to perceive the weight of the support target 54 supported by the robot 4 and further distinguish the weight without using a physical tactile sense presentation device.
[0060] (Fourth Modification) Unlike the embodiment and the above-described modified examples, the detection device 2 may detect muscle activity (e.g., myoelectric potential) for moving a predetermined part of the user associated with the control target 52. In this case, the identification unit 22 may identify the command position Pv1 based on the muscle activity using known or future available technology.
[0061] The predetermined part of the user may be a part that is congenitally or acquiredly missing. For example, the predetermined part may be a missing hand, in which case the detection device 2 may detect muscle activity that attempts to move the missing hand.
[0062] (Fifth Modification) Although not mentioned in the embodiment and the above-described modified examples, the spring constant of a joint regarded as a rotational spring may be determined based on the muscle activity (for example, myoelectric potential) of the user detected by a separate detection device.
[0063] 9 is a diagram showing the configuration of a weight sensation presentation system 1 according to another modification. Note that, although the case where the control target 52 is an object in a virtual world will be described here as an example, the technical concept of this modification can also be applied to a case where the control target 52 is a predetermined part of a robot 4 in the real world.
[0064] The weight sensation presentation system 1 includes a detection device 2, a display device 3, another detection device 5, and a control device 10. In this example, the another detection device 5 detects the myoelectric potential of the arm muscles. The another detection device 5 outputs the detection result to the control device 10 at a predetermined cycle.
[0065] The identification unit 22 identifies the command position Pv1 and further the actual position Pv2 in the same manner as in the embodiment. At this time, the identification unit 22 determines the spring constant k of the joint regarded as a rotational spring based on the myoelectric potential detected by another detection device 5. The identification unit 22 may determine the spring constant k of the joint based on the myoelectric potential using known or future available technology. The stronger the force applied by the user, the higher the spring constant k becomes, and therefore the difference between the command position Pv1 and the actual position Pv2 (i.e., the deviation of the angle θ) becomes smaller.
[0066] According to this modification, the user can perceive the weight of the support target 54 in terms of the spring constant in addition to or instead of the difference between the command position Pv1 and the actual position Pv2.
[0067] Any combination of the above-described examples and modifications is also useful as an embodiment of the present invention. A new embodiment resulting from a combination will have the combined effects of the combined examples and modifications. It will also be understood by those skilled in the art that the functions to be performed by each constituent element in the claims can be realized by each component shown in the examples and modifications, either individually or in combination. [Explanation of symbols]
[0068] 1 Weight sensation presentation system, 2,5 Detection device, 3 Display device, 4 Robot, 10 Control device.
Claims
1. 1. A weight sensation presentation system that allows a user to perceive a weight of an object supported by a controlled object based on a difference between a command position of the controlled object and an actual position of the controlled object, a control device that controls the actual position of the controlled object so that the difference becomes larger as the mass of the supported object increases when the supported object is supported without moving; Weight sensation presentation system.
2. the controlled object is rotatable around a joint; the control device determines the actual position as a balanced position when the joint is regarded as a rotational spring having the difference as an extension amount; The weight sensation presentation system according to claim 1 .
3. A weight sensation presentation system that allows a user to perceive the weight of a support object supported by a control object, the controlled object is rotatable around a joint; a control device for controlling an actual position of the controlled object based on a command position of the controlled object; the control device regards the joint as a rotational spring whose extension amount is the difference between the command position and the actual position as the actual position, Weight sensation presentation system.
4. a display device for displaying an image of a virtual world; the control object and the support object are objects included in the image; The weight sensation presentation system according to claim 1 .
5. The weight sensation presentation system according to claim 4 , wherein the control device displays the controlled object and the supported object only when the controlled object should be displayed at a predetermined position.
6. The control target is a predetermined part of a robot. The weight sensation presentation system according to claim 1 .
7. the control target corresponds to a predetermined part of the user, The command position is based on the position of the predetermined part of the user detected by a detection device. The weight sensation presentation system according to claim 1 .
8. the command position is based on a myoelectric potential of the user detected by a detection device; The weight sensation presentation system according to claim 1 .
9. the control device determines a spring constant of the rotation spring based on a myoelectric potential of the user detected by a detection device; The weight sensation presentation system according to claim 2 or 3.