Force feedback device and force feedback method

Abstract

To solve the problem that a force sense becomes spatially discontinuous when an interval between stimulation elements is too large on a force sense presentation device that generates a self motion sense by applying stimulation to human skin without actually moving or inclining a person.SOLUTION: A force sense presentation device that presents a force sense by forming a dynamic distribution on human skin, comprises: a plurality of stimulation elements; and a control unit. Each of the plurality of stimulation elements applies stimulation to human skin. The control unit controls the plurality of stimulation elements so as to form a dynamic distribution for generating a target force sense on skin. An interval between adjacent stimulation elements is an interval that can provide a spatially continuous force sense. The force sense produces a self motion sense of a person. The interval between adjacent stimulation elements is an interval that can present a spatially continuous force sense.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a force sense presentation device and a force sense presentation method for presenting a force sense by applying a stimulus to a person's skin. [Background technology]

[0002] In virtual reality, telepresence, etc., it is necessary to give the user a high sense of presence, so it is desirable to be able to present the sensation of force. For example, motion platforms used in entertainment facilities and flight simulators give the user the sensation of moving or tilting the user as if he or she were moving or displacing spatially (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special table 2016-533534 Summary of the Invention [Problem to be solved by the invention]

[0004] In this way, the motion platform reproduces actual movements, realizing the vestibular sensation and the sense of force exerted from the seat and back, giving the user the sensation of exercising. However, since it is necessary to reproduce a movement similar to the intended movement, there is a problem that the reproducible movement is highly dependent on the mechanical properties of the housing, and the movement that can be expressed is limited. Furthermore, there are issues such as the motion platform being large and expensive, and being unable to be attached to existing chairs.

[0005] It is expected that if it were possible to present force sensations, such as the kinesthetic sensation associated with three-dimensional movement, for example by applying stimulation to the user's skin, without actually moving or tilting the person as with a motion platform, this would make a significant contribution to solving the above problems.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to present a force sense that gives a person the sensation of being in motion by stimulating their skin. [Means for solving the problem]

[0007] In order to solve the above problems, a device according to one aspect of the present invention is a force sensation presentation device that presents a force sensation by forming a mechanical distribution on a person's skin. The device includes a plurality of stimulation elements and a control unit. Each of the plurality of stimulation elements applies a stimulus to the person's skin. The control unit controls the plurality of stimulation elements to form a mechanical distribution on the skin that generates a desired force sensation. The spacing between adjacent stimulation elements is a spacing that allows the presentation of a spatially continuous force sensation. The force sensation generates a sense of self-motion in the person.

[0008] The force sense presentation device may further include a memory unit that stores a relationship between a target force sense and a mechanical distribution that generates the force sense, and a force sense instruction unit. When the target force sense is instructed by the force sense instruction unit, the control unit may control the stimulation element to form, on the skin, the mechanical distribution that generates the force sense stored in the memory unit.

[0009] The control unit may calculate a dynamic distribution that generates a desired force sensation, and control the stimulation element to apply a stimulus to the skin that forms the dynamic distribution.

[0010] The haptic device may further include a learning unit that learns the relationship between the presented haptic sensation and the mechanical distribution that generates the haptic sensation.

[0011] The stimulation elements may provide force sensations to parts of the body including the back or buttocks.

[0012] The stimulation element may provide a force sensation to a body part including a hand or a foot. In this case, the control unit may control the stimulation element to form a mechanical distribution on the skin according to the stretching of the skin caused when the body part is moved.

[0013] The mechanical distribution may be a strain energy density distribution.

[0014] The control unit may control the stimulation element so that the greater the desired force sensation, the greater the strain energy density.

[0015] The stimulation element may stimulate the skin by a rotational motion. Hereinafter, such a stimulation element may be referred to as a "rotational stimulation element." In this case, the control unit may control the rotation angle and the rotation direction of the stimulation element.

[0016] The control unit may control the stimulation element such that the greater the desired force sensation, the greater the rotation angle of the stimulation element.

[0017] A set of multiple stimulator elements may be grouped together into a stimulator unit.

[0018] The stimulator unit may include a two-axis gimbal.

[0019] The control unit may move a stimulation point on the person's skin for providing stimulation, and the control unit may control the stimulation element such that the smaller the distance from the stimulation point, the larger the rotation angle of the stimulation element.

[0020] The mechanical distribution may be a distribution of strain, where the strain may be a principal strain or an equivalent strain.

[0021] The mechanical distribution may be a distribution of forces, where the forces may be shear forces or normal forces.

[0022] The mechanical distribution may be a distribution of stress, where the stress may be any of pressure, principal stress or von Mises stress.

[0023] The spacing between adjacent stimulation elements may be such that the ranges of haptics provided by the adjacent stimulation elements overlap.

[0024] The haptic device may include a mechanical change detection unit and a mechanical distribution calculation unit. The mechanical change detection unit detects mechanical changes in the person's skin caused by the person's movement. The mechanical distribution calculation unit calculates a mechanical distribution of the skin that generates the mechanical change from the change in the mechanical distribution of the person's skin detected by the mechanical change detection unit. The control unit controls the multiple stimulation elements to form the mechanical distribution calculated by the mechanical distribution calculation unit on the skin.

[0025] Each of the plurality of stimulation elements may include a mechanical change sensing unit.

[0026] The mechanical change detection unit may be a sensor provided independently of the stimulation element.

[0027] The haptic device may include a camera that captures a subjective image seen by a person, and an image synchronization unit that calculates a relationship between a mechanical change in the person's skin and a change in the subjective image. The mechanical distribution calculation unit may calculate a mechanical distribution of the skin that generates the mechanical change from the mechanical change in the person's skin when the subjective image changes, based on the relationship between the mechanical change in the person's skin calculated by the image synchronization unit and the change in the subjective image.

[0028] Yet another aspect of the present invention is a method for presenting a force sensation by forming a force distribution on a human's skin. The method includes the steps of determining a relationship between a force sensation and a force distribution that generates the force sensation, and applying a stimulus to a plurality of points on the skin so as to form a force distribution that generates the force sensation on the skin. The interval between adjacent points on the skin to which the stimulus is applied is an interval that can generate a spatially continuous force sensation. The force sensation generates a sense of self-motion in the human.

[0029] The mechanical distribution in the haptic feedback method may be a strain energy density distribution.

[0030] The force feedback method may include the steps of detecting mechanical changes in a person's skin due to the person's movement, calculating a mechanical distribution of the skin that generates the mechanical changes from the detected mechanical changes in the person's skin, and applying stimulation to multiple locations on the skin so as to form the calculated mechanical distribution on the person's skin.

[0031] The above-mentioned haptic feedback method may include a step of capturing a subjective image seen by a person, and a step of calculating a relationship between a mechanical change in the person's skin and a change in the subjective image. In this case, the step of calculating a mechanical distribution of the skin that generates the mechanical change may calculate the mechanical distribution of the skin that generates the mechanical change from the mechanical change in the person's skin when the subjective image changes, based on the calculated relationship between the mechanical change in the person's skin and the change in the subjective image.

[0032] Any combination of the above components, and any transformation of the present invention into an apparatus, method, system, recording medium, computer program, etc., are also effective as aspects of the present invention. Effect of the Invention

[0033] According to the present invention, by applying a stimulus to the person's skin, it is possible to present a force sense that gives the person the sensation of feeling as if he or she is moving. [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a functional block diagram of a force feedback device according to a first embodiment. [Diagram 2] FIG. 11 is a functional block diagram of a force feedback device according to a second embodiment. [Diagram 3] FIG. 13 is a functional block diagram of a force feedback device according to a fourth embodiment. [Figure 4] FIG. 13 is a schematic diagram of a force feedback device according to a fifth embodiment. [Diagram 5] 13 is a photograph of a force feedback device according to a seventh embodiment. [Figure 6] 1 is a photograph of a stimulator unit. [Figure 7] FIG. 23 is a flowchart of a force feedback method according to a tenth embodiment. [Figure 8] FIG. 23 is a functional block diagram of a force feedback device according to an eleventh embodiment. [Figure 9] FIG. 23 is a functional block diagram of a force feedback device according to a twelfth embodiment. [Figure 10] FIG. 23 is a functional block diagram of a force feedback device according to a thirteenth embodiment. [Figure 11] FIG. 23 is a functional block diagram of a force feedback device according to a fourteenth embodiment. [Figure 12] FIG. 23 is a flowchart of a force feedback method according to a fifteenth embodiment. [Figure 13] FIG. 23 is a flowchart of a force feedback method according to a sixteenth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] The present invention will be described below based on preferred embodiments with reference to the drawings. In the embodiments and modified examples, identical or equivalent components, steps, and parts are given the same reference numerals, and duplicated explanations are omitted as appropriate. The dimensions of the parts in each drawing are enlarged or reduced as appropriate for ease of understanding. Some parts that are not important for explaining the embodiments are omitted in each drawing. Terms including ordinal numbers such as first and second are used to describe various components, but these terms are used only for the purpose of distinguishing one component from other components, and the components are not limited by these terms.

[0036] Before describing specific embodiments, the inventor has found that applying shear deformation (deformation along the skin direction) to the skin of a person's buttocks can reproduce the sensation of acceleration induced when riding a vehicle and the sensation of vertical movement when overcoming undulations.

[0037] For example, when a driver sits in a car seat and accelerates or decelerates or turns the steering wheel, a part of the driver's buttocks undergoes shear deformation in the front-back and left-right directions. The inventors focused on this phenomenon and discovered that by providing a contactor that moves front-back and left-right on the seat and applying shear deformation to the subject's buttocks, it is possible to induce and reproduce the sensation of acceleration in the front-back and left-right directions.

[0038] In addition, when a person rides in a car and overcomes a step or undulation, the body moves and displaces in the vertical direction. It has been found that the sensation of such vertical movement can also be reproduced by shear deformation of the skin. For example, when the buttocks move vertically from the seating surface of the seat, the peak of the pressure distribution distributed near the ischium displaces toward the tailbone. The inventors have focused on this phenomenon and found that it is possible to induce a sensation associated with being pushed up from the seating surface simply by presenting the displacement of the peak position of the pressure distribution by shear deformation of the skin of the buttocks. Furthermore, the inventors have also found that it is possible to induce and reproduce the sensation of overcoming an undulation simply by presenting to the buttocks the shear force in the direction of travel that occurs when overcoming an undulation.

[0039] The inventor considers the reason why such a sensation is induced as follows. First, when such a shear force is applied to the skin, a distribution of mechanical quantity per unit volume, for example, a distribution of strain energy (hereinafter referred to as "strain energy density distribution") is formed on the skin. That is, when a person moves while in contact with a sheet or the like, strain is generated at each point on the skin. It is considered that the temporal and spatial distribution of this strain energy density reflects the characteristics of the type, magnitude, direction, etc. of the movement. The frequency of neuronal firing of the mechanoreceptors inside the skin that control the sense of force is determined by this strain energy density distribution. Therefore, it is considered that if a strain energy density distribution corresponding to various movements is formed on the skin by applying a stimulus to the human skin, it is possible to induce the sensation that the person is making such movements without actually moving the person.

[0040] The part of the skin to which the stimulation is applied is not limited to the buttocks, but may be any part that allows the formation of a strain energy density distribution corresponding to the desired movement, such as the back, abdomen, head, arms, wrists, or legs. The distribution of mechanical quantities corresponding to the desired movement (hereinafter referred to as "mechanical distribution") is not limited to the strain energy density distribution, but includes the distribution of strains (e.g., principal strain, equivalent strain, etc.), the distribution of forces (e.g., shear force, normal force, etc.), and the distribution of stresses (i.e., force per unit area, e.g., pressure, principal stress, Mises stress, etc.).

[0041] In the following description, the sensation that a person feels as if he or she is moving or displaced in space is referred to as "self-motion sensation." This self-motion sensation is not limited to the kinesthesia felt when the person is actually moving, such as when riding in a vehicle. For example, when an object in contact with the skin is moving, the person feels as if the person is moving relative to the object. This sensation, that is, the kinesthesia felt when the person is not actually moving, is also included in the self-motion sensation. In this specification, "presenting a force sensation" refers to presenting the magnitude and direction of a force that a person feels.

[0042] [First embodiment] 1 shows functional blocks of a force sense presentation device 1 according to a first embodiment. The force sense presentation device 1 presents a force sense by forming a mechanical distribution on human skin S. The force sense presentation device 1 includes n stimulation elements 11, 12, ..., 1n and a control unit 20 (n is an integer equal to or greater than 2).

[0043] Each of the stimulation elements 11, 12, ..., 1n applies a stimulus to the human skin S. The method by which the stimulation elements 11, 12, ..., 1n apply a stimulus to the skin S may be a mechanical method such as rotational motion, pulling in a shear direction, compression, or suction. Each of the stimulation elements 11, 12, ..., 1n may be capable of presenting a force sensation by itself. Alternatively, the force sensation may be presented only when a plurality of stimuli work together to form a predetermined mechanical distribution.

[0044] The control unit 20 controls the stimulation elements 11, 12, ..., 1n so as to form a mechanical distribution on the skin S that generates a desired force sensation. The control unit 20 may be configured using known computer hardware and software. The method by which the control unit 20 controls the stimulation elements 11, 12, ..., 1n may be based on a database stored inside the control unit 20, or may be based on an input from the outside, or may be based on learning such as machine learning.

[0045] The spacing between adjacent stimulation elements is a spacing that allows for the presentation of a spatially continuous force sensation. If the spacing between stimulation elements is too far apart, the force sensation will be spatially discontinuous. In this case, it is not possible to form a mechanical distribution on the skin that generates the desired force sensation. The spacing between adjacent stimulation elements may be, for example, a spacing that allows the ranges of force sensations presented by the adjacent stimulation elements to overlap. Alternatively, the spacing between adjacent stimulation elements may be a spacing that is within the range of the two-point discrimination threshold in the region of the skin.

[0046] The force sense generates a person's sense of self-motion. That is, the force sense presented by the force sense presentation device 1 gives the person a sense as if he or she is moving or displaced spatially. As described above, this sensation may be a sense of motion felt when the person is actually moving, or may be a sense of motion felt when the person is not actually moving.

[0047] According to this embodiment, it is possible to realize a device that provides a force sense to a person, giving the person the sensation of feeling as if he or she is moving, by applying a stimulus to the person's skin.

[0048] [Second embodiment] Fig. 2 shows functional blocks of a haptic device 2 according to the second embodiment. The haptic device 2 also presents a haptic sensation by forming a mechanical distribution on the human skin S. The haptic device 2 includes n stimulation elements 11, 12, ..., 1n, a control unit 20, a storage unit 30, and a haptic instruction unit 40 (n is an integer equal to or greater than 2). That is, the haptic device 2 includes the storage unit 30 and the haptic instruction unit 40 in addition to the configuration of the haptic device 1 in Fig. 1.

[0049] The configuration and operation of the stimulation elements 11, 12, ..., 1n are the same as those of the force sense presentation device 1, so the description will be omitted. The storage unit 30 stores the mechanical distribution that generates the desired force sense and the control method of each stimulation element for forming the mechanical distribution. For example, the storage unit 30 stores, as a database, what mechanical quantities should be distributed in time and space on the skin S in order to generate the desired force sense. In the case of a mechanical distribution such as a strain energy density distribution, for example, a device for directly measuring this is not known, but it can be calculated using a simulation as follows. For the simulation, for example, finite element analysis based on a finite element model of the human body is effective. In the simulation, the physical situation that causes the force sense to be presented is input. For example, if it is desired to induce a sense of self-motion while driving a car, the acceleration of the car is input. At this time, by defining the weight of the person and the elasticity of the skin as predetermined parameters, it is possible to solve the deformation of the skin according to the acceleration of the car. As a result, a model for calculating the strain energy density distribution can be obtained. The validity of the obtained model can be verified, for example, as follows. That is, a tester measures the surface pressure on his / her buttocks and back while driving a car equipped with a commercially available acceleration sensor and a surface pressure distribution measurement sheet. By comparing the measurement results with the surface pressure calculated from the model, the validity of the model can be verified. If the comparison shows a large difference between the actual measurement value and the calculated value from the model, a new simulation can be performed to improve the model.

[0050] The force sense instruction unit 40 instructs the control unit 20 to generate a desired force sense. When the desired force sense is instructed by the force sense instruction unit 40, the control unit 20 controls the stimulation elements 11, 12, ..., 1n to form a mechanical distribution on the skin S that generates the force sense stored in the storage unit 30.

[0051] According to this embodiment, by storing the relationship between the desired force sense and the mechanical distribution that generates the force sense in memory unit 30, the desired force sense can be presented at the desired timing.

[0052] [Third embodiment] 1, the control unit 20 may calculate a mechanical distribution that generates a desired force sensation, and control the stimulation elements 11, 12, ..., 1n to provide a stimulus that forms the mechanical distribution to the skin S. When the mechanical distribution can be calculated using a relatively simple model, such as when presenting a force sensation caused by rolling a roller on a person's back, there is no need to bother calculating and storing the mechanical distribution using finite element analysis based on a finite element model of the human body, as in the second embodiment, which saves hardware resources.

[0053] According to this embodiment, a desired force sense can be presented by calculating a force distribution that generates a desired force sense.

[0054] [Fourth embodiment] Fig. 3 shows functional blocks of a haptic device 3 according to the fourth embodiment. The haptic device 3 also presents a haptic sensation by forming a mechanical distribution on the human skin S. The haptic device 3 includes n stimulation elements 11, 12, ..., 1n, a control unit 20, a memory unit 30, a haptic instruction unit 40, and a learning unit 50 (n is an integer equal to or greater than 2). That is, the haptic device 2 includes a learning unit 50 in addition to the configuration of the haptic device 2 in Fig. 2. The configurations and operations of the stimulation elements 11, 12, ..., 1n, the control unit 20, and the haptic instruction unit 40 are common to those of the haptic device 2, and therefore will not be described.

[0055] The learning unit 50 learns the relationship between the force sense presented to the person and the mechanical distribution that generates the force sense. The presented force sense refers to the magnitude and direction of the force given to the person, and the mechanical distribution is estimated by a simulation such as finite element analysis as described in the second embodiment. The learning may be performed by a known AI. The specific method of the AI ​​is not particularly limited, but for example, a neural network such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a long short term memory (LSTM) network may be used, and in this case, different neural networks may be mixed for each calculation model with a common input layer. In this embodiment, a large number of pairs of the presented force sense and the mechanical distribution that generates the force sense are prepared, and the AI ​​learns these as learning data. As a result, when a desired force sense is input, the AI ​​can calculate and output the mechanical distribution that generates the force sense.

[0056] The memory unit 30 stores the learning results of the learning unit 50. When a target force sense is instructed by the force sense instructing unit 40, the control unit 20 controls the stimulation elements 11, 12, ..., 1n based on the learning results stored in the memory unit 30 so as to form a mechanical distribution on the skin S that generates the force sense.

[0057] [Fifth embodiment] FIG. 4 shows a schematic diagram of a force feedback device 4 according to a fifth embodiment. In this embodiment, the stimulation elements 11, 12, ..., 1n provide a force feedback to a body part including the back or buttocks. The inventors have found that by forming a mechanical distribution on a person's back or buttocks in particular, a force feedback with a higher degree of freedom and excellent resolution can be achieved over a large area of ​​the body. Therefore, it is considered that if a force feedback is provided to a body part including the back or buttocks, a self-motion sensation corresponding to a more complicated movement can be induced. Furthermore, this embodiment can be realized by attaching the stimulation elements 11, 12, ..., 1n to an existing chair or the like. This is a great advantage in terms of size and cost.

[0058] According to this embodiment, it is possible to present a sense of self-motion corresponding to complicated movements at low cost.

[0059] [Sixth embodiment] The mechanical distribution may be a strain energy density distribution. The present inventor has found that self-motion sensation can be induced by forming a strain energy density distribution, among various mechanical distributions. Recent research has confirmed that the frequency of neuronal firing of mechanoreceptors present in human skin is proportional to the strain energy density. For example, when a person feels that "strong force is applied," it corresponds to when the neuronal firing of the mechanoreceptors occurs at a high frequency. Therefore, it is considered important to control the frequency of neuronal firing in order to control the sensation. Considering this, it can be seen that the formation of a strain energy density distribution is particularly important in the case of inducing self-motion sensation.

[0060] Furthermore, the control unit 20 may control the stimulation elements 11, 12, ..., 1n so that the larger the desired force sensation, the larger the strain energy density. The inventors have found that there is often a positive correlation between the magnitude of the desired force sensation and the strain energy density. Therefore, it is believed that the larger the desired force sensation, the larger the strain energy density, by controlling the stimulation elements 11, 12, ..., 1n, it is possible to induce a sense of self-motion.

[0061] [Seventh embodiment] The stimulation elements 11, 12, ..., 1n may stimulate the skin by rotational motion. In this case, the control unit 20 may control the rotation angle and the rotation direction of the stimulation elements 11, 12, ..., 1n.

[0062] In particular, in the case of a device that presents a force sensation over a wide area of ​​the back or buttocks as shown in FIG. 4, it is desirable that the stimulation given to the skin is not by suction, electrical stimulation, temperature stimulation, pressure, or translational motion along the shear direction of the skin, but by rotational motion. The reasons are as follows. First, in the form as shown in FIG. 4, it is assumed that the stimulation is given to the skin through clothing. In this case, for example, stimulation by suction, electrical stimulation, or temperature stimulation cannot be directly applied. Second, the stimulation by pressure causes the user's posture on the chair to shift, which hinders the formation of an appropriate mechanical distribution on the skin. Furthermore, third, when trying to give a stimulation by translational motion along the shear direction of the skin, it is necessary to convert the rotational motion of the motor that serves as the power source into translational motion. This makes the mechanism complicated, which causes a problem that it is difficult to apply to a form in which stimulation elements are densely arranged as shown in FIG. 4. In this regard, the stimulation by rotational motion can be applied to the embodiment without the above-mentioned problems.

[0063] The inventors have experimentally confirmed that a mechanical distribution that generates a sense of self-motion can be formed on the skin by appropriately controlling the rotation angle and direction of the stimulation elements 11, 12, ..., 1n. In this case, not only the rotation angle but also the time required to reach the rotation angle may be included in the control parameters.

[0064] It should be noted that in this embodiment, the purpose of the stimulation by the rotational motion is not to present the direction of twisting itself as a force sensation, but to excite the strain energy density near the stimulation point. In actual experiments, the rotation direction of a single stimulation element is not perceived, and the force sensation can only be presented when multiple stimulation elements work together to form a predetermined strain energy density distribution.

[0065] According to this embodiment, it is possible to generate a sense of movement that is felt when the user is actually moving, for example, when riding in a vehicle, while it is also possible to generate a sense of movement that is felt when the user is not actually moving, for example, a sensation that the user is moving relative to an object pressed against the user's back.

[0066] The control unit 20 may control the stimulating elements 11, 12, ..., 1n so that the larger the desired force sensation, the larger the rotation angle of the stimulating elements 11, 12, ..., 1n. The inventors have found that the magnitude of the desired force sensation and the strain energy density are often positively correlated. It has also been found that the strain energy density and the rotation angle of the stimulating elements 11, 12, ..., 1n are also positively correlated. Therefore, by controlling the stimulating elements 11, 12, ..., 1n so that the larger the desired force sensation, the larger the rotation angle of the stimulating elements 11, 12, ..., 1n, it is possible to induce a sense of self-motion.

[0067] FIG. 5 is a photograph of a haptic device 5 according to a seventh embodiment. The stimulator elements 11, 12, ..., 1n are composed of a total of 24 stimulator elements. That is, n=24. A set of four stimulator elements among these stimulator elements is grouped into one stimulator unit. The stimulator unit is square, and a stimulator element is arranged at each of the four corners of the stimulator unit. FIG. 5 shows a state in which the stimulator elements 11, 12, 13, and 14 are grouped into a stimulator unit 60. Six stimulator units are arranged in an array of two stimulator units horizontally and three stimulator units vertically on the backrest 80 of the chair 70. That is, the stimulator elements 11, 12, ..., 124 are arranged in an array of four stimulator units horizontally and six stimulator units vertically on the backrest 80 of the chair 70. The interval between adjacent stimulator elements is 55 mm. This is because it has been experimentally verified that the two-point discrimination threshold on the human back is approximately 40 mm, while the distance at which spatially continuous force sensations can be generated is approximately 55 mm. Since the backrest 80 of the chair 70 is not flat but has a gently curved surface, the mount is adjusted so that the stimulation elements on the left and right sides of the backrest 80 are inclined toward the center.

[0068] Each of the stimulation elements 11, 12, ..., 124 is disk-shaped with a diameter of 20 mm and a thickness of 5 mm. If the stimulation elements 11, 12, ..., 124 are too hard, they will not conform to the body, and pain will occur due to contact with the edges of the stimulation elements 11, 12, ..., 124, and slippage will easily occur between the stimulation elements 11, 12, ..., 124 and clothing. Conversely, if the stimulation elements 11, 12, ..., 124 are too soft, they will not transmit rotational force easily. Taking these characteristics into consideration, an appropriate chloroprene rubber sponge is used as the material for the stimulation elements 11, 12, ..., 124.

[0069] A small servo motor is used as the actuator for rotating the stimulation elements 11, 12, ..., 124. The maximum torque that this servo motor can output is 0.2 N·m, which is enough to deform the skin on the back. The stimulation elements 11, 12, ..., 124 are controlled to rotate at a rotation angle of 30 degrees and a rotation speed of 60 degrees / second.

[0070] In one embodiment, the stimulator unit may include a two-axis gimbal. Figure 6 shows a photograph of a stimulator unit 61 that includes a two-axis gimbal, the x-axis and the y-axis, which are perpendicular to each other. As shown in the figure, the stimulator unit 61 can freely rotate about the x-axis and the y-axis independently.

[0071] According to this embodiment, the stimulator unit 61 can rotate along the contours of the body, improving the fit to the body surface and improving the efficiency of transmission of rotational stimulation.

[0072] [Eighth embodiment] The mechanical distribution may be a strain energy density distribution or may be the following: For example, the mechanical distribution is a distribution of strain, which may be a principal strain or an equivalent strain; or, the mechanical distribution is a distribution of force, which may be a shear force or a normal force; or, the mechanical distribution is a distribution of stress, which may be a pressure, a principal stress, or a Mises stress.

[0073] [Ninth embodiment] 1 to 3, the interval between adjacent stimulation elements may be such that the ranges of the force sensations presented by the adjacent stimulation elements overlap. The present inventors have found that by providing such an interval between adjacent stimulation elements, a spatially continuous force sensation can be presented more reliably.

[0074] [Tenth embodiment] Fig. 7 shows a flow of a force sense presentation method according to a tenth embodiment. This force sense presentation method includes step S1 of determining the relationship between the force sense to be presented and the mechanical distribution, and step S2 of applying stimulation to a plurality of points on the skin. The interval between adjacent points on the skin to which stimulation is applied is an interval that can generate a spatially continuous force sense. The force sense generates a person's sense of self-motion.

[0075] In step S1, the force feedback method determines a relationship between a force feedback to be presented and a dynamic distribution that generates the force feedback. The determination of the relationship between a force feedback to be presented and a dynamic distribution that generates the force feedback may be based on a database stored in a storage means or the like, may be based on an external input, or may be based on learning such as machine learning.

[0076] In step S2, the force sense presentation method provides stimuli to a plurality of points on the skin so as to form a mechanical distribution on the skin that generates the force sense. The method of providing the stimuli may be a kinematic method such as rotational motion, pulling in a shear direction, compression, or suction. The stimuli may be provided by means of devices that can provide a force sense by themselves. Alternatively, the stimuli may be provided by means of devices that can provide a force sense only after a plurality of devices work together to form a predetermined mechanical distribution.

[0077] The spacing between adjacent locations of the skin to which stimulation is applied may be, for example, such that the range of haptic sensations presented to the adjacent locations overlap, or the spacing between adjacent locations of the skin to which stimulation is applied may be such that the range of haptic sensations presented to the adjacent locations overlap, or the spacing between adjacent locations of the skin to which stimulation is applied may be such that the range of haptic sensations is within the two-point discrimination threshold for the areas of the skin.

[0078] The self-motion sensation generated by the force sense may be a motion sensation felt when the user is actually moving, or a motion sensation felt when the user is not actually moving.

[0079] The mechanical distribution may be a strain energy density distribution.

[0080] According to this embodiment, by applying a stimulus to the person's skin, it is possible to present a force sense that gives the person the sensation of feeling as if he or she is moving.

[0081] [Eleventh embodiment] Fig. 8 shows functional blocks of a haptic device 6 according to an eleventh embodiment. The haptic device 6 measures a mechanical distribution of the skin S caused by a person's movement and presents a haptic sensation by forming a mechanical distribution on the person's skin S. The haptic device 6 includes n stimulation elements 11, 12, ..., 1n, n mechanical change detection units 11s, 12s, ..., 1ns, a control unit 20, and a mechanical distribution calculation unit 90 (n is an integer of 2 or more). That is, the haptic device 6 includes n mechanical change detection units 11s, 12s, ..., 1ns in addition to the configuration of the haptic device 1 in Fig. 1. Detailed explanations of the configuration common to the haptic device 1 other than the mechanical change detection units 11s, 12s, ..., 1ns will be omitted.

[0082] Each of the mechanical change detection units 11s, 12s, . . . , 1ns detects a mechanical change in the skin S of a person due to the person's movement, and transmits the detection result to the mechanical distribution calculation unit 90.

[0083] The mechanical distribution calculation unit 90 calculates the mechanical distribution of the skin that generates the mechanical changes of the human skin detected by the mechanical change detection units 11s, 12s, ..., 1ns, and transmits the calculation result to the control unit. This mechanical distribution may be a strain energy density distribution, a strain distribution, a force distribution, a stress distribution, or the like.

[0084] The control unit 20 reproduces the mechanical distribution of the skin received from the mechanical distribution calculation unit 90 for each stimulation element. Alternatively, the control unit 20 controls the stimulation elements 11, 12, ..., 1n so as to form the mechanical distribution calculated by the mechanical distribution calculation unit 90 on the skin. The interval between adjacent stimulation elements is an interval that can present a spatially continuous force sensation. The force sensation generates a person's sense of self-motion.

[0085] Specific examples of use of the force sense display device 6 are as follows. The force sense display device 6 is configured with stimulation elements provided on the backrest or cushion. The user takes the force sense display device 6 with them when going for a drive or to an amusement park. The user sets the force sense display device 6 on the seat of a car or playground equipment, and calculates the mechanical distribution of their own back or buttocks while taking images. After returning home, the user is presented with a sense of force from the force sense display device 6 along with the images, allowing them to relive the experience with a high sense of realism.

[0086] When the stimulation element stimulates the skin by rotating, the relationship between the mechanical changes in the skin and the force can be obtained by measuring the rotation angle and the current value at that time, so that, for example, the hardness of the skin can be measured. Furthermore, by arranging these stimulation elements, it is possible to measure, for example, the distribution of the elasticity of the skin. Alternatively, if the stimulation element contact exerts pressure perpendicular to the surface of the skin, the distribution of pressure and elasticity of the skin can be obtained by measuring the current value at that time.

[0087] According to this embodiment, the dynamic change detection unit detects the dynamic change of the skin caused by the actual movement of a person, and then calculates the dynamic distribution of the skin that generates the dynamic change, thereby providing a sense of force that reproduces the sensation of the actual movement.

[0088] [Twelfth embodiment] Fig. 9 shows functional blocks of a haptic device 6 according to a twelfth embodiment. The haptic device 7 measures a mechanical distribution of the skin S caused by a person's movement and presents a haptic sensation by forming a mechanical distribution on the person's skin S. The haptic device 7 includes n stimulation elements 11, 12, ..., 1n, a control unit 20, and a mechanical distribution calculation unit 90 (n is an integer of 2 or more). The stimulation elements 11, 12, ..., 1n of the haptic device 7 include the mechanical change detection unit described above. As described below, the stimulation elements of this embodiment function as a means for applying a stimulus to the skin S, and also function as a detection means (sensor) for detecting a mechanical change in the skin S caused by the person's actual movement.

[0089] In this embodiment, each of the stimulating elements 11, 12, ..., 1n includes a mechanical change detection unit. That is, the stimulating elements 11, 12, ..., 1n detect mechanical changes in the skin S of the person due to the person's movement and provide stimulation to the person's skin. In other words, the stimulating elements also function as mechanical change detection units. Therefore, the stimulating elements 11, 12, ..., 1n function as a means for providing stimulation to the skin S as in the above-mentioned embodiment, and also function as a detection means (sensor) for detecting mechanical changes in the skin S associated with the actual movement of the person. The mechanical changes in the skin S detected by the stimulating elements 11, 12, ..., 1n depend on the method by which the stimulating elements 11, 12, ..., 1n provide stimulation to the skin S. For example, when the stimulating elements 11, 12, ..., 1n provide stimulation to the skin S by a rotational movement, the stimulating elements 11, 12, ..., 1n detect the rotation of the skin S as a mechanical change in the skin S associated with the person's movement. Alternatively, when the stimulation elements 11, 12, ..., 1n stimulate the skin S by pulling in the shear direction, the stimulation elements 11, 12, ..., 1n detect the stretching or contraction of the skin S in the shear direction as a mechanical change in the skin S accompanying the person's movement. Alternatively, when the stimulation elements 11, 12, ..., 1n stimulate the skin S by compression or suction, the stimulation elements 11, 12, ..., 1n detect the distortion of the skin S caused by the skin S being compressed or suctioned as a mechanical change in the skin S accompanying the person's movement, and the control unit 20 stores this.

[0090] The principle by which the stimulating elements 11, 12, ..., 1n detect mechanical changes in the skin S due to a person's movement is as follows. Suppose that the stimulating elements 11, 12, ..., 1n of the haptic device according to the above-mentioned embodiment apply stimuli to the skin S that form a mechanical distribution that induces a certain sense of self-motion. At this time, when a user wearing this haptic device actually performs the movement, each of the stimulating elements 11, 12, ..., 1n is considered to receive an action in the opposite direction to the stimulus to be applied to the skin S, or to generate a back electromotive force due to the action in the opposite direction. By measuring such an action in the opposite direction or the back electromotive force, the stimulating elements 11, 12, ..., 1n can detect mechanical changes in the skin S due to the person's actual movement.

[0091] According to this embodiment, since the stimulation element also functions as a mechanical change detection unit, a compact and inexpensive force feedback device can be realized with a small number of components.

[0092] [Thirteenth embodiment] 10 shows functional blocks of a haptic device 8 according to a thirteenth embodiment. The haptic device 8 also measures a mechanical distribution of the skin S due to human movement, and presents a mechanical sense by forming a mechanical distribution on the human skin S. The haptic device 7 includes n stimulation elements 11, 12, ..., 1n, n sensors 11ss, 12ss, ..., 1nss, a control unit 20, and a mechanical distribution calculation unit 90 (n is an integer equal to or greater than 2).

[0093] Each of the sensors 11ss, 12ss, . . . , 1nss detects a mechanical change in the skin S of a person due to the person's movement, and transmits the detection result to the mechanical distribution calculation unit 90.

[0094] The sensors 11ss, 12ss, ..., 1nss may be any suitable sensor capable of detecting a mechanical change in the skin caused by human movement, such as a speed sensor, an acceleration sensor, or a pressure sensor. In the haptic device 7 in Fig. 9, the stimulator also serves as the mechanical change detector. In contrast, the mechanical change detector of the haptic device 8 is realized by a sensor provided independently of the stimulator unit.

[0095] According to this embodiment, since mechanical changes in the skin can be detected using any suitable sensor, a more accurate and high-performance force feedback device can be realized.

[0096] [Fourteenth embodiment] Fig. 11 shows functional blocks of a haptic device 9 according to a fourteenth embodiment. The haptic device 9 also measures a mechanical distribution of the skin S due to human movement and forms a mechanical distribution on the human skin S to present a haptic sensation. The haptic device 8 includes n stimulation elements 11, 12, ..., 1n, n mechanical change detection units 11s, 12s, ..., 1ns, a control unit 20, a mechanical distribution calculation unit 90, a camera 100, and a video synchronization unit 110 (n is an integer of 2 or more). That is, the haptic device 9 includes a camera 100 and a video synchronization unit 110 in addition to the configuration of the haptic device 6 in Fig. 8. Detailed explanations of the configuration common to the haptic device 6 other than the camera 100 and the video synchronization unit 110 will be omitted.

[0097] The dynamic change detection units 11s, 12s, ..., 1ns detect the dynamic change of the skin S accompanying the movement of the person, and transmit the detection result to the video synchronization unit 110. The camera 100 captures a subjective image seen from the person operating the camera, that is, an image captured from the viewpoint of the person, and transmits the image to the video synchronization unit 110. The video synchronization unit 110 calculates the relationship between the dynamic change of the person's skin and the change of the subjective image. That is, the video synchronization unit 110 calculates what kind of dynamic change the person's skin will undergo when the subjective image undergoes a certain change. For example, what kind of dynamic change will the skin undergo when the person accelerates relative to the surrounding scenery in the subjective image, what kind of dynamic change will the skin undergo when the person tilts relative to the surrounding scenery in the subjective image, and so on. The video synchronization unit 110 transmits the calculated relationship between the dynamic change of the person's skin and the change of the subjective image to the dynamic distribution calculation unit 90.

[0098] The mechanical distribution calculation unit 90 calculates the mechanical distribution of the skin that generates the mechanical change from the mechanical change of the person's skin when the subjective image changes, based on the relationship between the mechanical change of the person's skin calculated by the image synchronization unit 110 and the change in the subjective image. In this way, the haptic device 7 presents a force that generates a sense of self-motion that the person operating the camera feels when the image seen from the person's viewpoint changes.

[0099] According to this embodiment, it is possible to present a more realistic sense of self-motion that is synchronized with the video.

[0100] [Fifteenth embodiment] Fig. 12 shows a flow of a force feedback method according to a fifteenth embodiment. This force feedback method includes step S3 of detecting a mechanical change in the skin caused by human movement, step S4 of calculating a mechanical distribution of the skin that generates the mechanical change, and step S5 of applying stimulation to multiple points on the skin. The interval between adjacent points on the skin to which stimulation is applied is an interval that can generate a spatially continuous force feedback. The force feedback generates a person's sense of self-motion.

[0101] In step S3, the force feedback method detects a mechanical change in the skin of a person caused by the movement of the person using a sensor or the like. The mechanical change in the skin caused by the movement may be a rotation of the skin, a stretch or contraction of the skin in a shear direction, a distortion of the skin caused by pressure or suction, etc.

[0102] In step S4, the force feedback method calculates, from the mechanical change in the human skin detected in step S3, a mechanical distribution of the skin that generates the mechanical change.

[0103] In step S5, the force feedback method reproduces the mechanical change detected in step 3. Alternatively, in step S5, the force feedback method applies stimulation to multiple points on the skin so as to form the mechanical distribution calculated in step S4 on the skin.

[0104] According to this embodiment, it is possible to detect the dynamic changes of the skin caused by the actual movement of a person and calculate the dynamic distribution of the skin that generates the dynamic changes, thereby providing a sense of force that reproduces the sensation of the actual movement.

[0105] [Sixteenth embodiment] Fig. 13 shows a flow of a force feedback method according to the sixteenth embodiment. This force feedback method includes step S3 of detecting a mechanical change in the skin caused by human movement, step S6 of capturing a subjective image, step S7 of calculating the relationship between the mechanical change in the skin and the change in the subjective image, step S4 of calculating a mechanical distribution in the skin that generates a mechanical change, and step S5 of applying stimulation to a plurality of points on the skin. That is, this method includes steps S6 and S7 in addition to the method of Fig. 12. Detailed explanations of the processes common to the method of Fig. 12 other than steps S6 and S7 will be omitted.

[0106] In step S3, the force feedback method detects mechanical changes in the person's skin caused by the person's movement using a sensor or the like.

[0107] In step S6, the force feedback method uses a camera or the like to capture a subjective image seen by a person operating the camera, that is, an image captured from the viewpoint of the person.

[0108] In step S7, the force feedback method calculates the relationship between the dynamic changes in the skin of the person photographed by the camera and the changes in the subjective image. That is, the process in step S7 calculates the dynamic changes in the skin of the person when the subjective image undergoes a certain change. For example, the dynamic changes in the skin are calculated when the person accelerates relative to the surrounding scenery in the subjective image, and when the person tilts relative to the surrounding scenery in the subjective image.

[0109] In step S4, this force feedback method calculates, based on the relationship between the calculated mechanical change in the person's skin and the change in the subjective image, a mechanical distribution of the skin that generates the mechanical change from the mechanical change in the person's skin when the subjective image changes.

[0110] In step S5, the force feedback method stimulates a plurality of points on the person's skin so as to form the force distribution calculated in step S4 on the person's skin.

[0111] According to this embodiment, it is possible to present a more realistic sense of self-motion that is synchronized with the video.

[0112] [Seventeenth embodiment] In the above embodiment, the stimulating element is an example of stimulating a body part that moves relatively little, such as the back or buttocks. However, the present invention is not limited to this, and the stimulating element may stimulate a body part that moves a lot, such as the hands or feet. In this case, the control unit may control the stimulating element so as to form a mechanical distribution on the skin according to the stretching of the skin that occurs when the body part is moved.

[0113] According to this embodiment, for example, a user who is sitting motionless in a chair can be given the sensation of self-motion as if he or she were moving his or her own limbs.

[0114] [Eighteenth embodiment] The stimulation element may give the person wearing the stimulation element a sensation accompanied by, for example, touching, tracing, grabbing, or a movement such as a small animal running around on the body, or a spatial flow. At this time, the rotating stimulation element in the stimulation element unit is controlled as follows. That is, a stimulation point for providing a stimulus is first given to the person wearing the stimulation element, and the stimulation point is moved on the skin of the person. Then, each stimulation element is controlled so that the closer it is to the stimulation point, the larger the rotation angle, and the farther it is from the stimulation point, the smaller the rotation angle. Specifically, the control unit moves the stimulation point for providing a stimulus on the skin of the person, and controls the stimulation element so that the smaller the distance from the stimulation point, the larger the rotation angle of the stimulation element. By performing such control, it is possible to provide the user with a sense of movement as if the stimulation point moved smoothly or flowed. By appropriately defining the relationship between the distance between the stimulation point and the stimulation element and the rotation angle of the stimulation element, a person can be given a sensation with a desired spatial flow, such as being touched, traced, grabbed, or as if a small animal were running around on the body.

[0115] The embodiments of the present invention have been described in detail above. These embodiments are merely examples, and it will be understood by those skilled in the art that various modifications and changes are possible within the scope of the claims of the present invention, and that such modifications and changes are also within the scope of the claims of the present invention. Therefore, the descriptions and drawings in this specification should be treated as illustrative rather than restrictive.

[0116] [Variations] The following describes the modified examples. In the drawings and description of the modified examples, the same components and members as those in the embodiment are denoted by the same reference numerals. Explanations that overlap with the embodiment will be omitted as appropriate, and the description will focus on the configurations that differ from the first embodiment.

[0117] In the embodiment, the stimulating element unit is illustrated as a square having two stimulating elements in the horizontal direction and two in the vertical direction. However, the present invention is not limited to this, and the stimulating element unit may be, for example, a stimulating element unit having three stimulating elements in the horizontal direction and three stimulating elements in the vertical direction, or a stimulating element unit having four stimulating elements in the horizontal direction and four stimulating elements in the vertical direction. Alternatively, the stimulating element unit may be a rectangular stimulating element unit having two stimulating elements in the horizontal direction and three stimulating elements in the vertical direction. This modification can increase the degree of freedom of the configuration.

[0118] In the embodiment, an example in which the haptic device is used alone has been shown. However, the present invention is not limited to this, and multiple haptic devices may be used in combination at different parts of the body. Also, an HMD (head mounted display) on which an image is displayed may be used in combination with the haptic device. This provides a more realistic sense of self-motion.

[0119] Each of these modified examples provides the same functions and effects as the embodiment.

[0120] Any combination of the above-described embodiments and modifications is also useful as an embodiment of the present invention. A new embodiment resulting from the combination has the combined effects of each of the combined embodiments and modifications. [Industrial Applicability]

[0121] The present invention can be used in a force feedback device and a force feedback method that provide a force feedback by applying a stimulus to a person's skin. [Explanation of symbols]

[0122] 1...force sense presentation device 2...force sense presentation device 3...force sense presentation device 4...force sense presentation device 5...force sense presentation device 6...force sense presentation device 7...force sense presentation device 8...force sense presentation device 9...force sense presentation device 11. Stimulator 12. Stimulator 13. Stimulator 14. Stimulator 1n stimulation element 20 Control section 30...Storage section 40...force sense indicator 50. Learning Department 60··Stimulator unit 61 Stimulator unit 70··Chair 80··Backrest 90... Mechanical distribution calculation section 100··Camera 110 Video synchronization section S··Skin S1: Step to determine the relationship between force sensation and mechanical distribution S2: Stimulating multiple points on the skin S3: Detecting mechanical changes in the skin caused by human movement S4: A step of calculating the mechanical distribution of the skin that generates the mechanical change. S5: Stimulating multiple points on the skin S6: Steps for shooting subjective footage S7: A step of calculating the relationship between the dynamic change of the skin and the change of the subjective image.

Claims

1. A force feedback device that provides a force feedback by forming a mechanical distribution on a person's skin, A plurality of stimulation elements and a control unit are provided, each of the plurality of stimulation elements applies a stimulus to the person's skin; The control unit controls the plurality of stimulation elements so as to form a dynamic distribution on the skin that generates a desired dynamic sensation; The interval between adjacent stimulation elements is an interval that allows a spatially continuous force sensation to be presented; the mechanical distribution is a strain energy density distribution, The stimulation element is a rotational stimulation element that stimulates the skin by a rotational motion, A force feedback device characterized in that the rotational stimulation element forms a strain energy density distribution on the skin by contacting the skin on a plane perpendicular to its rotation axis and twisting the skin.

2. 2. The force feedback device according to claim 1, wherein the interval between adjacent rotational stimulation elements is such that the ranges of the force feedback provided by the adjacent rotational stimulation elements overlap.

3. 2. The force feedback device according to claim 1, wherein the interval between adjacent rotational stimulation elements is within a range of a two-point discrimination threshold in the region of the human skin.

4. A force feedback method for providing a force feedback by forming a mechanical distribution on a person's skin, comprising: determining a relationship between the force sensation and a force distribution that generates the force sensation; and applying stimulation to a plurality of points on the skin so as to form a mechanical distribution on the skin that generates the force sensation; the spacing between adjacent stimulated points on the skin is such that a spatially continuous force sensation can be generated; the mechanical distribution is a strain energy density distribution, A force feedback method characterized in that the strain energy density distribution is formed on the skin by a rotational stimulation element that stimulates the skin through rotational motion, contacting the skin in a plane perpendicular to the rotation axis of the rotational stimulation element and twisting the skin.

5. 5. The force feedback method according to claim 4, wherein the interval between adjacent rotational stimulation elements is such that the ranges of the force feedback provided by the adjacent rotational stimulation elements overlap each other.

6. 5. The force feedback method according to claim 4, wherein the interval between adjacent rotational stimulation elements is within a range of a two-point discrimination threshold in the region of the human skin.

Images