Sensory control device, sensory control method, sensory control system
The sensory control system addresses inconsistent sensations by using a rotary operation section with detection and signal generation mechanisms to adjust sensations based on size and mass, ensuring consistent user feedback.
Patent Information
- Application Number
- JP2022196779
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-05-13
- Filing Date
- 2022-12-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-05-19
AI Technical Summary
Conventional sensory presentation technologies fail to account for the physical characteristics of operating sections, such as size and mass, resulting in inconsistent sensations for users.
A sensory control system that includes a rotary operation section with an operation detection mechanism, signal generation based on physical properties, and an adjustment unit to tailor sensations according to the mass and size of the rotary operation section, ensuring consistent reaction forces regardless of diameter or mass.
The system provides tailored sensory experiences that align with the physical characteristics of the operating unit, enhancing user interaction by maintaining consistent reaction forces across varying sizes and masses.
Smart Images

Figure 0007585575000008 
Figure 0007585575000009 
Figure 0007585575000010
Abstract
Description
[Technical field]
[0001] The present invention relates to a sensory control device, a sensory control method, and a sensory control system. [Background technology]
[0002] Conventionally, there are known operation units that provide a sense by providing some kind of stimulus to a person. Here, the sense is provided by a tactile sense, an auditory sense by sound, or a visual sense by displaying an image. The sense is adjusted by adjusting a signal that drives various operation units.
[0003] There is known a game controller in which buttons with built-in vibration devices can be replaced (see, for example, Patent Document 3). Patent Document 3 discloses a technique for replacing the vibration device itself in order to realize different vibration intensities. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2019-220168 A [Patent Document 2] Patent No. 5662425 [Patent Document 3] Special Publication No. 2020-523068 [Patent Document 4] Special Publication No. 2013-519961 Summary of the Invention [Problem to be solved by the invention]
[0005] However, the conventional technology has a problem in that it does not adequately present sensations according to the physical characteristics of the operation unit. For example, in the case of a rotary operation unit, the sensations conveyed to the user operating the operation unit differ depending on the size and mass of the operation unit, even if the actuator is driven in the same way.
[0006] In view of the above-mentioned problems, an object of the present aspect is to provide a technology for presenting sensations according to the physical characteristics of an operating unit. [Means for solving the problem]
[0007] In view of the above problems, the present invention provides a sensory presentation device that includes a rotary operation unit, an operation detection unit that detects an operation of the rotary operation unit to generate an operation signal, a signal generation unit that generates a sensory presentation signal based on the operation signal, and a sensory presentation unit that presents a sensory presentation to an operator based on the sensory presentation signal, the sensory presentation unit being configured to provide at least one of the operation signal, the sensory presentation signal, and the sensory presentation when operating the rotary operation unit based on the mass and size of the rotary operation unit as physical properties. Regardless of the diameter, the reaction force of the rotation operation is about the same. The larger the diameter of the rotary operation part, the greater the torque required for the operator to rotate it. Large Also, So that the reaction force of the rotation operation is the same regardless of the mass, The larger the mass of the rotary operation unit, the greater the torque required for the operator to rotate it. Small The present invention provides a sensory control device equipped with an adjustment unit that adjusts the sensory control device so that the sensory control device is operated in a controlled manner. Effect of the Invention
[0008] It is possible to provide a technology that presents sensations according to the physical characteristics of the operating unit. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a block diagram showing a basic configuration of a sensation control system according to an embodiment of the present disclosure. [Diagram 2] 1 is a block diagram showing a haptic control system as a first embodiment of a sensory control system according to the present disclosure. FIG. [Diagram 3] 3 is an explanatory diagram illustrating an example of the configuration of a tactile presentation unit included in the tactile control system of FIG. 2, using an equivalent circuit that uses an operator of Laplace transform. FIG. [Figure 4] 3 is an explanatory diagram showing an equivalent model of an example of a tactile presentation unit included in the tactile control system of FIG. 2. [Diagram 5] 5 is an explanatory diagram illustrating an equivalent circuit and an internal structure of the example of the actuator shown in FIG. 4. [Figure 6] 1 is a flowchart illustrating a conversion model generation method using the conversion model generation system of the present disclosure. [Figure 7] 1 is a flowchart illustrating a specific example of a transformation model generation method and a tactile presentation method according to the present disclosure. [Figure 8] 11 is a flowchart illustrating a haptic control method using the haptic control system of the present disclosure. [Figure 9] FIG. 11 is an explanatory diagram of the physical characteristics of a pressure type operating tool. [Figure 10] 11 is an explanatory diagram showing an example of physical parameters of a pressure type operating tool. FIG. [Figure 11] 13A to 13C are explanatory diagrams illustrating the operation of a pressure type operating tool. [Figure 12] 10 is an explanatory diagram showing the relationship between sensory parameters and physical parameters in a pressure-type operating tool. FIG. [Figure 13] 10 is an explanatory diagram showing the relationship between sensory parameters and physical parameters in a pressure-type operating tool. FIG. [Figure 14] 10 is an explanatory diagram showing the relationship between sensory parameters and physical parameters in a pressure-type operating tool. FIG. [Figure 15] 10 is an explanatory diagram showing the relationship between sensory parameters and physical parameters in a pressure-type operating tool. FIG. [Figure 16] FIG. 13 is an explanatory diagram of the physical characteristics of a pressure type operating tool. [Figure 17] 10 is an explanatory diagram showing the relationship between sensory parameters and physical parameters in a rotary operating tool; FIG. [Figure 18] 1 is an explanatory diagram of physical characteristics of a rotary operating tool. [Figure 19] FIG. 2 is a block diagram showing the configuration of a rotation operation mold tool. [Figure 20] FIG. 2 is a block diagram illustrating a haptic control system as a second embodiment of a sensation control system according to an embodiment of the present disclosure. [Figure 21] 21 is a sequence diagram showing the operation of the haptic control system shown in FIG. 20. [Figure 22] FIG. 2 is a diagram illustrating a first relational expression according to an example of a conversion model generating method of the present disclosure. [Figure 23] FIG. 13 is a diagram illustrating a second relational expression according to an example of the conversion model generating method of the present disclosure. [Figure 24] 11A and 11B are diagrams showing examples of changes over time in strength of a drive signal supplied to a weight based on a tactile sensation signal. [Diagram 25] FIG. 1 is a perspective view of a haptic control device. [Figure 26] This is an example of a client-server type haptic control system. [Figure 27] 1 is a diagram showing an example of an outline of an operation in which a user adjusts an operational feel using a haptic control device; [Figure 28] 1 is a diagram showing an example of an outline of an operation in which a user adjusts an operational feel using a haptic control device; [Figure 29] FIG. 2 is an example of a functional block diagram illustrating functions of a haptic control device. [Diagram 30] FIG. 11 is a flowchart illustrating an example of a learning flow in generating a classification unit. [Diagram 31] FIG. 11 is a flowchart showing a flow of learning the correspondence between the expression frequency of the affective parameter and the physical parameter. [Diagram 32] 11 is an example of a flowchart illustrating a flow in which the tactile control device presents a user's preferred operation feel by using a classification unit and first to third conversion models. FIG. [Diagram 33] 1A and 1B are diagrams showing an overview of an operation in which a user adjusts an operational feel using a haptic control device. [Diagram 34] FIG. 2 is an example of a functional block diagram illustrating functions of a haptic control device. [Diagram 35] FIG. 13 is a flowchart showing an example of a flow of learning a physical parameter (load-displacement curve) corresponding to an expression frequency. [Diagram 36] FIG. 13 is a flowchart illustrating an example of a flow of curve fitting a load-displacement curve of a reference operating tool. [Figure 37]11 is a flowchart illustrating an example of a flow in which the haptic control device presents a user's preferred operation feel by using a physical parameter conversion unit and a comparison unit. [Figure 38] FIG. 13 is a diagram illustrating an example of a neural network in which a classification unit is realized by a neural network. [Figure 39] FIG. 13 is a diagram showing an example of a decision tree when a classification unit is realized by a decision tree. [Diagram 40] FIG. 13 is a diagram illustrating a first input screen for expression frequency. [Diagram 41] 1 is an example of a functional block diagram of a haptic control system in which the haptic control device of the first embodiment is applied to a client-server system. [Diagram 42] FIG. 11 is an example of a sequence diagram illustrating an operation of the haptic control system. [Diagram 43] FIG. 13 is an example of a functional block diagram of a haptic control system in which the haptic control device of the second embodiment is applied to a client-server system. [Diagram 44] 13 is an example of a sequence diagram illustrating the operation of a haptic control system of a second embodiment. [Diagram 45] FIG. 13 is a diagram showing the configuration of a tactile control system of the sensation control system (Example 2). [Diagram 46] FIG. 11 is a diagram showing an example of an operation unit parameter. [Figure 47] 11A and 11B are diagrams illustrating differences in physical characteristics of the operation unit. [Figure 48] 1A to 1C are diagrams illustrating several methods for detecting the size and mass of an operation unit using an operation unit sensor. [Figure 49] 11A and 11B are diagrams illustrating a method for estimating the mass of an operation unit by calibration. [Figure 50] 11A and 11B are diagrams illustrating correction of the mass of the operation unit. [Figure 51] 11 is a flowchart showing a process of adjusting a haptic presentation signal in accordance with physical parameters of an operation unit to which the haptic control system is attached. FIG. [Figure 52]FIG. 11 is a flowchart showing a process of adjusting a haptic presentation signal in accordance with physical parameters of an operation unit to which the haptic control system is attached (modification). [Diagram 53] FIG. 46 is a diagram showing the configuration of a tactile control system as a second embodiment of the sensation control system shown in FIG. 45, together with a signal flow. [Figure 54] 11 is a sequence diagram showing how the communication device and the terminal device communicate with each other to estimate affective parameters of the attached operation unit. FIG. [Figure 55] 11A and 11B are diagrams illustrating static characteristics obtained by a rigid pressing tool and dynamic characteristics obtained by a finger model pressing tool in which a rigid body and an elastic body are integrated. [Figure 56] 11A and 11B are diagrams illustrating the relative positions of a finger and an operating tool when the finger is deformed. [Figure 57] FIG. 13 is a diagram illustrating a finger model pressing tool. [Figure 58] 11A and 11B are diagrams illustrating generation of a sensation providing signal that provides a clicking sensation. [Figure 59] 1 is a functional configuration diagram and a block diagram of a pressure type operating tool according to an embodiment of the present invention; [Figure 60] 11A and 11B are diagrams illustrating dynamic characteristics when the operating tool is pressed by the finger model pressing tool. [Figure 61] 13A to 13C are diagrams illustrating the transition over time of the relative positions of the finger model pressing tool and the operating tool. [Figure 62] FIG. 13 is a diagram for explaining dynamic characteristics in more detail along with periods A to C. [Figure 63] 11 is a diagram showing an example of dynamic characteristics of a plurality of operating tools with different dynamic characteristics when pressed by a finger model pressing tool; FIG. [Figure 64] FIG. 11 is a flowchart illustrating an example of a flow of determining a physical parameter correlated with an affective parameter. [Figure 65] FIG. 13 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool for the affective parameter "feeling of recovery (not feeling of recovery)", acquired by the processor in step ST153. [Figure 66]FIG. 13 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool for the affective parameter "feeling of being sucked in (not felt)", acquired by the processor in step ST153. [Figure 67] FIG. 13 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool for the affective parameter "feeling of recovery (not feeling of recovery)", acquired by the processor in step ST153. [Figure 68] FIG. 13 is a diagram showing a list of correlation coefficients between each sensory parameter and each dynamic characteristic. [Figure 69] 11 is an example of a sequence diagram in which a communication device and a terminal device communicate with each other to estimate affective parameters of a worn operating tool. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant description will be omitted. [Aspect 1]
[0011] (Sensory Control System) FIG. 1 shows a basic configuration of a sensory control system 100 according to the first embodiment of the present disclosure. The sensory control system 100 shown in FIG. 1 includes a sensibility database 16, a storage unit 11, an input unit 4, a processor 101, and a sensory presentation unit 102. The storage unit 11 stores a sensory parameter-physical parameter conversion model (hereinafter, simply referred to as a "conversion model 15"). The sensory presentation unit 102 is a component that presents a sense to a person, and can be configured, for example, as a tactile presentation unit (for example, a tactile presentation unit 30 described later) that presents a sense of touch, an auditory presentation unit such as a speaker that presents an auditory sense, a visual presentation unit such as a display device that presents a visual sense, or any combination of these.
[0012] The conversion model 15 is a conversion model capable of converting a sensibility parameter into a physical parameter correlated with the sensibility parameter. Here, the sensibility parameter is a parameter indicating the degree of sensibility expression for a sensory presentation. Specifically, in the case of a sensibility evaluation by, for example, the Semantic Differential Method (SD), the sensibility parameter may be a parameter indicating which of two sensory expressions (adjectives, onomatopoeias, sound symbolic words, etc.) the presented sensation is closer to, with a multi-level evaluation. Specifically, the combination of two sensory expressions is "comfortable-uncomfortable" or "light-heavy". For example, the multi-level evaluation by the SD method may be expressed by setting the expression level of the sensibility parameter of the "most comfortable" to "1", and increasing the expression level to "2", "3", "4", and so on, toward "uncomfortable", with "7" being the "least comfortable". The sensibility parameter is not limited to a combination of two sensory expressions, and may be the intensity of one sensory expression. Also, the sensibility parameter may be a parameter expressed in a multidimensional manner by taking multiple axes of sensory expression and combining these multiple axes. The physical parameters are included in the physical characteristics related to the sensation presentation, and there are a plurality of types. The physical characteristics related to the sensation presentation are physical characteristics that can affect the entire sensory transmission system including the sensation presentation means, such as the sensation presentation unit 102, and the body parts of the person when presenting a sensation to the person. In other words, the physical characteristics related to the sensation presentation are not limited to the physical characteristics of the sensation presentation means, and can also include the physical characteristics of the body parts of the person to which the sensation is presented.
[0013] Here, the sensibility database 16 is described as being stored in a storage unit (not shown) other than the storage unit 11, but the sensibility database 16 may be stored in the storage unit 11. The processor 101 controls the operation of the entire sensory control system 100. The processor 101 is a general term for one or more processors. For example, the components of the sensory control system 100 may be shared and controlled by a plurality of processors, or all the components may be controlled by one processor. In addition, the components of the sensory control system 100 may be capable of transmitting information to each other so as to be able to execute a conversion model generation method and a sensory control method, which will be described later, and the connection method is not particularly limited. For example, the connection method of the components of the sensory control system 100 may be a wired connection or a wireless connection including a network connection. The sensory control system 100 may be composed of a plurality of devices, or may be a single device.
[0014] The conversion model 15 included in the sensory control system 100 is obtained by the following conversion model generation method. In the conversion model generation method, first, the sensitivity database 16 stores, for one or more types of sensory presentation, correspondence information in which physical characteristics related to a predetermined sensory presentation are associated with sensitivity parameters indicating the degree of sensory expression for the sensory presentation (storage step). The processor 101 extracts physical parameters correlated with the sensitivity parameters from among multiple types of physical parameters included in the physical characteristics related to the sensory presentation based on the correspondence information for each of the one or more types of sensory presentation in the sensitivity database 16 (extraction step). After that, the processor 101 generates the conversion model 15 based on the sensitivity parameters and the extracted physical parameters (generation step). The conversion model 15 thus generated is a conversion model capable of converting a newly accepted sensitivity parameter into a physical parameter correlated with the sensitivity parameter. When executing the above-mentioned conversion model generation method, the sensory control system 100 functions as a conversion model generation system. In addition, in the extraction step, in order to derive multiple types of physical parameters included in the physical characteristics related to sensory presentation, extraction may be performed from physical characteristics related to the sensory presentation means, or from physical characteristics of a system including human body parts.
[0015] The conversion model generation method may be executed by a conversion model generation system other than the sense control system 100. In this case, the conversion model generation system includes at least a sensibility database 16 and a processor 101. The sense control system 100 may acquire a conversion model 15 obtained by the execution of the conversion model generation method by the other conversion model generation system, and store the conversion model 15 in the storage unit 11. In this case, the sense control system 100 may not include the sensibility database 16.
[0016] In addition, the above-mentioned correspondence information stored in the sensitivity database 16 may be updateable, and the conversion model 15 may also be updateable based on the updated correspondence information. In detail, in the storage step of the conversion model generation method, the sensitivity database 16 adds or updates the above-mentioned correspondence information for one or more types of sensory presentation. Next, in the extraction step, the processor 101 re-extracts physical parameters correlated with the sensitivity parameters based on the correspondence information for each of the one or more types of sensory presentation in the sensitivity database 16. Thereafter, in the generation step, the processor 101 updates the conversion model 15 based on the sensitivity parameters and the newly extracted physical parameters.
[0017] The sense control system 100 executes the following sense control method. First, the sense control system 100 accepts an input of a sensory parameter from a user or the like via the input unit 4 (reception step). After that, the processor 101 converts the accepted sensory parameter into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in the physical characteristics related to sensory presentation based on the conversion model 15 (conversion step). Then, the processor 101 generates a sensory presentation signal based on the converted physical parameter, and outputs it to the sense presentation unit 102 (output step). The sense presentation unit 102 presents a sense to the user or the like based on the sensory presentation signal (sense presentation step).
[0018] In this way, the sensory control system 100 can present a sensation to a user, etc. based on a sensory presentation signal based on physical parameters that correlate with the received sensory parameters, and can therefore present a sensation that reflects human sensibilities to a user, etc.
[0019] (Tactile Control System 1) FIG. 2 shows the configuration of a haptic control system 1 as a first embodiment of the sensation control system 100 shown in FIG. 1, together with the flow of signals.
[0020] The haptic control system 1 shown in Fig. 2 has a main control device 10. The main control device 10 is a personal computer, a server, or the like, and has a processor (CPU) 14 and a storage unit 11 such as RAM or ROM. The main control device 10 is provided with calculation function units 12 and 13 executed by the processor 14.
[0021] The haptic control system 1 shown in Fig. 2 has an input / output device 3. The input / output device 3 includes an input unit 4, a display unit 5, and a processor that operates the input unit 4 and the display unit 5. The input / output device 3 and the main control device 10 are connected via various interfaces.
[0022] The tactile control system 1 includes a tactile presentation device 20. The tactile presentation device 20 includes a terminal processor 18 that controls the operation of the tactile presentation device 20. The arithmetic function unit 13 that functions as an output unit of the main control device 10 and the tactile presentation device 20 are connected via an interface such as a cable and a connector, USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface, registered trademark), Ethernet (registered trademark), or Wi-Fi.
[0023] The storage unit 11 of the main control device 10 shown in FIG. 2 stores a conversion model 15. As described in the explanation of the sensory control system 100 in FIG. 1, the conversion model 15 is a conversion model capable of converting the received sensory parameter into a physical parameter correlated with the sensory parameter. The sensory parameter in this example is a parameter indicating the degree of sensory expression for the tactile presentation. For example, the sensory parameter in this example may be an evaluation by a user of the operation feeling when a specific operating tool is operated in an expression based on the sensitivity. In other words, the sensory parameter in this example is inputted reflecting the operation of the specific operating tool. The physical parameter in this example is included in the physical characteristics related to the tactile presentation, and there are a plurality of types. For example, the physical parameter in this example may be a physical parameter included in the physical characteristics that realize the tactile presentation when the specific operating tool is operated. The physical parameter in this example can be used to operate the tactile presentation device 20 to reproduce the sensory expression of the specific operating tool.
[0024] The tactile presentation device 20 includes at least a tactile presentation unit 30. The tactile presentation device 20 controls the tactile presentation unit 30 based on a tactile presentation signal to present a tactile sensation to the user. Here, the tactile presentation unit 30 is an example of the sensation presentation unit 102 in FIG. 1.
[0025] The tactile sense providing unit 30 may provide a tactile sense by generating a resistance or vibration. Examples of the tactile sense providing unit 30 that generates a resistance or vibration include a voice coil motor (VCM), a linear actuator (either a resonant type or a non-resonant type), a piezoelectric element, an eccentric motor, a shape memory alloy, a magnetorheological fluid, and an electroactive polymer.
[0026] The tactile sensation providing unit 30 may provide a sense of touch by providing a sense of warmth and cold. An example of the tactile sensation providing unit 30 that provides a sense of warmth and cold is a Peltier element. The Peltier element utilizes the transfer of heat caused by the Peltier effect when a direct current is applied to two opposing metal plates, and the amount of heat on the surface of the metal plates changes depending on the direction of the current. By controlling the direction and amount of the current, it is possible to make a part of the body, such as the user's fingers, that touches the Peltier element feel a warm or cold temperature.
[0027] The tactile presentation unit 30 may present a tactile sensation by applying an electric stimulus. An example of the tactile presentation unit 30 that applies an electric stimulus is a configuration that applies an electric stimulus by capacitively coupling with a body part such as a user's fingertip. The tactile presentation unit 30 may present an aerial tactile sensation. An example of the tactile presentation unit 30 that presents an aerial tactile sensation is a configuration that generates air vibrations using ultrasound or the like, and causes a body part such as a user's fingertip to resonate with the air vibrations to present a tactile sensation.
[0028] As shown in Fig. 2, the haptic control system 1 may include an operation device 33, and the haptic presentation unit 30 may present a haptic sensation to a user who operates the operation device 33. The haptic presentation unit 30 may present a predetermined operation sensation to a user who operates the operation device 33 by presenting a haptic sensation to the user. Specifically, the haptic presentation unit 30 may present an operation sensation that imitates the operation sensation of a predetermined operation tool. For example, examples of operation tools that imitate the operation sensation include a push switch that accepts a pressing operation, a rotary switch that accepts a rotating operation, a joystick that accepts a tilting operation, a slide switch that accepts a sliding operation on a slide operation unit, and a touch panel that accepts a touch operation, a pressing operation, a tracing operation, etc. on an operation panel.
[0029] Any type of device capable of performing the same operation as the above-mentioned predetermined operating tool can be used as the operating device 33. Specifically, the operating device 33 may be in a form imitating the predetermined operating tool, or may be in a form unrelated to the predetermined operating tool, such as an operating device such as an operating glove that is worn on the user's hand and receives operations by finger movements, etc.
[0030] The tactile sense providing unit 30 may provide a tactile sense to the user regardless of the operation of the operation device 33. In that case, the tactile sense control system 1 does not need to include the operation device 33.
[0031] 2, the tactile presentation device 20 may include various sensors such as a position sensor 27 and an acceleration sensor 28. By including various sensors, the tactile presentation device 20 can detect at least one physical quantity of the tactile presentation device 20 itself, the operation device, and a body part of the user, and control the driving of the tactile presentation unit 30 based on the physical quantity. In addition to the above, other sensors that can be used include, for example, a torque sensor, an angular velocity sensor, a temperature sensor, a pressure sensor (including an air pressure sensor), a humidity sensor, a magnetic sensor, an optical sensor, an ultrasonic sensor, and an electromyogram sensor.
[0032] [An example of the tactile presentation unit 30] An example of the tactile presentation unit 30 included in the tactile control system 1 according to this embodiment will be described with reference to Figs. 3 to 5. The tactile presentation unit 30 illustrated in Figs. 3 to 5 reproduces the tactile sensation when operating a pressure-type operating tool, and the model pressure-type operating tool is a pressure-type operating tool such as a Tact Switch (registered trademark) in which a dish-shaped leaf spring or a dome-shaped leaf spring generates an operating reaction force. The tactile presentation unit 30 reproduces a tactile sensation corresponding to a desired sensitivity parameter based on a tactile presentation signal provided from the main control device 10. By incorporating the tactile presentation unit 30 into the electronic circuits of various devices, the tactile presentation unit 30 can be used in place of an actual pressure-type operating tool as a pressure-type operating tool that realizes a tactile sensation (here, an operating sensation) corresponding to a desired sensitivity parameter. In addition, the operating reaction force can be reproduced by the tactile presentation device 20 to evaluate the relationship between the sensitivity parameter expressing the operating sensation and the physical parameter included in the physical characteristics that operate the tactile presentation device 20, and the evaluation can be used as a guideline for designing a pressure-type operating tool.
[0033] Fig. 4 shows an equivalent model showing an example of the components of the tactile presentation unit 30. Fig. 5 shows an equivalent circuit and internal structure of an actuator 39 included in the tactile presentation unit 30. An arrow F shown in Fig. 5 indicates an operation reaction force (vector quantity). In Fig. 3, the operating principle of the tactile presentation unit 30 is explained by an equivalent circuit using an operator of Laplace transform.
[0034] As shown in Fig. 4, the tactile sense presentation unit 30 may have a movable part 21. In this case, the operation device 33 shown in Fig. 2 is integrated with the movable part 21 shown in Fig. 4. Alternatively, the operation device 33 may be provided outside the system of the tactile sense presentation device 20, and the movable part 21 may be moved by operating the operation device 33. The tactile sense presentation unit 30 has an actuator 39. As shown in Fig. 5, the actuator 39 is provided with a bobbin 24 and a coil 25 wound around the bobbin 24. The bobbin 24 and the coil 25 are also part of the movable part 21.
[0035] As shown in FIG. 4, the tactile sense presentation unit 30 may include a spring member 26. The spring member 26 has a predetermined spring constant and is formed of, for example, a coil spring. The spring member 26 is held in a compressed state within the tactile sense presentation unit 30, for example, and applies an operation reaction force in the opposite direction (upward in FIG. 4) to the direction in which the movable unit 21 is pressed in a normal use state. In FIG. 4, the spring constant of the spring member 26 is indicated by "Ks". As shown in FIG. 4, an operation reaction force based on a viscosity coefficient "C" caused by lubricating oil, sliding friction on the mechanism, and the like acts on the movable unit 21. Also, in FIG. 4, the stroke amount in the direction in which the movable unit 21 is pressed (downward in FIG. 4) is indicated by "x".
[0036] As shown in FIG. 5, the actuator 39 includes a cylindrical yoke 31 made of an iron-based magnetic material. The yoke 31 includes an outer yoke 31a and a center yoke 31b. A cylindrical magnet 32 is fixed inside the outer yoke 31a. A cylindrical magnetic gap is formed between the center yoke 31b and the magnet 32, and a cylindrical bobbin 24 and a coil 25 are inserted into the magnetic gap. As shown in FIG. 5, the amount of current flowing through the coil 25 is "I", the magnetic flux density of the magnetic field generated by the magnet 32 and crossing the coil 25 is "B", the inductance of the coil 25 is "L", and the electrical resistance including the coil 25 is "R". The number of turns of the coil 25 is "N". The operation reaction force "F" acting on the movable part 21 from the actuator 39 is controlled by a tactile presentation signal provided from the main control device 10 to the tactile presentation device 20.
[0037] In this example, a position sensor 27 provided in the tactile presentation device 20 shown in Fig. 2 detects a movement amount (hereinafter referred to as a "stroke amount") "x" of the movable part 21 in the pressing operation direction. In this example, an acceleration sensor 28 provided in the tactile presentation device 20 shown in Fig. 2 detects the acceleration of the movable part 21. In this example, an operation range variable unit 29 provided in the tactile presentation device 20 shown in Fig. 2 can change the total length of the stroke amount of the movable part 21 in the pressing operation direction.
[0038] The basic operation of the tactile presentation device 20 will be described with reference to Fig. 3 to Fig. 5. The tactile presentation device 20 can present a tactile sensation to the movable part 21 via the operation device 33 by controlling the current "I" given to the coil 25 of the tactile presentation section 30. The tactile sensation presented here is a change in the operation reaction force "F" applied to a body part such as a user's finger pressing the movable part 21 in the pressing operation direction. This operation reaction force "F" is a resistance force that reproduces the operation reaction force of a pressure-type operation tool that generates an operation reaction force using a dish-shaped leaf spring or a dome-shaped leaf spring.
[0039] 4 shows a model of the tactile presentation unit 30. The following equation 1 expresses the operation of the tactile presentation device 20 as an equation of "force".
[0040]
number
[0041] The equation 2 obtained by manipulating the equation 1 is shown below.
[0042]
number
[0043] In the equivalent circuit of the tactile presentation unit 30 shown in Fig. 5, the voltage acting on the coil 25 is indicated by "V", and the back electromotive force acting on the coil is indicated by "e". The following equation 3 shows the differential equation of "Ve" and an equation expressing this differential equation using the Laplace transform variable "s".
[0044]
number
[0045]
number
[0046] [Conversion model 15 generation process] Fig. 6 shows an example of a generation process (a conversion model generation method) of the conversion model 15 stored in the haptic control system 1 of Fig. 2. The conversion model generation method is executed by a conversion model generation system including at least an input unit, a storage unit, and a processor. "ST" in Fig. 6 indicates a processing step.
[0047] In STa, the conversion model generation system accepts input of sensibility parameters from multiple users for each of one or more types of tactile presentation. Here, "one or more types of tactile presentation" is not limited to tactile presentation when the user operates an operating tool, but also includes tactile presentation given to the user when the user does not perform any operation. For example, one or more types of tactile presentation may be presented as tactile presentation according to content such as a game or video through a suit or gloves, and input of sensibility parameters based on the user's feeling for each may be accepted. This step is an example of a storage step in the conversion model generation method described for the sensory control system 100 in FIG. 1.
[0048] In STb of FIG. 6, the conversion model generation system extracts physical parameters correlated with the sensory parameters from among the physical characteristics related to various tactile presentations. This step is an example of an extraction step in the conversion model generation method described for the sensory control system 100 in FIG. 1. In STc, the conversion model generation system generates a conversion model 15. This step is an example of a generation step in the conversion model generation method described for the sensory control system 100 in FIG. 1. The generation of the conversion model 15 can be performed manually, by multiple regression analysis, machine learning, or various other analysis methods. The conversion model 15 has various variations, such as a model capable of converting one type of sensory parameter into one type of physical parameter, a model capable of converting one type of sensory parameter into multiple types of physical parameters, a model capable of converting multiple types of sensory parameters into one type of physical parameter, and a model capable of converting multiple types of sensory parameters into multiple types of physical parameters. A model capable of converting multiple types of sensory parameters into multiple types of physical parameters may be generated by deriving information on a complex correlation from information on the correlation between one type of sensory parameter and one type of physical parameter using machine learning or the like. The data structure of the conversion model 15 may be a correspondence table between sensory parameters and tactile parameters, or may be stored so as to be calculable by a function.
[0049] Here, an example of a method for generating a conversion model 15 capable of converting a plurality of types of sensory parameters into a plurality of types of physical parameters will be described. In this example, first, in an extraction step, the conversion model generation system extracts information about the degree of correlation between each of the plurality of types of physical parameters and the sensory parameters for the plurality of types of sensory parameters. In detail, the conversion model generation system extracts information about the above-mentioned plurality of correlation degrees by multiple regression analysis using each of the plurality of types of sensory parameters as a response variable and the plurality of types of physical parameters as explanatory variables. Here, examples of the information about the correlation degrees include the coefficient of determination in the multiple regression analysis, a constant term, or values derived from these.
[0050] Next, in a generation step, the conversion model generation system generates a first relational expression that describes each of the plurality of types of sensory parameters by using the plurality of types of physical parameters and information on the plurality of correlation degrees (first generation step). 2、 A n (n is a natural number), and multiple types of physical parameters are P1, P2, ... P n Among the constant terms and coefficients of determination in the multiple regression analysis, the sensitivity parameter A m (m is a natural number less than or equal to n) m1 , B m2 , B mn Then, the first relational expression can be expressed by the following equation 5.
[0051]
number
[0052] When Expression 5 is expressed as a matrix equation with one side (here, the left side) being a column vector indicating a plurality of types of sensory parameters, and the other side (here, the right side) being the product of a coefficient matrix indicating information on a plurality of correlation degrees and a column vector indicating a plurality of types of physical parameters, the first relational expression is expressed as shown in Fig. 22. Here, the coefficient matrix is a square matrix with n rows and n columns.
[0053] After a first generation step included in the generation step, the conversion model generation system generates a second relational expression that describes each of the multiple types of physical parameters based on the first relational expression, using multiple types of sensory parameters and information on multiple correlation degrees (second generation step). Specifically, the conversion model generation system generates the second relational expression by multiplying both sides of the first relational expression shown in Fig. 22 by the inverse matrix of the coefficient matrix from the left. As shown in Fig. 23, the second relational expression can be expressed with one side (here, the left side) being a column vector indicating the multiple types of physical parameters, and the other side (here, the right side) being the product of the inverse matrix of the coefficient matrix and the column vector indicating the multiple types of sensory parameters.
[0054] After the second generation step included in the generation step, the conversion model generation system generates a conversion model 15 capable of converting a plurality of types of sensory parameters into a plurality of types of physical parameters correlated with the plurality of types of sensory parameters based on the second relational expressions (third generation step). In this way, the conversion model generation system can generate a conversion model 15 capable of converting a plurality of types of sensory parameters into a plurality of types of physical parameters.
[0055] In the above example, the coefficient matrix is described as a square matrix, but the coefficient matrix does not necessarily have to be a square matrix. For example, by using a pseudo-inverse matrix as the inverse matrix, it is possible to generate the conversion model 15 capable of converting from a plurality of types of sensory parameters to a plurality of types of physical parameters in the same manner even when the coefficient matrix is not a square matrix.
[0056] The sensation control method by the sensation control system 100 shown in Fig. 1 can be executed as follows when using the conversion model 15 obtained in this example. First, in a receiving step, the sensation control system 100 receives input of multiple types of sensory parameters from a user or the like via the input unit 4. Then, in a converting step, the processor 101 converts the acquired multiple types of sensory parameters into multiple types of physical parameters correlated with the multiple types of sensory parameters based on the conversion model 15. Note that the outputting step and the sensation presenting step are similar to those described above, and therefore description thereof will be omitted.
[0057] [Specific examples of tactile presentation] In the following, an example of tactile presentation imitating the operation feel of a predetermined operation tool by the tactile presentation device 20 shown in FIG. 2 will be described. The sensibility parameters of the conversion model 15 in this example are the expression levels of adjectives expressing the operation feel of operating a pressure-type operation tool as a predetermined operation tool. The physical parameters of the conversion model 15 in this example are included in the physical characteristics that realize the presentation of sensation when a pressure-type operation tool as a predetermined operation tool is operated. When the tactile control system 1 receives an input of a specific sensibility parameter via the input unit 4, it converts the received specific sensibility parameter into a physical parameter using the conversion model 15. The sensibility parameters assuming a pressure-type operation tool are the degree of sensory expression by adjectives, onomatopoeias, etc. that express the operation feel when a person presses the pressure-type operation tool. The physical characteristics realized by the physical parameters are, for example, a displacement accompanying an operation (for example, a stroke amount), an operation reaction force (load), the speed, acceleration, and jerk of the movable part 21, the elasticity characteristics of a body part such as the operator's finger, or an amount derived from these physical characteristics. A physical parameter is defined herein to include one or more variations of a physical property.
[0058] Fig. 7 is a flowchart for explaining a specific example of a conversion model generating method and a tactile presentation method. In the flowchart shown in Fig. 7, processing steps are indicated by "ST", but ST1, ST2, etc. include artificial processing, and ST3, ST4, etc. include processing executed by the processor 14 of the main control device 10 shown in Fig. 2.
[0059] In ST1 in FIG. 7, a plurality of operating tools with the same function but different operating feel are prepared. In ST2, a sensory test is performed by a plurality of users, and the operating feel of the prepared plurality of operating tools is classified by the expressiveness of adjectives as sensibility parameters. In ST3, the processor 14 of the haptic control system 1 associates the expressiveness of adjectives as sensibility parameters with physical parameters included in the physical characteristics that realize the sensory presentation when the operating tool is operated, based on a correlation coefficient or the like. The sensibility parameters and physical parameters each include at least one variable. The associated sensibility parameters and physical parameters are stored as the sensibility database 16 shown in FIG. 1. In ST4, the processor 14 uses the conversion model 15 to convert the physical parameters into physical parameters that are correlated with the expressiveness of adjectives as sensibility parameters that are newly accepted as input. The calculation function unit 12 generates a tactile presentation signal based on the physical parameters, and the calculation function unit 13 outputs the tactile presentation signal. The tactile presentation device 20 is operated by this tactile presentation signal, and a tactile sensation is presented. By controlling at least one of the coefficients "Kv", "Ks" and "C" shown in FIG. 4 using a tactile presentation signal based on the physical parameters, a tactile sensation corresponding to the sensory parameters is presented via the tactile presentation device 20.
[0060] In ST1 of FIG. 7, for example, a plurality of pressing type operating tools such as Tact Switches (registered trademark), which are actual products having a plate-shaped leaf spring or a dome-shaped leaf spring, are prepared as operating tools. FIG. 11 shows a schematic diagram of a change in the operation reaction force when the pressing type operating tool is pressed. FIG. 11 shows physical characteristics that realize the presentation of sensation when the pressing type operating tool is operated as an operating tool, in a coordinate plane with the horizontal axis representing the displacement caused by the operation and the vertical axis representing the operation reaction force (load) acting on a body part such as the finger of the operating user. In this specification, the "displacement caused by the operation of the operating tool" includes the operation amount of the operating tool, the operation time of the operating tool, or a combination of the operation amount and the operation time. That is, the physical characteristics that realize the presentation of sensation when the operating tool is operated can be expressed by any of the relationship between the operation amount of the operating tool and the operation reaction force, the relationship between the operation time of the operating tool and the operation reaction force, and the relationship between the combination of the operation amount of the operating tool and the operation time and the operation reaction force. In addition, the "displacement associated with the operation of the operating tool" may also include a displacement caused by the elasticity of the body part, such as the finger of the operator who operates the operating tool. In FIG. 11, the "displacement associated with the operation of the operating tool" is the operation amount of the pressing type operating tool as the operating tool, and is hereinafter referred to as "stroke amount "x"" as appropriate. In addition, the operation amount of the operating tool is an amount in one-dimensional space, two-dimensional space, or three-dimensional space. In FIG. 11, the operation amount of the pressing type operating tool as the operating tool is an amount in one-dimensional space along the pressing operation direction. Note that the operating tool may have a movable part that moves with the operation of the operating tool. The pressing type operating tool as the operating tool has a knob part that is pressed by the user or the like as a movable part. Therefore, the operation amount of the pressing type operating tool may be the movement amount of the movable part of the pressing type operating tool.
[0061] A curve showing the physical characteristics that realizes the presentation of sensation when the operating tool is operated in a coordinate plane with the horizontal axis being the operating amount (the amount of movement of the movable part) of the operating tool and the vertical axis being the operation reaction force as shown in FIG. 11 is called an FS curve (Force Stroke Curve), a feeling curve, an operating force curve, a load displacement curve, etc. Hereinafter, it will be referred to as a "load displacement curve" as appropriate. As shown in FIG. 11, when a user presses the pressing type operating tool, as the stroke amount "x" in the pressing operation direction increases, the operation reaction force gradually increases with the compression deformation of the plate-shaped leaf spring or the dome-shaped leaf spring. When the stroke amount "x" reaches the maximum position Pmax, the operation reaction force reaches a maximum value Tmax. When the pressing type operating tool is further pressed, the plate-shaped leaf spring or the dome-shaped leaf spring buckles and deforms, inverts, and the operation reaction force drops sharply. When the stroke amount "x" reaches the minimum position Pmin, the operation reaction force reaches a minimum value Tmin. When the user subsequently presses the pressure-type operating tool further, the plate-shaped leaf spring or dome-shaped leaf spring after buckling deformation is compressed, and the operation reaction force continues to increase until the plate-shaped leaf spring or dome-shaped leaf spring reaches the final stroke position where it contacts the fixed contact. In Fig. 11, the stroke at which the operation reaction force becomes equal to the maximum value Tmax while being pressed from the minimum position Pmin to the final position is defined as the load recovery position Pend.
[0062] When the user releases the pressure on the pressure type operating tool after pushing the tool all the way to the final stroke position where the contacts are in contact, the knob portion, which serves as the movable portion of the pressure type operating tool, returns to its initial position due to the elastic restoring force of the disc-shaped leaf spring or dome-shaped leaf spring. The load-displacement curve when the operating device 33 returns has hysteresis with respect to the load-displacement curve when the displacement associated with the pressing operation is increased as shown in Fig. 11. For ease of explanation, the operation will be explained below using only the load-displacement curve when the displacement associated with the pressing operation is increased.
[0063] Several pressure-type operating tools (23 in total) were classified into groups (A), (B), and (C) according to the total stroke amount when pressed to the final stroke. Classification (A) is a total stroke amount of 0.25 mm or more and 0.35 mm or less, classification (B) is a total stroke amount of 0.15 mm or more and less than 0.25 mm, and classification (C) is a total stroke amount of less than 0.15 mm.
[0064] A sensory test was conducted by 25 users on the above-mentioned multiple pressure-type operating tools. In the sensory test, the operating feel (tactile sensation) felt by the users was classified by the expression frequency using the SD method. In this sensory test, a predetermined sensitivity parameter A was used as the sensitivity parameter, and evaluation was performed on a seven-level scale of "1", "2", "3", "4", "5", "6", and "7". In this sensory test, the pressure-type operating tools in category (A) showed a wide variation in the expression frequency of sensitivity parameter A from around "1" to around "6". The pressure-type operating tools in category (B) showed a variation in the expression frequency of sensitivity parameter A in the intermediate region from around "2.5" to around "3.5". The pressure-type operating tools in category (C) showed a variation in the expression frequency of sensitivity parameter A from around "3.5" to around "6". Here, the affective parameter A may be, for example, a parameter relating to "sense of decisiveness," "comfort," "touch," etc., and specifically, if it is a parameter relating to "sense of decisiveness," the parameter may be such that the lower the expression degree, the "higher the sense of decisiveness" and the higher the expression degree, the "lower the sense of decisiveness."
[0065] As described above, the correlation between the sensory parameter A and the total stroke amount of the pressure-type operating tool as a physical parameter is not necessarily clear. Therefore, for the above-mentioned 23 pressure-type operating tools, we focused on physical characteristics other than the classified total stroke amount, and examined whether there is a correlation between the physical parameters extracted from the physical characteristics and the sensory parameter A. Figure 9 shows the load-displacement curves (i), (ii), and (iii) of three pressure-type operating tools with different total stroke amounts. In Figure 10(A), the area S4-1 of the depression in the load-displacement curve (i) and the area S4-2 of the depression in the load-displacement curve (ii) are taken out as variables of the physical amount of the operation, and in Figure 10(B), the areas S4-1 and S4-2 are translated so that their respective minimum values Tmin are the same, and compared.
[0066] As shown in FIG. 11, the area S4 is the area of the recessed portion from the maximum value Tmax through the minimum value Tmin to the position where the reaction force returns to the same reaction force as the maximum value Tmax in a coordinate plane with the amount of operation of the operating tool as the horizontal axis and the reaction force as the vertical axis. In other words, the area S4 is the area of the region in the coordinate plane defined by the load-displacement curve and a straight line passing through the maximum value Tmax of the load-displacement curve and parallel to the horizontal axis. The dimension expressing the area S4 is expressed as "distance (of the stroke amount) x load (of the reaction force)", and this dimension is equivalent to energy (amount of work). In other words, the area S4 corresponds to the energy (lost energy) that is reduced from the energy consumption that the user anticipated due to the reduction in the reaction force when the user operates the pressure-type operating tool. The presence of the area S4 causes the user to feel a sense of being drawn in the direction of the pressing operation.
[0067] In addition, the operation reaction force shown by the load-displacement curve (iii) in Figure 9 has a preload when the stroke is zero. This preload causes a so-called "play" in the operation. This "play" can also be used as one of the physical parameters.
[0068] FIG. 12 is a graph showing the relationship between the affective parameter A, which is the expression frequency according to the SD method, and the area S4, which is the physical parameter extracted from the physical characteristics that realize the sensation presentation when the operating tool is operated. The horizontal axis of FIG. 12 shows the affective parameter A, and the vertical axis shows the area S4, which is the physical parameter. As shown in FIG. 12, for a total of 23 press-type operating tools with a total stroke amount of 0.35-0.15 mm, it can be seen that there is a correlation between the size of the area S4 shown in FIG. 11 and the expression frequency of the affective parameter A. That is, it can be seen that there is a negative correlation in which the expression frequency of the affective parameter A decreases as the area S4 increases for the 23 press-type operating tools. Here, when the affective parameter and the physical parameter have a correlation, the absolute value of the correlation coefficient between the affective parameter and the physical parameter is preferably 0.5 or more, and more preferably 0.7 or more.
[0069] For normalization of the area S4, which is a physical quantity, it is preferable to limit the total stroke amount of the pressure type manipulation tool to a predetermined range. For example, the total stroke amount of the pressure type manipulation tool is preferably 0.05 mm or more and less than 0.5 mm, and more preferably 0.05 mm or more and less than 0.35 mm.
[0070] In this way, in the above example, the change in the operation reaction force with respect to the displacement accompanying the operation of the operating tool has at least a maximum part and a minimum part. The physical parameter includes a variable based on the area of a depression part from the maximum part through the minimum part to the coordinate where the operation reaction force transitions to the same magnitude as the maximum part in a coordinate plane having the displacement accompanying the operation and the operation reaction force as its axes. Here, the maximum part is a part including the maximum value Tmax in the load-displacement curve shown in FIG. 11, and the minimum part is a part including the minimum value Tmin in the load-displacement curve shown in FIG. 11.
[0071] 2 uses the conversion model 15 to convert the expression frequency of the sensory parameter A, which has been received as an input, into an area S4, which is a physical parameter correlated with the sensory parameter A, and the calculation function unit 12 calculates a load-displacement curve including the area S4 to set one tactile presentation signal including the load-displacement curve. Alternatively, the calculation function unit 12 calculates a plurality of load-displacement curves having the same area S4 but different strokes, loads, etc., and sets a plurality of tactile presentation signals including these load-displacement curves. Alternatively, the conversion model 15 may store a plurality of types of load-displacement curves associated with the size of the area S4 in advance in association with the expression frequency of the sensory parameter A, and the calculation function unit 12 may read out information on the load-displacement curve corresponding to the expression frequency of the sensory parameter A, which has been received as an input, from the storage unit 11, to generate a tactile presentation signal.
[0072] The input unit 4 of the input / output device 3 can accept input of not only integer expression frequencies such as "2", "3", and so on, or expression frequencies including decimals such as "2", "2.5", "3", "3.5", and so on, but also numerical ranges of expression frequencies such as "2-2.5", "2.5-3", "3-3.5", "3.5-4", and so on. The haptic control system 1 converts one or more load-displacement curves having an area S4, which is a physical parameter corresponding to the expression frequency of the sensitivity parameter inputted through the input unit 4, using the conversion model 15. The information of the converted one or more load-displacement curves is output to the input / output device 3, and the input / output device 3 displays one or more load-displacement curves on the display unit 5. The user checks one load-displacement curve displayed on the display unit 5, or selects one of the multiple load-displacement curves displayed. When this confirmation command or selection command is given from the input unit 4 to the processor 14, a tactile presentation signal based on the selected load-displacement curve is set in the calculation function unit 12, and a tactile presentation signal is output from the calculation function unit 13 to the tactile presentation device 20. As a result, when the operation device 33 of the tactile presentation device 20 is operated, an operation feel corresponding to the expression degree of the sensory parameter desired by the user can be presented.
[0073] Also, physical parameters such as "stroke amount" and "magnitude of operation reaction force" may be directly specified as input items from the input unit 4 together with the expression frequency of the sensibility parameter A. For example, when the haptic control system 1 receives an input of "stroke amount 0.25-0.35 mm" as a physical parameter together with the expression frequency of the sensibility parameter A via the input unit 4, the haptic control system 1 selects a load-displacement curve having an area S4 that matches the expression frequency of the adjective from among the multiple load-displacement curves included in the classification (A), and generates a haptic presentation signal based on this load-displacement curve. Alternatively, when the haptic control system 1 receives an input of a numerical item of "magnitude of operation reaction force" as a physical parameter together with the expression frequency of the sensibility parameter A via the input unit 4, the haptic presentation signal may be generated based on both the expression frequency of the sensibility parameter A and the "magnitude of operation reaction force" as a physical parameter.
[0074] In the above description, the total stroke amount is limited to a range of, for example, 0.35-0.15 mm, and the physical parameter, which is the size of the area S4, and the affective parameter, which is the expressiveness of the adjective, are associated with each other based on this range. However, the size of the area S may be associated with the expressiveness of the affective parameter based on a numerical range other than the total stroke amount. For example, a predetermined numerical range may be set for the maximum value Tmax, the minimum value Tmin, the maximum value minus the minimum value (Tmax-Tmin), the click stroke (Pend-Pmax), the push-in stroke (Pmax / (Pend-Pmax)), the click stroke ratio (Pmax / Pend), and the push-in stroke ratio (Pmax / (Pend-Pmax)) shown in FIG. 11, and this numerical range may be used as the standard. Alternatively, a predetermined numerical range may be set for the areas S1, S2, and S3 other than S4, or their ratios, and this numerical range may be used as the standard. Using these numerical ranges as standards, it is possible to associate the size of the area S4, which is a physical parameter, with the expression degree of the affective parameter.
[0075] For the 23 pressure type operating tools described above, a sensory test of the operating feel was conducted by 25 users for sensory parameters other than sensory parameter A. The results are shown in Fig. 13 to Fig. 15. Fig. 13 to Fig. 15 show the relationship between the expression frequency of sensory parameters other than sensory parameter A and physical parameters other than area S4 that change according to the expression frequency.
[0076] In Fig. 13, the horizontal axis indicates the expression level of the sensory parameter B. The vertical axis indicates a variable related to the stroke amount of the pressure-type operating tool as a physical parameter, for example, the "click stroke (Pend-Pmax)" shown in Fig. 11. Fig. 13 shows a negative correlation in which the expression level of the sensory parameter B decreases as the "click stroke (Pend-Pmax)" as a physical parameter increases. The sensory parameter B is, for example, a parameter related to "decisiveness," "comfort," "touch," etc., and specifically, when it is a parameter related to "comfort," it may be a parameter in which the lower the expression level, the more "comfortable" it is, and the higher the expression level, the more "uncomfortable" it is.
[0077] Thus, in the above example, the physical parameters include a variable related to the amount of displacement accompanying the operation, more specifically, a "click stroke (Pend-Pmax)" which is the amount of displacement from the maximum point to the coordinate at which the operation reaction force transitions from the maximum point through the minimum point to the same magnitude as the maximum point.
[0078] In Fig. 14, the horizontal axis indicates the expression frequency of the sensory parameter C. The vertical axis indicates a variable related to the load of the pressure-type operating tool as a physical parameter, for example, Pmax shown in Fig. 11. Fig. 14 shows a positive correlation in which the expression frequency of the sensory parameter C decreases as Pmax as a physical parameter decreases. The sensory parameter C is, for example, a parameter related to "sense of decision," "comfort," "touch," etc., and specifically, when it is a parameter related to "touch," it may indicate that the smaller the expression frequency, the softer the operation feel, and the higher the expression frequency, the harder the operation feel.
[0079] In Fig. 15, the horizontal axis indicates the expression degree of the sensory parameter D. The vertical axis is a variable related to the stroke amount of the pressure-type operating tool as a physical parameter, for example, the "push-in stroke ratio (Pmax) / (Pend-Pmax)" shown in Fig. 11. Fig. 15 shows a positive correlation in which the expression degree of the sensory parameter D increases as the "push-in stroke ratio (Pmax) / (Pend-Pmax)" as a physical parameter increases. The sensory parameter D is, for example, a parameter related to "decisiveness," "comfort," "touch," etc., and specifically, when it is a parameter related to "touch," it may indicate that the higher the expression degree, the sharper the touch is felt, and the lower the expression degree, the duller the touch is felt.
[0080] Thus, in the above example, the physical parameters include a variable related to the amount of displacement accompanying the operation. More specifically, the physical parameters include a variable related to the "push stroke ratio (Pmax) / (Pend-Pmax)" which is the ratio between the "click stroke (Pend-Pmax)" which is the amount of displacement from the maximum point to the coordinate where the operation reaction force transitions from the maximum point through the minimum point to the same magnitude as the maximum point, and the "Pmax" which is the amount of displacement from the start of the operation to the maximum point.
[0081] In the conversion model 15, a plurality of relationships may be stored as correlations between sensory parameters and physical parameters, including (1) the relationship between the expression frequency of sensory parameter A and the area S4, which is a physical parameter, shown in Fig. 12, (2) the relationship between the expression frequency of sensory parameter B and the click stroke, which is a physical parameter, shown in Fig. 13, (3) the relationship between the expression frequency of sensory parameter C and the maximum value minus the minimum value, which is a physical parameter, shown in Fig. 14, and (4) the relationship between the expression frequency of sensory parameter D and the push-in stroke ratio, which is a physical parameter, shown in Fig. 15. Any one or a plurality of relationships (1) to (4) are combined to calculate a physical parameter included in a physical quantity such as a load-displacement curve, and a tactile presentation signal is generated.
[0082] As described above, the acceleration of the movable part 21 of the tactile sense providing unit 30 shown in Fig. 4 can be detected by the acceleration sensor 28. In an actual pressure type operating tool, when a plate-shaped leaf spring or a dome-shaped leaf spring is pressed and buckles and deforms, and then inverts, vibrations are generated, and the vibrations are transmitted to a body part, such as a finger, performing the pressing operation, thereby providing an operating sensation.
[0083] 16(A), (B), and (C) are simulation data showing the acceleration of the movable part of the press-type operating tool when the press-type operating tool is pressed as the three operating tools. The relationship between the expression frequency of the sensitivity parameter E related to the operation feel of the pressing operation and the acceleration of the movable part of the operating tool as a physical parameter was investigated by a sensory test by a user using the three press-type operating tools. The peak-to-peak value of the acceleration when the dish-shaped leaf spring or dome-shaped leaf spring of the press-type operating tool buckles and deforms is the largest for the press-type operating tool in FIG. 16(A), and decreases in the order of (B) and (C). In addition, in the sensory test by a user, the expression frequency of the sensitivity parameter E for the operation of the press-type operating tool in FIG. 16(A) is the smallest, and the expression frequency increases in the order of (B) and (C). The sensory parameter E may be, for example, a parameter relating to "decisiveness," "comfort," "touch," etc., and specifically, if it is a parameter relating to "comfort," the parameter may be one in which the smaller the expression frequency, the more "comfortable" it is, and the larger the expression frequency, the more "uncomfortable" it is.
[0084] Based on the above-mentioned sensory test, the conversion model 15 may store a correlation between the expression frequency of the sensory parameter E and the acceleration of the movable part of the operating tool, which is a physical parameter. Using the conversion model 15, the haptic control system 1 converts the expression frequency of the sensory parameter E input by the input unit 4 into the acceleration of the movable part of the operating tool, which is a physical parameter, generates a haptic presentation signal based on this acceleration, and outputs the haptic presentation signal, thereby reproducing a desired operating feel by the haptic presentation device 20. For example, based on the physical parameters (movement amount, speed, acceleration, jerk, etc.) of the movable part of the operating tool, a haptic presentation signal for controlling the corresponding physical parameters of the movable part 21 of the haptic presentation device 20 may be generated.
[0085] [Example of operation of the tactile presentation device 20] FIG. 8 shows a flowchart of an example of the control operation of the tactile presentation device 20. The process shown in the flowchart is executed by the control operation of the processor 18 included in the tactile presentation device 20. In ST11 of FIG. 8, a tactile presentation signal is provided from the arithmetic function unit 13 to the processor 18 of the tactile presentation device 20, and in ST12, control based on the load-displacement curve selected based on the physical parameters is started. In ST13, when the operation device 33 is operated, a detection signal related to the movable part 21 is obtained from the position sensor 27 and the acceleration sensor 28. In the processor 18, the difference between the operation profile of the load-displacement curve set corresponding to the expression frequency, which is a sensitivity parameter, and the detection position of the movable part 21 is calculated. In ST14, the current I provided to the coil 25 of the tactile presentation unit 30 is optimized, and a tactile sensation is presented so as to reproduce the expression frequency of the sensitivity parameter desired by the user.
[0086] [Modification of the tactile presentation device 20] 17 to 19, a modified example of the tactile presentation device 20 included in the tactile control system 1 will be described. A tactile presentation device 40 shown in Fig. 19 reproduces the tactile sensation of a rotary operation tool. The rotary operation tool is, for example, a rotary switch.
[0087] 19 includes a processor 41, a tactile presentation unit 43, and a sensor 45. The tactile presentation device 40 presents a tactile sensation to a user who rotates an operation device 42. The operation device 42 may be mechanically incorporated in the tactile presentation device 40, or may be provided outside the tactile presentation device 40.
[0088] The tactile sense presentation unit 43 includes a resistance torque generator 43a and a rotation torque generator 43b. The resistance torque generator 43a variably applies a resistance torque in a direction opposite to the rotation direction to the rotation operation of the rotation operation part of the operation device 42. The resistance torque generator 43a has, for example, a yoke made of a magnetic material and a coil that applies a magnetic field to the yoke. A rotating plate that rotates in conjunction with the rotation operation of the rotation operation part of the operation device 42 is located in the magnetic gap of the yoke, and in the magnetic gap, a magnetorheological fluid is filled between the yoke and the rotating plate. It is also possible to use magnetic powder instead of the magnetorheological fluid. By controlling the current given to the coil, the coagulation state of the magnetorheological fluid changes, and the resistance torque is made variable. In addition to the above configuration, the resistance torque generator 43a includes, for example, a rotation motor, and the resistance torque can be made variable by the rotation motor. The rotation torque generator 43b variably applies a rotation torque in a rotation direction to the rotation operation of the rotation operation part of the operation device 42. The rotation torque generator 43b includes, for example, a rotation motor. The sensor 45 detects the rotation angle of the rotary operating part of the operating device 42 .
[0089] FIG. 18 shows a load-displacement curve for the operation reaction force of a rotary switch, which is a rotary operating tool. In a rotary switch, 360 degrees (one rotation) is divided into a plurality of divided angles, and the operation reaction force changes within each divided angle, and the same change in operation reaction force is repeated within each divided angle. FIG. 18 shows the change in operation reaction force within one divided angle. The horizontal axis of FIG. 18 shows the rotation angle of the rotary operating part as the operation amount of the rotary switch, the positive side of the vertical axis shows the magnitude of the resistance torque acting on the rotary operating part of the rotary switch in the opposite direction to the operation direction, and the negative side of the vertical axis shows the magnitude of the rotation torque acting on the rotary operating part in the same direction as the operation direction. The rotary switch has spring contacts within each divided angle. When a rotation operation is started within the divided angle, the spring contacts contract and the resistance torque acting on the rotary operating part increases. When the resistance torque exceeds the maximum value Rmax, the rotation operation unit is pushed in the direction of rotation operation by the restoring force of the spring contact, the resistance torque decreases, and a rotation torque in the direction of operation acts on the rotation operation unit from the spring contact. Therefore, when rotating the rotation operation unit, an operating sensation is felt with the finger for each division angle.
[0090] The conversion model 15 stores the correlation between the expression degree of the sensory parameter related to the rotation operation and the physical parameter. The haptic control system 1 receives the input of the expression degree of the sensory parameter via the input unit 4. Then, the processor 14 of the haptic control system 1 converts the received sensory parameter into a physical parameter using the conversion model 15, and generates a haptic presentation signal based on the physical parameter. The processor 14 outputs the generated haptic presentation signal to a processor 41 included in a haptic presentation device 40 shown in FIG. 19. When the rotation operation unit of the operation device 42 is rotated by a body part such as a user's finger, the haptic presentation device 40 detects the rotation angle of the rotation operation unit with a sensor 45 and feeds back the detection output to the processor 41. The processor 41 controls the haptic presentation unit 43, thereby controlling the resistance torque and rotation torque when the rotation operation unit of the operation device 42 is rotated, and a haptic sensation simulating a rotary switch that reproduces the expression degree of the sensory parameter can be presented.
[0091] FIG. 17 is an explanatory diagram for explaining the change in resistance torque as an example of a physical characteristic related to the expression degree of the sensibility parameter. FIG. 17(A) shows the operation reaction force when four rotary switches are rotated by a load-displacement curve, and FIG. 17(B) shows the change in curvature on each load-displacement curve shown in FIG. 17(A). In a sensory test by a plurality of users, when the rotary operation unit is rotated by a body part such as a finger, the resistance torque passes through a peak where the maximum value Rmax is reached, and it was concluded that the smaller the curvature of the change in the action line at this peak, the higher the expression degree of the sensibility parameter F. That is, it was confirmed that the expression degree of the sensibility parameter F is correlated with the curvature of the inflection part where the rotation load transitions from an increase to a decrease. Therefore, the conversion model 15 can present a rotation operation feel that realizes the expression degree of the sensibility parameter F as a tactile sensation by storing the correlation between the expression degree of the sensibility parameter F and a physical parameter having the curvature of the change in the resistance torque as a variable, in the tactile presentation device 40. The sensory parameter F may be, for example, a parameter relating to "sense of decision," "comfort," "touch," etc., and specifically, if it is a parameter relating to "touch," it may be a parameter such that the higher the expression frequency, the sharper the touch is felt, and the lower the expression frequency, the duller the touch is felt.
[0092] In this way, in the above example, the change in the manipulation reaction force in response to the displacement caused by the manipulation of the manipulation tool has at least a maximum part. Also, the physical parameters include a variable related to the curvature of the maximum part including the maximum value Rmax. Here, the maximum part is the part including the maximum value Rmax in the load-displacement curve shown in FIG. 18.
[0093] 18, it is possible to associate physical parameters including variables related to the rise of an increase in rotation load, such as the angle of the rise vector Tb of the resistance torque from the starting point of the division angle in the rotation angle as the operation amount of the rotary switch, and the ratio of areas Sa and Sb shown on the load-displacement curve at the rise part of the resistance torque, with the expression frequency of adjectives such as "stiff operation, resistance" as sensitivity parameters. Thus, in this example, the physical parameters include variables related to the rise of the operation reaction force from the start of the operation to the maximum part.
[0094] Here, the area Sa shown in Fig. 18 is an area defined by the load-displacement curve, the horizontal axis, and a straight line passing through the intersection of the load-displacement curve and the maximum value Rmax and parallel to the vertical axis. In other words, the area Sa is an integrated value of the load-displacement curve in the range of rotation angles from the starting point of the division angle in the rotation angle as the operation amount to the maximum value Rmax of the operation reaction force. The area Sb is an area defined by the load-displacement curve, the vertical axis, and a straight line passing through the maximum value Rmax and parallel to the horizontal axis. In other words, the area Sb is an area obtained by subtracting the area Sa from the area of a rectangle having one side equal to the rotation angle value of the intersection of the load-displacement curve and the maximum value Rmax and the other side equal to the maximum value Rmax. That is, if the load-displacement curve changes linearly on the coordinate plane from the start of the operation until the operation reaction force reaches the maximum value Rmax as shown by the dashed line in FIG. 18, the area Sa:area Sb=1:1, and the smaller the area Sb is relative to the area Sa, the more the load-displacement curve bulges toward the positive side of the vertical axis on the coordinate plane. In other words, the ratio of the area Sa to the area Sb indicates the degree of bulging of the load-displacement curve. In addition, the rising vector Tb of the resistance torque as a physical parameter shown in FIG. 18 includes a variable related to the derivative of the operation reaction force with respect to the operation amount. Similarly, the physical parameter may include a variable related to the derivative of the operation reaction force with respect to the operation time, or may include a variable related to the second derivative with respect to the transition of the operation reaction force.
[0095] Also, a variable related to the maximum value Dmax of the rotational torque (pull-in torque) acting in the same direction as the operation direction shown in Fig. 18, i.e., the magnitude of the pull-in amount at which the direction of the rotational load is reversed, can be associated with the frequency of expression of adjectives such as "fast rotation". Thus, in this example, the physical parameters include a variable related to the magnitude of the pull-in amount whose minimum part has a negative sign. Here, the minimum part is the part including the maximum value Dmax in the load-displacement curve shown in Fig. 18.
[0096] In the above example, FIG. 18 has been described as showing a load-displacement curve relating to the operation reaction force of a rotary switch, which is a rotary operating tool. However, FIG. 18 can also be used as a diagram showing a load-displacement curve relating to the operation reaction force of a slide switch that accepts a slide operation to a slide operation part. That is, the horizontal axis of FIG. 18 indicates the amount of slide operation of the slide operation part, and the positive side of the vertical axis indicates the operation reaction force against the slide operation of the slide operation part. The operation reaction force gradually increases with an increase in the amount of slide operation of the slide operation part, and then reaches a maximum value Rmax. When the maximum value Rmax is exceeded, the operation reaction force starts to decrease, and becomes a pulling force acting in the same direction as the operation direction, and reaches a minimum value (maximum value on the negative side of the vertical axis) Dmax. In this way, an operation feel can be presented with the operation of the slide switch. The correlation between the sensitivity parameters and the physical parameters described for the rotary switch is also applicable to the slide switch.
[0097] [First modified example of the sensory control method] A first modified example of the sensory control method executed by the sensory control system 100 of the present disclosure further includes an acquisition step of acquiring a sensory stimulation signal, and a designation step of designating a sensory parameter based on the acquired sensory stimulation signal. The reception step of accepting the input of the sensory parameter described above is not limited to an input from a user or the like, but is a step of accepting the sensory parameter designated in the designation step. This allows the sensory control system 100 according to the first modified example to designate a sensory parameter based on the acquired sensory stimulation signal, and to output a sensory presentation signal based on a physical parameter correlated with the designated sensory parameter.
[0098] Here, the sensory stimulation signal is an auditory stimulation signal based on an auditory stimulation element such as sound, a visual stimulation signal based on a visual stimulation element such as an image or video, a tactile stimulation signal based on a tactile stimulation element such as an operation reaction force or vibration, or a signal based on any combination of these. Furthermore, the sensory control system 100 according to the first modified example may generate and acquire the sensory stimulation signal by sensing an auditory stimulation element, a visual stimulation element, a tactile stimulation element, or a combination of these in the acquisition step.
[0099] Furthermore, in the designation step, the sensory control system 100 according to the first modification may convert and designate a physical parameter included in the physical characteristics of at least one of an auditory stimulation element, a visual stimulation element, and a tactile stimulation element (hereinafter collectively referred to as a sensory stimulation element) that is the basis of the sensory stimulation signal into a sensory parameter with which the physical parameter correlates. When converting the physical parameter into a sensory parameter with which the physical parameter correlates, the above-mentioned conversion model 15 may be used, or a conversion model different from the conversion model 15 may be used. A conversion model different from the conversion model 15 can be generated by AI analysis including machine learning and the like based on the corresponding information stored in the sensory database 16, similar to the conversion model 15. Furthermore, a physical parameter included in the physical characteristics of a sensory stimulation element such as a sound, an image, or a video can be extracted by AI analysis including machine learning and the like.
[0100] As described above, the sensory control system 100 according to the first modified example can acquire a sensory stimulation signal based on a sensory stimulation element such as sound, image, video, etc., extract physical parameters included in the physical characteristics of the sensory stimulation element by AI analysis, etc., specify a correlated sensory parameter, and output a sensory presentation signal based on the physical parameter correlated with the specified sensory parameter. Thus, for example, it is possible to output a tactile presentation signal based on a sensory parameter adjusted based on sound, image, video, etc.
[0101] [Second modified example of the sensory control method] The operation device 33 of the present disclosure may have an operation surface that accepts a slide operation. The slide operation is an operation in which a body part such as a user's finger is kept in contact with the operation surface of the operation device 33 and the contact position is moved. In this case, the tactile sense presentation unit 30 of the present disclosure generates an operation reaction force by vibrating the operation surface of the operation device 33. A method of vibrating the operation surface of the operation device 33 includes, for example, vibration of a weight by an actuator or the like. The sensation presentation step in the second modified example of the sensation control method of the present disclosure can be a process of presenting a tactile sensation by using such an operation device 33 and the tactile sense presentation unit 30 to generate an operation reaction force from the tactile sense presentation unit 30 in response to a slide operation of the operation device 33. In detail, the sensation presentation step includes, when a slide operation is performed on the operation surface of the operation device 33, detecting the slide operation by the operation device 33 and generating an operation reaction force from the tactile sense presentation unit 30 in response to the detected slide operation.
[0102] The physical parameters that can be converted based on the conversion model 15 stored in the storage unit 11 of the sensation control system 100 according to the second modification include parameters related to the change in the operation reaction force with respect to the displacement accompanying the slide operation of the operation device 33, and the change in the operation reaction force includes at least a maximum part or a minimum part. In the sensation presentation step, the tactile presentation unit 30 is controlled by a tactile presentation signal based on such physical parameters, so that the change in the operation reaction force with respect to the displacement accompanying the slide operation of the operation device 33 can be synthesized to include the above-mentioned maximum part or minimum part. Here, the tactile presentation unit 30 supplies a drive signal that generates vibrations in the operation surface of the operation device 33 based on the received tactile presentation signal, thereby driving the operation surface in a first direction at the rising edge of the drive signal and driving the operation surface in a second direction opposite to the first direction at the falling edge of the drive signal. Therefore, by making the time changes of the rising and falling edges of the drive signal different from each other and making the power in the first direction corresponding to the rising edge on a predetermined time average or in the second direction corresponding to the falling edge larger than the other, it is possible to artificially combine the above-mentioned maximum and minimum parts. Here, the drive signal that generates vibrations on the operation surface of the operating device 33 may be, for example, a signal that drives a weight by an actuator or the like, or the vibration of the weight may indirectly generate vibrations on the operation surface.
[0103] FIG. 24 is a diagram showing an example of the change over time in the strength of a drive signal supplied to a weight based on a tactile presentation signal. In the example shown in FIG. 24, when the change over time in the strength of the drive signal is positive, the weight is driven in a first direction, and when the change over time in the strength of the drive signal is negative, the weight is driven in a second direction. As shown in FIG. 24(a), when the change over time in the rising edge of the drive signal of the weight is greater than the change over time in the falling edge of the drive signal of the weight on average over a predetermined time, the force in the first direction corresponding to the rising edge of the drive signal is greater than the force in the second direction corresponding to the falling edge of the drive signal. On the other hand, as shown in FIG. 24(b), when the change over time in the falling edge of the drive signal of the weight is greater than the change over time in the falling edge of the drive signal of the weight on average over a predetermined time, the force in the first direction corresponding to the rising edge of the drive signal is greater than the force in the second direction corresponding to the falling edge of the drive signal. In this way, by controlling switching between a period in which the power in the first direction is increased as shown in FIG. 24(a) and a period in which the power in the second direction is increased as shown in FIG. 24(b), the above-mentioned maximum or minimum portions can be artificially combined.
[0104] The first direction and the second direction may be a direction intersecting the operation surface of the operation device 33, or may be a direction along the operation surface (parallel direction). For example, if the first direction and the second direction are set to be a direction intersecting the operation surface of the operation device 33, the resistance in the pressing direction against the operation surface for the body part of the user, such as the finger, performing the slide operation on the operation surface will change, and the friction force between the body part and the operation surface associated with the slide operation, that is, the operation reaction force, can be changed. Also, for example, if the first direction and the second direction are set to be a direction along the operation surface of the operation device 33, the resistance in the slide operation direction on the operation surface for the body part of the user, such as the finger, performing the slide operation on the operation surface will change, and the friction force between the body part and the operation surface associated with the slide operation, that is, the operation reaction force, can be changed.
[0105] The conversion model 15 stored in the storage unit 11 of the sensory control system 100 according to the second modification may be obtained by a conversion model generation method including the following storage step. That is, in the storage step of the conversion model generation method according to the second modification, the sensibility database 16 stores, for each of one or more types of operation tools, correspondence information in which physical characteristics that realize the sensation presentation when a specific operation tool is operated are associated with sensibility parameters that are input to reflect the operation of the operation tool. Here, the operation tool has an operation surface that accepts a slide operation. Also, the change in the operation reaction force with respect to the transition accompanying the slide operation of the operation tool includes at least a maximum part and a minimum part. Here, the operation reaction force is generated by vibration of the operation surface of the operation tool. The vibration of the operation surface of the operation tool may be indirect vibration caused by vibration of a weight caused by an actuator or the like, similar to the vibration of the operation surface of the above-mentioned operation device 33. The maximum or minimum part included in the change in the reaction force against the displacement caused by the sliding operation of the operating tool is artificially synthesized by making the time changes of the rising and falling edges of the drive signal that generates the vibration of the operating surface of the operating tool different from each other, and by making the power in the direction corresponding to the rising edge or the direction corresponding to the falling edge greater than the other on a predetermined time average. By using such an operating tool, the conversion model 15 of this example can be generated more easily.
[0106] [Modification of Sensitivity Database 16] As described above, the sensitivity database 16 of the present disclosure stores, for one or more types of sensory presentation, correspondence information in which physical characteristics related to a predetermined sensory presentation are associated with sensory parameters indicating the degree of sensory expression for the sensory presentation. Although tactile presentation has been mainly described as a sensory presentation, the "tactile sense" mainly referred to in this specification is tactile sense in a broad sense, and the broad sense of tactile sense is a concept including tactile sense in a narrow sense, pressure sense, force sense, and the like. In this specification, when "tactile sense" is simply described, it means tactile sense in a broad sense. Here, the narrow sense of tactile sense is, for example, a sense related to the texture of an object surface that a body part comes into contact with, and is highly correlated with sensory parameters related to sensory expression such as unevenness and roughness. The pressure sense is, for example, a sense related to resistance between a body part and an object, and is highly correlated with sensory parameters related to sensory expression such as hardness. The force sense is, for example, a sense related to an external force acting on a body part, and is, for example, a sense of being pulled or pushed. It is known that the receptors primarily responsible for the narrow sense of touch, pressure, and force are different, and that the response characteristics of each receptor also differ.
[0107] Moreover, the physical characteristics related to tactile presentation include static characteristics and dynamic characteristics. The static characteristics are physical characteristics obtained, for example, when an operating tool is operated at a constant operating speed with an instrument or the like having a high rigidity such that elasticity can be ignored (hereinafter, simply referred to as a "rigid body"). The dynamic characteristics are physical characteristics obtained, for example, when an operating tool made of a flexible material that imitates a body part such as a human finger is operated while changing the operating speed, and unlike the static characteristics, they are physical characteristics that also include physical parameters such as the elasticity of the body part, the operating speed, the operating acceleration, the operating jerk, and the friction force.
[0108] The correspondence information stored in the sensibility database 16 may be information related to at least one of information on touch in the narrow sense, pressure, and force sense included in touch in the broad sense, and information on static characteristics and dynamic characteristics included in physical characteristics. For example, the correspondence information stored in the sensibility database 16 may be information in which the weighting of the static characteristics and dynamic characteristics of touch in the narrow sense, pressure, and force sense changes depending on the operation stage of the operating tool. More specifically, for example, the weighting of the static characteristics may be set to be greater than the weighting of the dynamic characteristics at an operation stage immediately after the start of the operation, and the weighting of the dynamic characteristics may be set to be greater than the weighting of the static characteristics at an operation stage where the change in the operation reaction force to the displacement associated with the operation becomes large (for example, an operation stage corresponding to the maximum part in the load-displacement curve shown in FIG. 11 and FIG. 18 or the minimum part in the load-displacement curve shown in FIG. 11). This is because the influence of the operation speed, etc. may be small at the operation stage immediately after the operation is started, and in that case, the physical characteristics can be reproduced with high accuracy even if the physical characteristics are approximated with static characteristics, but the influence of the operation speed, etc. may be large at the operation stage where the change in the operation reaction force against the displacement caused by the operation becomes large, and in that case, the physical characteristics can be reproduced with higher accuracy by approximating with dynamic characteristics. In addition, the correspondence information stored in the sensitivity database 16 may be information including physical characteristics that reflect the difference in the response characteristics of the receptors mainly involved in the narrow sense of touch, pressure sense, and force sense. By generating the conversion model 15 based on such correspondence information, it becomes possible to present a tactile sensation that better reflects human sensibility.
[0109] (Tactile Control System 2) FIG. 20 shows the configuration of a haptic control system 2 as a second embodiment of the sensation control system 100 shown in FIG. 1, together with the signal flow.
[0110] The haptic control system 2 shown in FIG. 20 includes a terminal device 80 and a communication device 70, which are connected to each other via a network 9 so as to be able to communicate with each other. The terminal device 80 includes a main control device 6, an input / output device 3, and a haptic presentation device 20. The main control device 6 includes a processor 7 and a storage unit 8, and controls the operations of the input / output device 3 and the haptic presentation device 20. The haptic presentation device 20 includes a haptic presentation unit 30, an operation range variable unit 29, and sensors such as a position sensor 27 and an acceleration sensor 28. The communication device 70 is, for example, a server device, and includes a processor 14, a storage unit 11, a calculation function unit 12, and a calculation function unit 13. A conversion model 15 is stored in the storage unit 11.
[0111] Among the components of the tactile control system 2, the input / output device 3, the tactile presentation device 20, the processor 14, the storage unit 11, the arithmetic function unit 12, and the arithmetic function unit 13 are similar to the components of the tactile control system 1 shown in Fig. 2 and are designated by the same reference numerals, and therefore will not be described. Similarly to the tactile control system 1, the terminal device 80 may include an operation device 33, and the tactile presentation unit 30 may present a tactile sensation to a user who operates the operation device 33. Furthermore, the tactile control system 2 may include the tactile presentation unit 43 shown in Fig. 19 instead of the tactile presentation unit 30, and may include the operation device 42 shown in Fig. 19 instead of the operation device 33.
[0112] FIG. 21 is a sequence diagram showing the operation of the haptic control system 2. In FIG. 21, the processes executed by the terminal device 80 and the communication device 70 included in the haptic control system 2 are explained in steps (ST). First, in ST31, the terminal device 80 accepts input of the affective parameters. Specifically, the terminal device 80 accepts the affective parameters input by the user or the like via the input unit 4 of the input / output device 3. Next, in ST32, the terminal device 80 encodes the information on the affective parameters and transmits the encoded information on the affective parameters to the communication device 70 via the network 9. The terminal device 80 may include an encoder for encoding the information on the affective parameters. Furthermore, the terminal device 80 may encode the entire information on the affective parameters, or may encode only a part of the information on the affective parameters.
[0113] After ST32, in ST21, the communication device 70 decodes the information received from the terminal device 80 to obtain information on the sensory parameters. The communication device 70 may include a decoder for decoding the information on the sensory parameters. Next, in ST22, the communication device 70 converts the sensory parameters into physical parameters correlated with the sensory parameters using the conversion model 15. Next, in ST23, the communication device 70 encodes the converted physical parameters and transmits the encoded physical parameter information to the terminal device 80 via the network 9. The communication device 70 may include an encoder for encoding the physical parameter information. Also, the communication device 70 may encode the entire physical parameter information or only a part of it.
[0114] After ST23, in ST33, the terminal device 80 decodes the received information to obtain information on the physical parameters. The terminal device 80 may include a decoder for decoding the information on the physical parameters. Thereafter, in ST34, the terminal device 80 generates a tactile presentation signal based on the physical parameters and operates the tactile presentation device 20. Note that the encoding and decoding processes in ST32, ST21, ST23, and ST33 are not essential.
[0115] In this way, when a sensory parameter is input to the terminal device 80, the tactile control system 2 according to this embodiment can receive information on a physical parameter correlated with the sensory parameter from the communication device 70 via the network 9, and present a tactile sensation in the form of a tactile presentation signal based on the physical parameter. Thus, the tactile control system 2 makes it possible to present a tactile sensation that reflects human sensibility by communicating tactile information via the network 9. The tactile control system 2 is particularly useful in the field of the Tactile Internet.
[0116] Furthermore, when communicating all physical parameters included in the physical characteristics related to tactile presentation, problems such as communication delays are likely to occur due to an increase in the amount of data, but in the tactile control system 2 according to this embodiment, physical parameters correlated with sensory parameters are extracted and communicated, so the amount of data can be reduced. This can contribute to faster communication and reduced load on each processor, etc. This effect is similar to that of the tactile control system 1 according to the first embodiment, but is particularly useful in the tactile control system 2 according to this embodiment that uses tactile internet.
[0117] The haptic control system 2 according to the present embodiment may include a plurality of terminal devices 80. That is, the communication device 70 may be connected to each of the plurality of terminal devices 80 via the network 9. In this case, the communication device 70 may store identification information such as an address or ID that specifies each of the plurality of terminal devices 80, and a conversion model 15 associated with each identification information. This allows the conversion model 15 to be optimized and configured for each user who uses each terminal device 80.
[0118] Furthermore, the communication device 70 of the haptic control system 2 according to this embodiment may store a plurality of conversion models 15 according to, for example, applications (for games, for in-vehicle use, etc.), and different conversion models may be used according to the application, etc., required by the terminal device 80. This allows the conversion model 15 to be optimized and configured according to the application, etc., so that different physical parameters can be selected according to the application, etc., even if the physical parameters are converted from the same sensory parameters.
[0119] 20 shows an example in which the conversion model 15 is stored in the storage unit 11 of the communication device 70, the conversion model 15 may be stored in the storage unit 8 of the main control device 6 of the terminal device 80. In this case, for example, information on sensory parameters (including encoded information, etc.) may be distributed from the communication device 70, and the sensory parameters may be converted into physical parameters in the main control device 6 of the terminal device 80, thereby generating a tactile presentation signal based on the physical parameters correlated with the sensory parameters.
[0120] (Application examples of haptic control systems 1 and 2) The haptic control system 1 according to the first embodiment can be used for entertainment purposes such as games, videos, and music. When the haptic control system 1 is used for entertainment purposes, a haptic sensation from the haptic presentation device 20 may be presented to a user through an operation unit such as a button, a joystick, or a trigger switch included in an operation device 33 such as a game controller. In addition, a haptic sensation from the haptic presentation device 20 may be presented to a part other than the operation unit of the operation device 33, for example, the whole or a part of a body part such as the hand of the user holding the operation device 33. The game controller may be, for example, a steering controller that imitates the steering wheel of an automobile.
[0121] Examples of the timing for presenting a tactile sensation to the user through the operation device 33 include the timing for detecting an operation on an operation section included in the operation device 33, the timing for detecting an operation such as movement, rotation, acceleration or deceleration on the whole or part of the operation device 33, and the timing for presenting a tactile sensation according to content. The timing for presenting a tactile sensation according to content is a tactile sensation presentation timing that is preset to enhance the sense of realism within each piece of content such as a game, video, or music, and may be a timing when no operation from the user is detected.
[0122] When the haptic control system 1 is used for entertainment purposes, the haptic presentation from the haptic presentation device 20 is not limited to being performed through the above-mentioned operation device 33. The haptic presentation from the haptic presentation device 20 may be performed, for example, through a seat on which the user sits, a suit worn by the user, a headset used for virtual reality (VR) or augmented reality (AR), a glove or other wearable device that the user wears on a body part such as the hand, or other wearable device. For example, the feeling of operating a virtual switch in the VR or AR space may be presented through a wearable device.
[0123] The haptic control system 1 according to the first embodiment can be used, for example, in an in-vehicle application. When used in an in-vehicle application, the haptic presentation device 20 may present a haptic sensation to an occupant through a device used for driving operations such as a steering wheel, pedals, or shifter, an operation device 33 such as an infotainment system, an air conditioning unit, or a decorative panel, or a seat, etc. Here, the decorative panel is a device that is provided at any location inside the vehicle, such as a door trim, pillar, glove box, center console, dashboard, or overhead console, that constitutes the interior of the vehicle, and that can display information by a touch operation or a proximity operation.
[0124] When the haptic control system 1 is used for in-vehicle applications, the main purpose of presenting a haptic sensation is to notify the occupants that an input operation has been performed on the operating device 33, etc., and to warn the occupants of lane departure, approaching another vehicle, etc. In other words, the purpose may be different from the above-mentioned entertainment application, which is mainly intended to present a sense of realism. Therefore, the haptic control system 1 may store a conversion model 15 that can convert physical parameters converted from the same sensory parameters into different physical parameters depending on the application.
[0125] When the tactile control system 1 is used for in-vehicle applications, tactile sensations may be presented to the occupants when an input operation is detected on the operating device 33, or when a danger such as lane departure or approaching another vehicle is detected.
[0126] The haptic control system 2 according to the second embodiment can be used for the same purposes as the haptic control system 1 according to the first embodiment. That is, the haptic control system 2 can be used for entertainment purposes such as games, videos, and music, and for in-vehicle purposes.
[0127] When the haptic control system 2 according to the second embodiment is used for entertainment purposes, in addition to the same usage as the haptic control system 1 of FIG. 1, a haptic presentation signal may be transmitted, received, or distributed in association with live distribution (including broadcasting) of content via a network 9, data updates of content, and interactions and battles between users. For example, when a communication device 70 communicates with a plurality of terminal devices 80, common affective parameters may be set for each terminal device 80, or individual affective parameters may be set for each terminal device 80, or some affective parameters may be set in common while other affective parameters may be set individually for each terminal device 80. For example, when users of a plurality of terminal devices 80 are made to work in a common VR or AR environment, the affective parameters indicating the magnitude of the sensation may be common to each terminal device 80, and the affective parameters indicating the sharpness of the sensation may be adjusted according to the preferences of the users of the terminal devices 80, thereby adjusting the environment individually.
[0128] When the haptic control system 2 is used for in-vehicle applications, in addition to being used in the same manner as the haptic control system 1, it may receive a haptic presentation signal for warnings and the like based on communication between vehicles via the network 9, communication with road objects such as traffic signs, and traffic information distributed from a server. Communication between vehicles and communication with road objects can also be realized by the haptic control system 1 according to the first embodiment as long as direct communication is possible without going through the network 9.
[0129] The haptic control system 2 according to the second embodiment can be used for, for example, medical applications and industrial applications. An example of a medical application is the transmission of haptic information associated with remote medical care. An example of an industrial application is the transmission of haptic information associated with remote operation of an industrial robot. If the haptic information transmitted in these applications can be customized based on sensitivity values, it would be possible to present a more realistic haptic sensation to the user and allow the user to perform operations comfortably.
[0130] The haptic control system 2 according to the second embodiment can be used, for example, for Internet shopping. For example, the tactile sensations of a product, how it feels to wear, and how comfortable it is to write with a writing implement can be presented to the user by haptic transmission. In addition, the tactile sensations of a product and how it feels to wear can be customized based on the sensitivity value, and a product that is closer to the tactile sensation and how it feels to wear that the user desires can be suggested to the user.
[0131] The haptic control system 2 according to the second embodiment can be used for communication between users in remote locations. It can present the feeling of shaking hands between users in remote locations, the feeling of touching each other, and the like. It can also present the feeling of touching an animal such as a pet. In these applications, it is particularly useful to use or combine a temperature sensation presentation as the haptic presentation unit 30, since it can convey warmth. [Aspect 2]
[0132] [Background technology] Conventionally, there are known operating tools that provide a sense of sensation by providing some kind of stimulus to a person. Here, the sense of sensation includes tactile sensation, auditory sensation by sound, and visual sensation by image display. The sense of sensation is adjusted by adjusting the signals that drive various operating tools.
[0133] There is known a technology for producing a product according to a user's preferences (see, for example, Patent Document 2). Patent Document 2 discloses a technology in which a user selects a reference model, and in a subsequent process, color, size, material, position, and the like are added or changed based on the user's selection.
[0134] [Summary of the Invention] [Problem to be solved by the invention] However, conventional techniques have a problem in that they cannot adjust sensory presentation by sensory input. In other words, although the sensations preferred by users vary from user to user, users may express their preferences sensorily. However, conventionally, this sensory expression has not been used to change sensory presentation.
[0135] In view of the above-mentioned problems, an object of the present invention is to provide a tactile control device that can adjust the operational feel through sensory input.
[0136] [Explanation of mode 2] In the first aspect, a sensory control method for converting sensory parameters into physical parameters using the conversion model 15 has been described. However, even if a manufacturer prototypes an operating tool in which physical parameters converted from sensory parameters are applied to tactile presentation, it is often necessary to carry out several trials and errors in order to obtain an operating feel that is preferred by a user. Since prototyping an operating tool requires many steps, it may take a long time to complete an operating tool that has an operating feel that is preferred by a user.
[0137] In this embodiment, a haptic control device capable of reproducing the operation feel preferred by the user in real time and a haptic control method performed by the haptic control device will be described.
[0138] [Example of a haptic control device] Fig. 25 is a perspective view of the tactile control device 50. Fig. 25 shows a standalone tactile control device 50. As shown in Fig. 25, the tactile control device 50 has three reference operating tools 51a to 51c (multiple reference operating tools), a reproduction operating tool 52, a touch panel 53, and a display 260. Note that, hereinafter, any reference operating tool among the reference operating tools 51a to 51c will be referred to as a "reference operating tool 51." There may be two or more reference operating tools 51.
[0139] The display 260 displays how to use the haptic control device 50, an operation menu, etc. The touch panel 53 displays sensory parameters (e.g., adjectives) for which expression levels are input, allowing the user to input the expression level for each sensory parameter. When reproducing the user's preferred operation feel, the haptic control device 50 accepts input of the expression level for each sensory parameter multiple times, and displays on the touch panel 53 the sensory parameters for which the expression level can be input each time.
[0140] The three standard operating tools 51a to 51c are operating tools with different operating feel prepared as standards, that is, the three standard operating tools 51a to 51c each have a different load-displacement curve.
[0141] By inputting the expression level that the user prefers, the operation feel of the reference operation tool 51 selected by the haptic control device 50 from the three reference operation tools 51a to 51c is reproduced in the reproduction operation tool 52. In other words, the haptic control device 50 copies the physical parameters of one of the three reference operation tools 51a to 51c to the reproduction operation tool 52. The user can adjust the operation feel to his or her preference by operating the reproduction operation tool 52 and inputting the expression level.
[0142] Therefore, the user can adjust the operation feel in real time by inputting the expression degree while operating the reproduction operation tool 52 and checking the operation feel, and repeating the process of adjusting the operation feel of the reproduction operation tool 52. In addition, the user can compare the operation feel of the adjusted reproduction operation tool 52 with the operation feels of the reference operation tools 51a to 51c, making it easier for the user to adjust the expression degree to his / her preference.
[0143] The shape and appearance of FIG. 25 are merely an example, and a general-purpose system configuration in which the reference operating tool 51 and the reproduction operating tool 52 are connected to a PC or a tablet terminal via a USB cable or the like may also be used.
[0144] FIG. 26 illustrates a client-server type haptic control system 2. In the haptic control system 2 of FIG. 26, a terminal device 80 and a server 200 can communicate with each other via a network. The terminal device 80 may execute a web browser, for example, or may execute a dedicated application. The terminal device 80 performs a screen display required for inputting the expression degree for each affective parameter, and accepts the input of the expression degree from the user. The terminal device 80 transmits the expression degree to the server 200, and the server 200 transmits the selection result of the reference operating tools 51a to 51c, the physical parameters corresponding to the reference operating tools 51a to 51c, and the adjusted physical parameters to the terminal device 80.
[0145] In this way, even in a client-server type system, the user can adjust the operational feel in real time, just like the haptic control device 50.
[0146] <First form of haptic control device> First, an outline of the operation of the haptic control device 50 will be described with reference to Figures 27 and 28. Figures 27 and 28 show an outline of the operation in which the user adjusts the operational feel using the haptic control device 50.
[0147] (1) First, the user inputs an expression frequency (an example of a first expression frequency) that indicates the user's preference for a plurality of affective parameters (e.g., adjectives) (FIG. 27(a)). A first input screen 281 of FIG. 27(a) is displayed on the touch panel 53, and the first input screen 281 has an affective parameter presentation field 282 and a reference operating tool field 112. In the affective parameter presentation field 282, the user can input an expression frequency for each affective parameter (an example of a first affective parameter) using a slide bar (an example of an input means). In the reference operating tool field 112, the probability of the reference operating tools 51a to 51c being selected for the input expression frequency is displayed.
[0148] (2) The haptic control device 50 selects the reference operation tool 51a-51c that is closest to the user's preference (the expression level of each inputted affective parameter) based on the correspondence between the expression level of each affective parameter and the reference operation tool 51a-51c that has been learned in advance (FIG. 27(b)). This process is called STEP 1.
[0149] (3) The tactile control device 50 reproduces the operation feel of the reference operation tools 51a to 51c with the reproduction operation tool 52 (FIG. 27(c)). In FIG. 27, the number of the reference operation tools 51a to 51c is three, but this is merely an example. The user operates the reproduction operation tool 52 to check whether the operation feel is as desired.
[0150] (4) If the operating feel is different from the user's preference, the user again inputs the expression levels (an example of the second expression levels) representing the user's preference for the plurality of affective parameters (FIG. 28(a)). The second input screen 120 of FIG. 28(a) is displayed on the touch panel 53, and the second input screen 120 has an affective parameter presentation field 121. In the affective parameter presentation field 121, the user can input the expression level for each affective parameter (an example of the second affective parameter) using a slide bar. The number of affective parameters in the affective parameter presentation field 121 may be smaller than the number of affective parameters in the affective parameter presentation field 282. This is because the user's preferred reference operating tool 51 has already been selected in the affective parameter presentation field 282. Furthermore, the smaller number of affective parameters in the affective parameter presentation field 121 reduces the user's workload.
[0151] In the initial state of the affective parameter presentation field 121, the expression frequency of the slide bar indicates the median value. Even if the user sets the expression frequency of the same affective parameter to the minimum or maximum value in the affective parameter presentation field 282, in the initial state of the affective parameter presentation field 121, the expression frequency of the slide bar is the median value. This makes it easy for the user to adjust the expression frequency in the affective parameter presentation field 121 to a range including the expression frequency input in the affective parameter presentation field 282. In addition, the expression frequency in the initial state of the affective parameter presentation field 121 is the expression frequency corresponding to the physical parameter set in the reference operating tool 51. The user can adjust the expression frequency from this initial state to a value before or after the initial value.
[0152] (5) The haptic control device 50 converts the expression frequency of each sensory parameter input by the user into a physical parameter based on the correspondence between the expression frequency of each sensory parameter and a physical parameter (e.g., a regression model) that has been learned in advance, and reflects the physical parameter in the reproduction operation tool 52 (FIG. 28(b)). This process is called STEP 2. (6) The user operates the reproduction operation tool 52 to check whether the operation feel is to his / her liking (FIG. 28(c)).
[0153] Thereafter, the user repeats steps (4) to (6), whereby the haptic control device 50 can determine the physical parameters of the user's preferred operation feel.
[0154] [Functions of the haptic control device] Fig. 29 is a functional block diagram explaining the functions of the haptic control device 50. As shown in Fig. 29, the haptic control device 50 has a display control unit 61, a first input receiving unit 62, a second input receiving unit 63, a classification unit 64, a first conversion model 65a, a second conversion model 65b, a third conversion model 65c, and a physical parameter setting unit 66. Each of these functions of the haptic control device 50 is realized by a CPU or processor included as an information processing device executing a program expanded in a RAM. Alternatively, each function may be realized by a hardware circuit.
[0155] The display control unit 61 displays preset affective parameters and five or seven levels of expression levels set for the affective parameters on the touch panel 53 in a selectable manner (displaying the first input screen 281 and the second input screen 120). The expression level can be adjusted in any step or continuously. The user may select the expression level by tapping the touch panel 53 or by sliding a slide bar. The user may select the expression level by voice input or button input. The display control unit 61 displays different affective parameters in the above STEP 1 and STEP 2. The number of affective parameters in STEP 1 may be greater than the number of affective parameters in STEP 2.
[0156] The first input receiving unit 62 receives an input of the expression frequency of each affective parameter in response to a user operation in STEP 1. The second input receiving unit 63 receives an input of the expression frequency of each affective parameter in response to a user operation in STEP 2.
[0157] The classification unit 64 is an identification model that learns the correspondence between the expression frequency of the affective parameter received by the first input receiving unit 62 and three conversion models. There are many types of classification learning methods, such as deep learning, decision tree, and support vector machine, but in this embodiment, any learning method may be used for learning. The classification unit 64 outputs identification information of the first conversion model 65a to the third conversion model 65c for the expression frequency of the affective parameter received by the first input receiving unit 62 (specifying the conversion model 15 that is closest to the user's preference from the multiple conversion models 15).
[0158] As described in the first embodiment, the first to third conversion models 65a to 65c are conversion models capable of converting a sensory parameter into a physical parameter correlated with the sensory parameter. The first to third conversion models 65a to 65c correspond to the three reference operating tools 51a to 51c, and can convert the expression frequency of the sensory parameter into a physical parameter in each of the reference operating tools 51a to 51c. The physical parameters include, for example, the stroke amount of the operating tool, the operation reaction force (load), the speed, acceleration, and jerk of the movable part, and the elastic properties of the body parts such as the operator's fingers. The first to third conversion models 65a to 65c are generated by multiple regression or the like based on the expression frequency of a sensory test for physical parameters with different load-displacement curves in order to reproduce different operating sensations.
[0159] Then, the first conversion model 65a to the third conversion model 65c convert the expression frequencies of the affective parameters accepted by the second input accepting unit 63 into different physical parameters. In this way, the reference conversion model selected in STEP 1 can convert the expression frequencies close to the user's preferences input in STEP 2 into physical parameters.
[0160] The physical parameter setting unit 66 sets the physical parameters output by any one of the first conversion model 65a to the third conversion model 65c in the reproduction operation tool 52. Therefore, the haptic control device 50 can reproduce in real time the operation feel desired by the user.
[0161] [Generation of classification units, learning the correspondence between the expression frequency of sensitivity parameters and physical parameters] Next, the generation of the classification unit 64 will be described with reference to Fig. 30 etc. Fig. 30 is a flow chart showing the flow of learning in the generation of the classification unit 64. Note that although various learning is performed by the haptic control device 50, the learning can be performed by any information processing device.
[0162] In ST41, the haptic control device 50 accepts input of the expression frequency. The sensory parameters used for generation by the classification unit 64 are shown in Fig. 27(a). The sensory parameters are, for example, the following 24 parameters. 24 is just an example, and the number may be less or more. "Light (heavy) operating force" "There is no sense of decision" "Inaccurate (Accurate)" "Clear (ambiguous)" "Soft (hard)" "Blurred (clear)" "Smooth" "Tired (not tired)" "Strict (kind)" "Coarse (fine)" "There is no sensation of being sucked in (there is)." "Innovative (Traditional)" "Cheap (luxury)" "Durable (not)" "I don't want to operate it again (I want to)" "Fun (boring)" "Not comfortable (good)" "I hate (like)" "There is no bouncy sensation (there is)." "Mild (Sharp)" "Dry (wet)" "Bright (dark)" "Cold (warm)" "With (or without) play" These affective parameters may be automatically generated by web analysis, tweet analysis, SNS analysis, papers, clustering analysis for each market, feature and adjective extraction, etc. In other words, the affective parameters do not have to be fixed, but may be dynamically changeable.
[0163] In ST42, the haptic control device 50 learns the correspondence between the expression levels of the sensitivity parameters and the reference operating tools 51a to 51c by machine learning. The classification unit 64 has this correspondence.
[0164] Machine learning is a technology that allows a computer to acquire human-like learning capabilities, in which the computer autonomously generates algorithms required for judgments such as data identification from training data that is previously imported, and applies these to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods. Machine learning methods include perceptrons, deep learning, support vector machines, logistic regression, naive Bayes, decision trees, random forests, and the like, and the learning methods are not limited. Deep learning and decision trees will be explained later as examples of learning methods.
[0165] In ST43, the classifier 64 generated by machine learning is incorporated into the haptic control device 50.
[0166] FIG. 31 is a flow chart showing the flow of learning the correspondence between the expression frequency of the sensitivity parameters and the physical parameters.
[0167] In ST51, the haptic control device 50 accepts an input of the expression frequency. The sensory parameters used for learning the correspondence between the expression frequency of the sensory parameters and the physical parameters are shown in Fig. 28(a). The sensory parameters are, for example, the following five. Five is just an example, and the number of parameters may be less or more. "Mild (Sharp)" "Coarse (fine)" "Bright (dark)" "Soft (hard)" "Light (heavy)" In ST52, the haptic control device 50 determines the correspondence between the expression frequency of the sensitivity parameters and the physical parameters by multiple regression analysis. In this embodiment, three reference operating tools 51a to 51c are prepared, and therefore a load-displacement curve is obtained for each of the three reference operating tools 51a to 51c. The physical parameters that realize this load-displacement curve are also known. The user operates the reference operating tools 51a to 51c and inputs the operation feel of the reference operating tools 51a to 51c as the expression frequency. When the expression frequencies of a sufficient number of people are input, the haptic control device 50 performs multiple regression analysis using Equation 5. The multiple regression analysis has been described with reference to Equation 5 in embodiment 1, and Figs. 22 and 23. Therefore, the coefficient of determination B 11 ~B mn can be determined, and a transformation model 15 as shown in Fig. 23 is obtained for each of the reference operating tools 51a to 51c. The transformation models for the three reference operating tools 51a to 51c are a first transformation model 65a to a third transformation model 65c.
[0168] In ST53, the first conversion model 65a to the third conversion model 65c generated by the multiple regression analysis are incorporated into the haptic control device 50.
[0169] [Tactile presentation flow] FIG. 32 is a flow chart showing the flow in which the tactile control device 50 presents the user's preferred operation feel using the classification unit 64 and the first to third conversion models 65a to 65c.
[0170] In ST61, the first input receiving section 62 receives an input of the expression frequency of the affective parameter on the first input screen 281 in order to select the standard operating tools 51a to 51c (STEP 1).
[0171] In ST62, the classification unit 64 specifies the reference operating tools 51a to 51c based on the expression frequency of each affective parameter input on the first input screen 281. When the reference operating tools 51a to 51c are determined, one of the first to third conversion models 65a to 65c is also determined.
[0172] In ST63, the physical parameter setting section 66 sets the physical parameters of the selected reference operating tools 51a to 51c in the reproduced operating tool 52. The user can operate the reproduced operating tool 52 to check whether it provides the desired operating feel.
[0173] In S64, the user determines whether to adjust the operation feel to a different one from the reference operation tools 51a to 51c depending on whether the operation feel is the user's preference. The tactile control device 50 receives an instruction to start readjustment from the user.
[0174] In S65, when the user adjusts the operation feel to be different from that of the reference operation tools 51a to 51c, the second input receiving unit 63 receives an input of the expression degree of the affective parameter on the second input screen 120 (STEP 2). n (corresponding to ST63) is a model in which any one of the first conversion model 65a to the third conversion model 65c selected in ST63 is used as the physical parameters P1 to P n The physical parameter setting unit 66 converts the physical parameters P1 to P n is set in the reproduction operation tool 52. The user can operate the reproduction operation tool 52 again to check whether the operation feel is the one he or she prefers.
[0175] Thereafter, the user can repeatedly use the second input screen 120 to adjust the preferred operation feel until the user obtains the operation feel that he or she prefers.
[0176] In this manner, the haptic control device 50 of this embodiment can reproduce the operation feel preferred by the user in real time.
[0177] <Second form of haptic control device> Next, a second embodiment of the haptic control device 50 will be described.
[0178] First, an outline of the operation of the haptic control device 50 of the second embodiment will be described with reference to Fig. 33. Fig. 33 shows an outline of the operation in which the user adjusts the operational feel using the haptic control device 50.
[0179] (1) First, the user inputs expression levels that represent his / her preferences for a plurality of affective parameters on a first input screen 281 (FIG. 33(a)). The first input screen 281 may be the same as that shown in FIG. 27(a).
[0180] (2) The tactile control device 50 determines a physical parameter (an example of a second physical parameter) corresponding to the expression frequency of each sensory parameter based on the correspondence between the expression frequency of each sensory parameter and a physical parameter (load-displacement curve) that has been learned in advance by regression (Figure 33(b)).
[0181] (3) The tactile control device 50 performs curve fitting using an appropriate fitting model on the load-displacement curves of the reference operating tools 51a to 51c that have been prepared in advance (FIG. 33(c)). This fitting model is, for example, a polynomial with physical parameters as coefficients. Therefore, physical parameters (an example of first physical parameters) that represent the load-displacement curves of the reference operating tools 51a to 51c are obtained. The tactile control device 50 compares the physical parameters in (2) with the physical parameters in (3).
[0182] (4) If the physical parameters of (2) and (3) are similar, the tactile control device 50 presents a similar reference operating tool 51; if they are not similar, the tactile control device 50 suggests adjusting the new sensation using the reproduction operating tool 52 (Figure 33(d)).
[0183] [Functions of the haptic control device] Fig. 34 is a functional block diagram for explaining the functions of the haptic control device 50. Note that the explanation of Fig. 34 may mainly explain the differences from Fig. 29. The haptic control device 50 has a display control unit 61, a first input receiving unit 62, a second input receiving unit 63, a physical parameter conversion unit 67, a curve fitting unit 68, a comparison unit 69, a first conversion model 65a, a second conversion model 65b, a third conversion model 65c, and a physical parameter setting unit 66. Each of these functions of the haptic control device 50 is realized by a CPU included as an information processing device executing a program expanded in a RAM. Alternatively, each function may be realized by a hardware circuit.
[0184] The physical parameter conversion unit 67 uses the correspondence between the expression frequency and the physical parameters obtained by the multiple regression analysis to determine the physical parameters corresponding to the expression frequency accepted by the first input acceptance unit 62. Note that since the load-displacement curve is also determined when the physical parameters are determined, it may be said that the physical parameter conversion unit 67 determines the load-displacement curve.
[0185] The curve fitting unit 68 fits the load-displacement curves of the reference operating tools 51a to 51c (the first conversion model 65a to the third conversion model 65c) with an appropriate fitting model (for example, a polynomial). Note that curve fitting is a form of multiple regression analysis. By setting physical parameters as coefficients of the polynomial, the curve fitting unit 68 can estimate the physical parameters for each of the reference operating tools 51a to 51c. Therefore, it is preferable to select a fitting model that allows fitting of the load-displacement curves with physical parameters.
[0186] The comparison unit 69 compares the physical parameters determined by the physical parameter conversion unit 67 with the physical parameters determined by the curve fitting unit 68, and judges whether they are similar to each other. n The comparison unit 69 calculates the sum of squares of the differences for each reference operating tool 51a to 51c and judges whether it is less than a threshold value. If there are similar physical parameters, the comparison unit 69 indicates to the physical parameter setting unit 66 the physical parameters corresponding to the reference operating tools 51a to 51c.
[0187] The physical parameter setting unit 66 sets the specified physical parameters of the reference operating tool 51 in the reproduced operating tool 52 .
[0188] [Learning the physical parameters (load-displacement curve) corresponding to the expression frequency, and curve fitting of the load-displacement curve of the standard operating tool] Next, learning of physical parameters (load-displacement curve) corresponding to expression frequencies will be described with reference to Fig. 35 etc. Fig. 35 is a flow chart showing the flow of learning of physical parameters (load-displacement curve) corresponding to expression frequencies.
[0189] In ST71, the haptic control device 50 receives an input of the expression frequency. The sensitivity parameters used for generation by the classifier 64 are shown in FIG.
[0190] In ST72, the haptic control device 50 determines the correspondence between the expression frequency of the sensory parameters and the physical parameters by multiple regression analysis. The user inputs the type of operation feeling for the operating tool whose physical parameters are known as the expression frequency. The operating tool whose physical parameters are known may be the reference operating tool 51 or any operating tool. When the expression frequencies of a sufficient number of people have been input, the haptic control device 50 performs multiple regression analysis using equation 5. Multiple regression analysis has been described with reference to equation 5 in embodiment 1, and Figs. 22 and 23. Therefore, the haptic control device 50 uses the coefficient of determination B 11 ~B mn can be determined, and a transformation model 15 as shown in FIG. 23 is obtained.
[0191] In ST73, the physical parameter conversion section 67 generated by the multiple regression analysis is incorporated into the haptic control device 50.
[0192] FIG. 36 is a flow chart showing a flow of curve fitting the load-displacement curves of the reference operating tools 51a to 51c.
[0193] In ST81, the curve fitting unit 68 performs curve fitting on the load-displacement curves of the reference operating tools 51a to 51c. As shown in FIG. 9, the correspondence between the stroke amount x and the operation reaction force is obtained for each of the reference operating tools 51a to 51c. The curve fitting unit 68 extracts pairs of stroke amount and operation reaction force, preferably at regular intervals, from x=0 to the maximum stroke amount. The curve fitting unit 68 applies pairs of stroke amount x and operation reaction force y to a fitting model to perform curve fitting. The fitting model is an equation for determining the operation stress from the stroke amount x using physical parameters as coefficients. The following fitting model is an example, and any appropriate model (equation) for determining the operation reaction force y from the stroke amount x using physical parameters as coefficients may be adopted. Fitting model: y=P1×x 0 +P2×x 1 +P3×x 2 +……P n x n The curve fitting unit 68 can obtain P1 to Pn by multiple regression analysis. n corresponds to a physical parameter.
[0194] In ST82, the physical parameters of the reference operating tools 51a to 51c generated by curve fitting are set in the comparison section 69.
[0195] [Tactile presentation flow] FIG. 37 is a flow chart showing the flow in which the haptic control device 50 uses the physical parameter conversion section 67 and the comparison section 69 to present the user's preferred operation feel.
[0196] In ST91, the first input receiving section 62 receives an input of the expression frequency of the affective parameters for selecting the reference operating tools 51a to 51c.
[0197] In ST92, the physical parameter conversion section 67 converts each sensory parameter into a physical parameter (load-displacement curve) based on the expression frequency of each sensory parameter.
[0198] In ST93, the comparison section 69 compares the physical parameters determined by the physical parameter conversion section 67 with the physical parameters determined by the curve fitting section 68 for each of the reference operating tools 51a to 51c.
[0199] In S94, the comparison unit 69 judges whether or not there are any of the reference operating tools 51a to 51c having physical parameters similar to the physical parameters converted by the physical parameter conversion unit 67. This judgment is made based on the physical parameters P1 to P2 determined by the physical parameter conversion unit 67 as described above. n and the physical parameters P1 to P2 obtained by curve fitting of the reference operating tools 51a to 51c. n There is a method for determining whether the sum of squares of the differences is less than a threshold.
[0200] If the determination in S94 is Yes, in S95, the physical parameter setting section 66 sets the physical parameters of the reference operating tools 51a to 51c which are similar to the physical parameters determined by the physical parameter conversion section 67 in the replica operating tool 52.
[0201] If the determination in S94 is No, in S96, the physical parameter setting unit 66 sets the physical parameters of the reference operating tool 51a-51c having the highest similarity to the reproduced operating tool 52. Alternatively, the first type of classification unit 64 may be provided, and the classification unit 64 may determine the reference operating tools 51a-51c (first conversion model 65a-third conversion model 65c).
[0202] Thereafter, the user can repeatedly use the second input screen 120 to adjust the preferred operation feel until the user obtains the operation feel that he or she prefers.
[0203] In this manner, the haptic control device 50 of this embodiment can reproduce the operation feel preferred by the user in real time.
[0204] [Classification learning example] A classification learning method will be described with reference to FIG. 38 and the like. FIG. 38 shows an example of a neural network in which the classification unit 64 is realized by a neural network. In the neural network of FIG. 38, three nodes in the output layer 133 output an output value yi for data input to the input layer 131. This output value yi is a probability, and y1+y2+y3 is 1.0. In this embodiment, the three nodes in the output layer 133 correspond to the three reference operating tools 51a-51c, and output the probability that one of the three reference operating tools 51a-51c is likely according to the expression frequency.
[0205] FIG. 38 shows a neural network in which L layers (for example, three layers) from the input layer 131 to the output layer 133 are fully connected. A neural network with a deep hierarchy is called a DNN (Deep Neural Network). The layer between the input layer 131 and the output layer 133 is called an intermediate layer 132. The number of layers and the number of nodes in the intermediate layer can be set arbitrarily, so the number of layers and the number of nodes 130 in each layer are merely examples. In this embodiment, the number of nodes 130 in the input layer is the number of affective parameters (24 in FIG. 27(a)). Note that the expression level may be set for each affective parameter in any stage adjustment, such as five stages, three stages, etc., or may be continuously adjustable.
[0206] In a neural network, all nodes 130 in the l-1th layer are connected to one node 130 in the lth layer (l: 2, 3) excluding the input layer, and the product of the output z of node 130 in the l-1th layer and the connection weight w is input to the node in the lth layer. Equation (1) shows how to calculate the output signal of node 130.
[0207]
number
[0208]
number
[0209] Each node in the output layer 133 receives z i is input, and each node of the output layer 133 is z i Then, an activation function for the output layer is used for the nodes of the output layer 133. In the case of multi-value classification (selection of the reference operating tools 51a to 51c), the activation function of the output layer 133 is generally a softmax function. Each node of the output layer 133 is the output value y i During learning, a teacher signal (1 or 0) is set after each node of the output layer 133 corresponds to a reference operating tool. If learning is performed appropriately, each node of the output layer 133 can output the probability of the reference operating tools 51a to 51c corresponding to the 24 sensitivity parameters. In the figure, the operating tools correspond to the reference operating tools 51a to 51c from the top. However, if the output value is less than the threshold value, it may be determined to be unclassified.
[0210] The learning of the neural network will now be described. A number of users operate the three reference operating tools 51a to 51c and input the expression level for each of the reference operating tools 51a to 51c. In this way, training data consisting of a set of 24 sensitivity parameters and one teacher signal (which reference operating tool) is obtained in the number of users x the number of reference operating tools. The teacher signal is either (1,0,0), (0,1,0), or (0,0,1).
[0211] The neural network processes the expression frequency input to the input layer 131 and outputs an output value yi from the output layer 133. A teacher signal contained in the training data paired with the input expression frequency is input to the node of the output layer 133. During learning, the error between the output value yi of the node of the output layer 133 and the teacher signal is calculated by a loss function. When the activation function of the output layer 133 is a softmax function, the loss function is cross entropy. The error between the teacher signal and the output value calculated by the loss function is propagated to the node of the input layer 131 by a calculation method called the backpropagation method. The weight w between the nodes is learned during the propagation process. Details of the backpropagation method are omitted.
[0212] As a result of learning, for example, when an expression frequency is input for the reference operating tool 51a, the neural network is expected to have the node 130 corresponding to the reference operating tool 51a in the output layer 133 output a value close to 1.0, and the nodes 130 corresponding to the reference operating tools 51b and 51c output values close to 0.0.
[0213] In FIG. 38, the nodes are fully connected, but convolutional layers, pooling layers, etc. may be included.
[0214] Fig. 39 shows an example of a decision tree when the classification unit 64 is realized by a decision tree. A decision tree is a machine learning technique that finds a cluster of data in which a specific feature often appears, and generates a classification rule for the cluster. In this embodiment, learning corresponds to determining the affective parameters that often appear in each of the three reference operating tools 51a to 51c and their expression frequencies. A method using entropy is known as one method for learning the structure of a decision tree.
[0215] In addition, examples of machine learning suitable for classification include neural networks and decision trees, as well as support vector machines, random forests, logistic regression, and the like.
[0216] [Additional information on the first input screen] FIG. 40 is a diagram illustrating a first input screen 281 for the expression frequency in STEP 1. The user operates a slide bar for each affective parameter to input the expression frequency. The classification unit 64 described in the first embodiment uses the learning result to calculate the probability that each of the reference operating tools 51a to 51c will be selected for the current expression frequency. The display control unit 61 displays the probability of each of the reference operating tools 51a to 51c in the reference operating tool field 112. Therefore, the user can grasp which of the reference operating tools 51a to 51c the current expression frequency is closest to by operating the reference operating tools 51a to 51c. The probability may be displayed in real time or when the user inputs a decision operation.
[0217] Furthermore, when the user presses the icon of the reference operating tool 51a-51c in the reference operating tool field 112, the display control unit 61 initializes the slide bar in the affective parameter presentation field 282 to the expression frequency set for the reference operating tools 51a-51c. Therefore, the user can easily check the expression frequency of each of the reference operating tools 51a-51c. Note that the expression frequency at the time of initialization may be the median or average value of the expression frequency input for the reference operating tool 51 in a sensory test, for example.
[0218] [Operation of client-server system] Next, the operation of the client-server system will be described with reference to Fig. 41 etc. Fig. 41 is a functional block diagram of a haptic control system 2 in which the haptic control device 50 of the first embodiment is applied to a client-server system. In the explanation of Fig. 41, differences from Fig. 29 will be mainly described. As shown in Fig. 41, the terminal device 80 and the server 200 have the same functions as the haptic control device 50 of Fig. 29, except that the terminal device 80 and the server 200 have a first communication unit 71 and a second communication unit 72, respectively.
[0219] Fig. 42 is a sequence diagram for explaining the operation of the haptic control system 2. In the explanation of Fig. 42, differences from Fig. 32 will be mainly explained.
[0220] In ST101, the first input receiving unit 62 receives an input of the expression levels of the affective parameters for selecting the reference operating tools 51a to 51c inputted into the first input screen 281 (STEP 1).
[0221] In ST102, the first communication unit 71 of the terminal device 80 transmits the expression frequency of each affective parameter to the server 200.
[0222] In ST103, the classification unit 64 of the server 200 selects the reference operating tools 51a to 51c based on the expression frequency of each affective parameter.
[0223] In ST104, the second communication unit 72 of the server 200 transmits the physical parameters of the reference operating tools 51a to 51c to the terminal device 80. The first communication unit 71 of the terminal device 80 receives the physical parameters of the reference operating tools 51a to 51c, and the physical parameter setting unit 66 sets them in the reproduction operating tool 52.
[0224] In ST105, the user determines whether to adjust the operation feel to a different one from the reference operation tools 51a to 51c depending on whether the operation feel is the user's preference. If the user adjusts the operation feel to a different one from the reference operation tools 51a to 51c, the second input receiving unit 63 receives an input of the expression level of the affective parameter input on the second input screen 120 (STEP 2).
[0225] In ST106, the first communication unit 71 of the terminal device 80 transmits the expression frequency of the affective parameter to the server 200.
[0226] In ST107, one of the first conversion model 65a to the third conversion model 65c of the server 200 (already selected in ST103) is used to calculate the expression frequency of the physical parameters P1 to P n Convert to.
[0227] In ST108, the physical parameter setting unit 66 of the server 200 transmits the converted physical parameters to the terminal device 80 via the second communication unit 72. The physical parameters of the reference operating tools 51a to 51c received by the first communication unit 71 of the terminal device 80 are set in the reproduced operating tool 52.
[0228] In this way, the haptic control system 2 of this embodiment can reproduce the operation feel preferred by the user in real time even in a client-server system.
[0229] Figure 43 is a functional block diagram of a haptic control system 2 in which the haptic control device 50 of the second embodiment is applied to a client-server system. In the explanation of Figure 43, the differences from Figure 34 will be mainly explained. As shown in Figure 43, the terminal device 80 and the server 200 have the same functions as the haptic control device 50 of Figure 34, except that the terminal device 80 and the server 200 have a first communication unit 71 and a second communication unit 72, respectively.
[0230] Fig. 44 is a sequence diagram for explaining the operation of the haptic control system 2 of the second embodiment. In the explanation of Fig. 44, differences from Fig. 37 will be mainly explained.
[0231] In ST111, the first input receiving section 62 receives an input of the expression frequency of the affective parameter on the first input screen 281.
[0232] In ST112, the first communication unit 71 of the terminal device 80 transmits the expression frequency of each affective parameter to the server 200.
[0233] In ST113, the physical parameter conversion section 67 of the server 200 converts each sensory parameter into a physical parameter (load-displacement curve) based on the expression frequency of each sensory parameter.
[0234] In ST114, the comparison section 69 of the server 200 compares the physical parameters converted by the physical parameter conversion section 67 with the physical parameters previously determined by the curve fitting section 68 for each of the reference operating tools 51a to 51c.
[0235] In ST115, if there are reference operating tools 51a-51c with physical parameters similar to the physical parameters determined by the physical parameter conversion unit 67, the second communication unit 72 transmits the physical parameters of one of the similar reference operating tools 51a-51c to the terminal device 80. The first communication unit 71 of the terminal device 80 receives the physical parameters of the reference operating tools 51a-51c, and the physical parameter setting unit 66 sets them in the reproduction operating tool 52.
[0236] In ST116, if there is no reference operating tool 51a-51c with physical parameters similar to those determined by the physical parameter conversion unit 67, the second communication unit 72 transmits the physical parameters of one of the reference operating tools 51a-51c with the highest similarity to the terminal device 80. The first communication unit 71 of the terminal device 80 receives the physical parameters of the reference operating tools 51a-51c, and the physical parameter setting unit 66 sets them in the reproduced operating tool 52. Alternatively, the first form of classification unit 64 may be provided, and the classification unit 64 may determine the reference operating tools 51a-51c (first conversion model 65a-third conversion model 65c).
[0237] In this way, the haptic control system 2 of this embodiment can reproduce the operation feel preferred by the user in real time even in a client-server system.
[0238] [Additional Note for Mode 2] [Claim 1] A tactile control device for controlling the operation feel of an operating tool, a display control unit that displays an input means for inputting a first expression frequency associated with a first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a physical parameter setting unit that sets a pre-prepared physical parameter in the reproduction operating tool based on the first expression degree, the display control unit displays an input means for inputting a second expression frequency associated with a second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation; A conversion unit that converts the second expression frequency into a physical parameter by a regression model, The haptic control device according to claim 1, wherein the physical parameter setting unit sets the physical parameters converted by the conversion unit to the reproduction operating tool. [Claim 2] a classification unit that classifies the first expression frequency into one of a plurality of reference operating tools; 2. The haptic control device according to claim 1, wherein the physical parameter setting unit sets the physical parameters set for the reference operating tool classified by the classification unit to the reproduced operating tool. [Claim 3] a curve fitting unit that performs curve fitting on a load-displacement curve realized by a first physical parameter of a plurality of reference operating tools, and estimates the first physical parameter for each of the plurality of reference operating tools; a physical parameter conversion unit that converts the first expression frequency into a second physical parameter by a regression model, The tactile control device according to claim 1, characterized in that the physical parameter setting unit sets the first physical parameter of the reference operating tool, which has the first physical parameter that is most similar to the second physical parameter, to the reproduced operating tool. [Claim 4] The tactile control device according to claim 2, characterized in that the input means for the second expression frequency can take on an expression frequency corresponding to the physical parameters set for the reference operating tool classified by the classification unit, as well as values before and after the expression frequency. [Claim 5] 2. The haptic control device according to claim 1, characterized in that the first sensory parameters and the second sensory parameters are each multiple, and the number of the first sensory parameters is greater than the number of the second sensory parameters. [Claim 6] The tactile control device according to claim 2, characterized in that the classification unit is generated by learning a correspondence between the operation feels of the plurality of reference operating tools and the expression levels input by a user for each of the first affective parameters by operating the plurality of reference operating tools. [Claim 7] 3. The tactile control device according to claim 2, characterized in that the regression model is generated by performing a regression analysis on the correspondence between the physical parameters of the plurality of reference operating tools and the expression levels input for each of the second affective parameters by a user operating the plurality of reference operating tools. [Claim 8] The tactile control device according to claim 3, characterized in that the regression model is generated by performing a regression analysis on the correspondence between physical parameters of an arbitrary reference operating tool and expression levels input for each of the first affective parameters by a user operating the arbitrary reference operating tool. [Claim 9] The tactile control device according to claim 3, characterized in that the curve fitting unit performs curve fitting on the load-displacement curve using a fitting model that uses the first physical parameter as a coefficient to determine the operation stress from the stroke amount, thereby estimating the first physical parameter. [Claim 10] the first affective parameter and the second affective parameter are adjectives, 2. The haptic control device according to claim 1, wherein the first expression degree and the second expression degree are values indicating the degree of the adjective. [Claim 11] 11. The tactile control device according to any one of claims 1 to 10, characterized in that the first expression frequency and the second expression frequency are tactile information obtained when a user operates an operating tool, respectively. [Claim 12] The tactile control device according to claim 11 , wherein in the regression model, the first expression frequency and the second expression frequency are correlated with an actuation force as a tactile sensation obtained when operating each of the operating tools. [Claim 13] A tactile control device for controlling the feel of an operating tool, a display control unit that displays an input means for inputting a first expression frequency associated with a first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a physical parameter setting unit that sets a pre-prepared physical parameter in the reproduction operating tool based on the first expression degree; the display control unit displays an input means for inputting a second expression frequency associated with a second affective parameter; Further, a second input receiving unit that receives an input of the second expression frequency in response to a user operation; a conversion unit that converts the second expression frequency into a physical parameter by a regression model; The physical parameter setting unit sets the physical parameters converted by the conversion unit to the reproduction operating tool. [Claim 14] A haptic control method in which a haptic control device that controls an operation feel of an operating tool controls a haptic sensation, comprising: displaying an input means for a first expression frequency associated with a first affective parameter; receiving an input of the first expression frequency in response to a user operation; A step of setting a prepared physical parameter for a reproduction operating tool based on the first expression degree; displaying an input means for inputting a second expression frequency associated with the second affective parameter; receiving an input of the second expression frequency in response to a user operation; converting the second expression frequency into a physical parameter by a regression model; setting the converted physical parameters to the reproduction operation tool; A tactile control method comprising: [Claim 15] A haptic control system in which a terminal device and a server communicate with each other via a network, The terminal device a display control unit that displays an input means for inputting a first expression frequency associated with a first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; a first communication unit that transmits the first expression frequency to the server; a physical parameter setting unit that sets the physical parameters transmitted from the server to the reproduction operation tool, the display control unit displays an input means for inputting a second expression frequency associated with a second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation, The first communication unit transmits the second expression frequency to the server; The server, a second communication unit that determines the physical parameter prepared in advance based on the first expression frequency received from the terminal device, and transmits the determined physical parameter to the terminal device; a conversion unit that converts the second expression frequency received from the terminal device into a physical parameter by a regression model, A haptic control system characterized in that the second communication unit transmits the physical parameters converted by the conversion unit to the terminal device. [Claim 16] a display control unit that displays an input means for inputting a first expression frequency associated with a first affective parameter; a first input receiving unit that receives an input of the first expression frequency in response to a user operation; A first communication unit that transmits the first expression frequency to a server, and a physical parameter setting unit that sets the physical parameters transmitted from the server to a reproduction operating tool. the display control unit displays an input means for inputting a second expression frequency associated with a second affective parameter; a second input receiving unit that receives an input of the second expression frequency in response to a user operation, and the first communication unit communicates with a terminal device that transmits the second expression frequency to the server via a network, a second communication unit that determines a pre-prepared physical parameter based on the first expression frequency received from the terminal device, and transmits the determined physical parameter to the terminal device; a conversion unit that converts the second expression frequency received from the terminal device into the physical parameter by a regression model, The server is characterized in that the second communication unit transmits the physical parameters converted by the conversion unit to the terminal device. [Aspect 3]
[0239] [Background technology] Conventionally, there are known operation units that provide a sense by providing some kind of stimulus to a person. Here, the sense is provided by a tactile sense, an auditory sense by sound, or a visual sense by displaying an image. The sense is adjusted by adjusting a signal that drives various operation units.
[0240] There is known a game controller in which buttons with built-in vibration devices can be replaced (see, for example, Patent Document 3). Patent Document 3 discloses a technique for replacing the vibration device itself in order to realize different vibration intensities.
[0241] [Summary of the Invention] [Problem to be solved by the invention] However, the conventional technology has a problem in that it does not adequately present sensations according to the physical characteristics of the operation unit. For example, in the case of a rotary operation unit, the sensations conveyed to the user operating the operation unit differ depending on the size and mass of the operation unit, even if the actuator is driven in the same way.
[0242] In view of the above-mentioned problems, an object of the present aspect is to provide a technology for presenting sensations according to the physical characteristics of an operating unit.
[0243] [Effects of the invention] It is possible to provide a technology that presents sensations according to the physical characteristics of the operating unit.
[0244] [Explanation of Aspect 3] In this embodiment, a sensation control method for making adjustments based on the physical characteristics of an operation unit (for example, an operation device 33 in FIG. 45 described later) will be described. When the tactile presentation device 20 generates a tactile sensation through the operation unit by driving an actuator, depending on the physical characteristics (size, mass, etc.) of the operation unit, the sensation (operation sensation perceived by the user) that is conveyed to the user operating the operation unit (an example of an operator) may differ even if the actuator is driven in the same way.
[0245] In other words, the physical parameters correlated with the sensory parameters are composed of a combination of the physical parameters of the operation unit and the physical parameters of the actuator. Therefore, the tactile presentation device 20 of this embodiment adjusts the tactile presentation signal to be suitable for physical parameters such as the size and mass of the operation unit. The tactile control system 110 includes an adjustment unit that adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation based on the physical characteristics of the operation unit.
[0246] For example, the difference in the physical characteristics of the operation unit is detected as follows. The user inputs information about the difference in physical characteristics of the operation unit into the input / output device 3. The size and mass of the operation unit are identified. The tactile presentation device 20 detects the ID, size, mass, etc., which represent the differences in physical characteristics of the operation part, using a sensor. The sensor that detects the difference in the physical characteristics of the control part is a camera that reads one-dimensional and two-dimensional codes. The camera also identifies the control part by recognizing the image of the control part. Alternatively, the sensor is an IC tag reader that reads the ID.
[0247] [Tactile Control System 110] Fig. 45 is a diagram showing the configuration of the haptic control system 110 of the sense control system 100 in this embodiment. In this embodiment, components with the same reference numerals in Fig. 2 perform similar functions, so that only the main components of this embodiment may be mainly described.
[0248] The tactile presentation device 20 in FIG. 45 newly includes an operation unit sensor 254, a torque sensor 251, and a communication unit 256. When a detachable operation unit is attached to the tactile presentation device 20, the operation unit sensor 254 detects the attachment and information that can identify the operation unit. The information that can identify the operation unit includes an IC tag built into the operation unit, a one-dimensional code or a two-dimensional code attached to the operation unit, and the appearance of the operation unit. When the information that can identify the operation unit is an IC tag, the operation unit sensor 254 is an IC tag reader and acquires an ID (identification information) of the operation unit from the IC tag. When the information that can identify the operation unit is a one-dimensional code or a two-dimensional code, the operation unit sensor 254 is a camera and acquires the ID of the operation unit from the one-dimensional code or the two-dimensional code. When the information that can identify the operation unit is the appearance of the operation unit, the operation unit sensor 254 is a camera and a classifier, and identifies the operation unit using a classifier that has learned the correspondence between image data of the appearance of the operation unit and the ID (the ID of the operation unit is known).
[0249] The operation unit is an example of the operation device 33, and may be an attachment unit that is detachable from at least a part (which may be the whole or a part) of the operation device 33. The main control device 10 and the tactile presentation device 20 are examples of a sensory control device.
[0250] The torque sensor converts the current that drives the actuator into torque during calibration to estimate the mass of the operation unit, as will be described in detail later.
[0251] The communication unit 256 receives the size of the operation unit from the mobile terminal 60 by communicating with the mobile terminal 60. Details will be described later.
[0252] Furthermore, main control device 10 in Fig. 45 newly includes operation unit parameters 54, a calibration unit 55, and a mass correction unit 261. Operation unit parameters 54 will be described in Fig. 46. Calibration unit 55 estimates the mass of the operation unit by calibration. Mass correction unit 261 corrects the mass of the operation unit. Calibration unit 55 and mass correction unit 261 will be described later.
[0253] 46 shows an example of the operation unit parameters 54. In the operation unit parameters 54, the ID of the operation unit is associated with mass, size, and other physical parameters. Mass and size are physical characteristics of the operation unit 201, and in this embodiment, mass and size are included in the physical parameters.
[0254] The size may be the radius, diameter, or total length (length of the largest part) if the operation unit is a rotary operation unit that accepts rotation operations. Also, if the operation unit is a pressure type operation unit, the size may be the length in the pressing direction. If the operation unit is a slide operation unit that accepts slide operations, the size may be any of the length of the slide amount, height, width, or thickness. If the operation unit is a pivot operation unit that accepts tilt operations, the size may be the length of the operation unit.
[0255] The other physical parameters are the same as those described in embodiment 1. As shown in Fig. 46, when the ID of the operation unit is detected by the operation unit sensor 254, the physical parameters are known. [Detection of operation section by operation section sensor] A method for detecting an operation unit by operation unit sensor 254 will be described with reference to Figs. 47 and 48. First, Fig. 47 is a diagram for explaining the difference in physical characteristics of rotary operation units. Fig. 47(a) shows small operation unit 201a, and Fig. 47(b) shows large operation unit 201b. In the following, any one of operation units 201a, 201b will be referred to as "operation unit 201".
[0256] The operation units 201a and 201b in Fig. 47 are rotary type, but the tactile sensation transmitted to the user operating the operation units differs depending on the size (diameter) and mass of the operation units 201a and 201b, even if the processor 14 drives the actuators in the same way. For example, the torque required to rotate the operation unit 201 can be smaller as the diameter increases. For this reason, if the reaction force against the rotation operation of the operation units 201a and 201b is made the same, the operation unit 201a may feel difficult to turn, or the operation unit 201b may feel no operation sensation when operating it.
[0257] Even if the sizes of the operation units 201 differ, the operation units are usually similar in shape, so there is a certain relationship between size and mass. For example, mass is proportional to the cube of size (e.g., radius), and an approximate proportionality constant can be calculated. For this reason, as described later, it is possible to find the mass from the size of the operation unit 201, or find the size from the mass, using a conversion formula.
[0258] Fig. 48 is a diagram for explaining several methods for detecting the size and mass of the operation unit 201 by the operation unit sensor 254. In Fig. 48(a), the operation unit 201 has an IC tag 202 built in or attached thereto. In this case, the operation unit sensor 254 is an IC tag reader 204, which energizes the IC tag 202 with electromagnetic waves, communicates with the IC tag 202, and receives the ID of the operation unit from the IC tag 202. The IC tag reader 204 is preferably installed in the tactile presentation device 20, but may be an external device such as a mobile terminal 60.
[0259] 48(b), a barcode 203 is attached to the operation unit 201. In this case, the operation unit sensor 254 captures an image of the barcode 203 with the camera 205 and obtains the ID of the operation unit by decoding the barcode 203. The camera 205 is preferably installed in the tactile presentation device 20, but may be an external device such as the mobile terminal 60.
[0260] In Fig. 48(c), the operation unit sensor 254 captures the operation unit 201 itself with a camera. The operation unit sensor 254 estimates the size of the operation unit sensor 254 from the image data based on the preset distance between the camera 205 and the operation unit 201 and the focal length of the camera 205. If the classifier has learned the correspondence between the ID and the distance, focal length, and image data of the appearance of the operation unit, it can output the ID of the operation unit from the image data. Regarding mass, a conversion formula is used to calculate mass from size.
[0261] 48 may be built into the tactile presentation device 20, or may exist separately from the tactile presentation device 20. For example, the operation sensor 254 may be an information processing device carried by a user, such as a mobile terminal 60.
[0262] [When the control unit detected by the control unit sensor is not included in the control unit parameters] There may be a case where the operation unit detected by the operation unit sensor 254 is not included in the operation unit parameters. For example, In the case of controllers used by users, such as game controllers, the user may be able to change the knobs and other operating parts, and may wish to obtain an operating feel that suits the attached part. In some cases, a user may wish to change the operating handle of a vehicle or the like to obtain an operating feel that is suitable for the installed handle.
[0263] If the operation unit detected by the operation unit sensor 254 is not included in the operation unit parameters, the mobile terminal 60 estimates the physical parameters. The user starts a predetermined application on the mobile terminal 60. The user takes an image of the operation unit 201 attached to the tactile presentation device 20 with a camera controlled by the application. As a result, the application detects the size of the operation unit 201 from the image data of the operation unit 201. Therefore, the camera of the mobile terminal 60 is preferably a stereo camera or a LiDAR (lidar) scanner. The application transmits the size of the operation unit 201 to the tactile presentation device 20. The communication unit 256 receives the size of the operation unit 201.
[0264] In addition, even if the communication unit 256 receives the size, the mass is unknown, so a conversion formula for calculating the mass from the size is used for the mass of the operation unit 201. Alternatively, the application calculates the mass from the size using the conversion formula and transmits the mass to the tactile presentation device 20.
[0265] [Estimation of mass of control unit through calibration] Next, a method for estimating mass through calibration by the calibration unit 55 will be described. When the operation unit 201 is attached, the calibration unit 55 operates the operation unit with a current pattern (rotates the operation unit in the case of a rotary type), and estimates the mass of the operation unit from the correspondence between the current and the position.
[0266] Fig. 49 is a diagram for explaining a method for estimating the mass of the operation unit by calibration. First, Fig. 49(a) is a diagram for explaining the position of the rotary type operation unit 201. In the case of the rotary type operation unit 201, the position may be the rotation angle of the center of rotation. In the case where the top surface of the operation unit 201 is circular, the center of rotation is the center of the circle. When the calibration unit 55 rotates the rotary type operation unit 201, the heavier the mass, the larger the current required.
[0267] FIG. 49(b) is a diagram for explaining the relationship between the current and the position required to change the position of the operation unit 201. The relationship between the current and the position shown in FIG. 49(b) is an example for explanation. In general, the more the position is changed, the larger the current required. Also, the current has a certain relationship with the torque that rotates the operation unit, and the torque required to rotate the operation unit can be found from the current. It is known that the current required to change the position is larger as the mass of the operation unit is larger. The torque sensor 251 converts this current into torque.
[0268] If the relationship "I=αM" between the current I required to rotate the operation unit 201 to a certain position and the mass M is known, then the mass M of the attached operation unit 201 can be estimated by measuring the current I required when the calibration unit 55 rotates the operation unit 201 to a certain position. Note that α can be easily found by measuring the current when rotating the operation unit 201 to a certain position for several operation units 201 with known masses.
[0269] In this manner, the mass M of the operation unit 201 to which the calibration unit 55 is attached is estimated. Regarding the size, a conversion formula for finding the mass from the size is used.
[0270] Therefore, if the operation unit detected by the operation unit sensor 254 is not included in the operation unit parameters, the size and mass of the attached operation unit can be estimated by calibration in addition to using an application on the mobile terminal 60.
[0271] [Mass correction according to the installation location of the control unit] The degree of inclination of the operation unit 201 differs depending on the installation location. For example, the inclination of the operation unit 201 differs when the operation unit 201 is attached to a steering wheel or a center console. When the inclination differs, the operation feel of the pressure-type operation unit in particular differs due to the effect of gravity. Therefore, the tactile presentation device 20 measures the inclination of the installation location of the operation unit 201 using the acceleration sensor 28, and corrects the mass of the operation unit 201.
[0272] Fig. 50 is a diagram for explaining the correction of the mass of the operation unit 201. Fig. 50(a) shows the operation reaction force F1 when the operation unit 201 placed at an installation location with zero tilt is pressed down. The operation reaction force F1 is, for example, the maximum value Tmax in Fig. 11. Fig. 50(b) shows the operation reaction force F2 when the operation unit 201 placed at an installation location with a tilt of θ is pressed down. Due to the relationship between the operation reaction forces F1, F2 and the tilt θ shown in Fig. 50(b), the operation reaction force F2 is as follows: F2=F1 / cosθ In this way, a large operation reaction force is required in an installation location with an inclination, but there is a correlation between the operation reaction force and mass. Therefore, the mass correction unit 261 corrects the mass of the operation unit 201 by regarding the difference in the operation reaction force as a difference in mass. The mass correction unit 261 corrects the mass of the operation unit 201 using a relationship such as "mass after correction = original mass / cos θ". In this way, even if the operation unit 201 is installed in an inclined location, it is possible to control a preferable operation feel.
[0273] [Operation and Processing] FIG. 51 is a flow chart showing a process of adjusting a haptic presentation signal in accordance with the physical parameters of an operation unit to which the haptic control system 110 is attached.
[0274] First, the haptic control system 110 obtains the correspondence between the physical parameters including the mass and size of the operation unit and the sensory parameters by SD method or the like (ST121).
[0275] Next, when the user wears the operation unit, the operation unit sensor 254 detects the operation unit worn by the user (ST122).
[0276] The tactile presentation device 20 judges whether or not the detected operation unit is included in the operation unit parameters 54 (ST123). The case where the operation unit sensor 254 cannot detect an ID is also included in the case where the operation unit parameters 54 does not include the case where the detected operation unit is not included in the operation unit parameters 54.
[0277] If the determination in step ST123 is Yes, the conversion model 15 converts the physical parameters already registered in the operation unit parameters 54 into sensory parameters (ST124). Note that the conversion model 15 in this embodiment calculates sensory parameters from physical parameters as shown in FIG.
[0278] If the determination in step ST123 is No, the user takes an image of the operation unit using an application on the mobile terminal 60, and transmits the size and mass to the tactile presentation device (ST125).
[0279] The communication unit 256 receives the size and mass from the application of the mobile terminal 60 (ST126). As described above, the size and mass determined by the calibration unit 55 through calibration may be used.
[0280] The conversion model 15 converts the estimated physical parameters (size, mass) into sensory parameters (ST127).
[0281] Then, the calculation function unit 12 generates a tactile sensation providing signal using physical parameters such as size and mass (which have been registered in the operation unit parameters 54 or have been estimated) (ST128).
[0282] The arithmetic function unit 13 transmits a tactile presentation signal to the tactile presentation device 20. When the user rotates the operation unit 201, the processor 18 generates an operation signal. If the operation unit is a rotary operation unit, the operation signal is, for example, a rotation angle. If the operation unit is another type of operation unit, the operation signal is an amount of operation of the operation unit. The tactile presentation unit 30 controls the actuator by the tactile presentation signal corresponding to the operation signal (ST129).
[0283] The calculation function unit 12 may convert the sensory parameters converted from the physical parameters in step ST127 back into physical parameters to generate a tactile sensation signal. A dedicated conversion model 15 may be prepared for the reconversion.
[0284] In this way, the haptic control system 110 can estimate the physical parameters even when an unregistered operation unit is attached. The calculation function unit 12 as an adjustment unit generates a haptic presentation signal based on the physical parameters, so that the haptic presentation signal can be adjusted according to the attached operation unit.
[0285] The adjustment unit is not limited to adjusting the "tactile sensation providing signal", and may be capable of adjusting the "operation signal", the "sensory providing signal", the "sensory presentation" itself, or any combination of these. Specifically, there are the following cases. A case in which the processor 18 (an example of an operation detection unit) functions as an adjustment unit and reflects the adjustment in the "operation signal." A case in which the calculation function unit 12 (an example of a signal generating unit) functions as an adjustment unit and reflects the adjustment in the "sensory presentation signal." A case in which the tactile presentation unit 30 functions as an adjustment unit and reflects adjustments in the "sensory presentation."
[0286] In addition, in this embodiment, since the conversion model estimates the sensory parameters from the physical parameters, it is possible to construct a correlation between the sensory parameters and the physical parameters that also reflect the physical parameters of the operation unit. To supplement this, the present content can also be applied to the content according to "adjusting the sensory presentation signal". That is, when the physical parameters of the operation unit change due to replacement of the operation unit, if the actuator is driven in the same way as before the operation unit 201 was replaced, the reproduced sensation, i.e., the sensory parameters, will be different. When the sensory parameters to be realized are constant, the sensory presentation signal can be adjusted to adjust the physical parameters of the actuator, and sensory presentation according to the set sensory parameters can be performed.
[0287] Moreover, Fig. 52 is a flowchart showing a process of adjusting a haptic presentation signal in accordance with a physical parameter of an operation unit to which the haptic control system 110 is attached, as a modified example of Fig. 51. In the explanation of Fig. 52, the differences from Fig. 51 will be mainly explained.
[0288] In FIG. 52, if the determination in step ST123 (an example of a predetermined condition) is No, the calculation function unit 12 stops generating the sensation providing signal (ST130).
[0289] In this way, when an operating unit with unknown physical parameters is attached and it is difficult to generate an appropriate sensation providing signal, the sensation providing signal can be stopped.
[0290] Further, the arithmetic function unit 12 may generate a predetermined sensation providing signal such as an initial value, instead of stopping the generation of the sensation providing signal.
[0291] [Haptic control system having a communication device (server) and a terminal device] Next, a haptic control system 111 having a communication device 70 (server) and a terminal device 80 will be described with reference to Fig. 53. Fig. 53 shows the configuration of the haptic control system 111 as a second embodiment of the sensation control system 100 shown in Fig. 45, together with the signal flow. Note that the explanation of Fig. 53 will mainly focus on the differences from Fig. 45.
[0292] 53, the tactile presentation device 20 of the terminal device 80 has a torque sensor 251, an operation unit sensor 254, and a communication unit 256. Furthermore, the communication device 70 has an operation unit parameter 54, a calibration unit 55, and a mass correction unit 261. The torque sensor 251, the operation unit sensor 254, the communication unit 256, the operation unit parameter 54, the calibration unit 55, and the mass correction unit 261 may be the same as those described in FIG.
[0293] FIG. 54 is a sequence diagram in which the communication device 70 (server) and the terminal device 80 communicate with each other to estimate the affective parameters of the attached operation unit.
[0294] In step ST131, the haptic control system 111 obtains the correspondence between the physical parameters including the mass and size of the operation unit and the sensory parameters by SD method or the like.
[0295] In step ST132, when the user wears the operation unit, the operation unit sensor 254 detects the operation unit worn by the user.
[0296] In step ST133, the terminal device 80 transmits the ID of the operation unit detected by the operation unit sensor 254 to the communication device 70. If the operation unit sensor 254 cannot detect the ID, the terminal device 80 transmits a non-detection of the ID to the communication device 70.
[0297] In step ST134, the communication device 70 determines, based on the ID of the operation unit received, whether or not the operation unit parameter 54 includes an installed operation unit.
[0298] If an operation unit is registered in the operation unit parameters 54, in step ST135, the conversion model 15 converts the physical parameters registered in the operation unit parameters 54 into sensory parameters.
[0299] If the operation unit is not registered in the operation unit parameters 54, in step ST136, a message is transmitted to the terminal device 80 indicating that the communication device 70 is not registered.
[0300] In step ST137, the user takes an image of the operation unit using an application on the mobile terminal 60, and transmits the size and mass of the image to the tactile presentation device 20.
[0301] In step ST138, the communication section 256 receives the size and mass from the application of the mobile terminal 60.
[0302] In step ST139, the portable terminal 60 transmits the size and mass to the communication device .
[0303] In step ST140, the conversion model 15 converts the estimated physical parameters (size, mass) into perceptual parameters.
[0304] In step ST141, the calculation function unit 12 generates a tactile sensation providing signal by using physical parameters such as size and mass (which have been registered in the operation unit parameters 54 or have been estimated).
[0305] In step ST142, the communication device 70 transmits a haptic presentation signal to the terminal device 80.
[0306] In step ST143, the haptic sense providing unit 30 controls the actuator by a haptic sense providing signal corresponding to the operation signal according to the user's operation. At least one of the operation signal, the sense providing signal, and the sense providing may be adjusted by either the communication device 70 or the terminal device 80.
[0307] [Major Effects] According to the tactile control systems 110, 111 of this embodiment, the physical parameters of the operating unit are adjusted according to the size, mass, etc. of the operating unit, so that even if the size or mass of the operating unit changes, the feeling conveyed to the user operating the unit can be controlled to a feeling that is preferable to the user.
[0308] [others] For example, the operation unit of aspect 3 is not limited to being detachable. For example, in a system in which a plurality of operation units are implemented, when a plurality of operation units with different knob sizes and designs are arranged, the difference can be recognized and an appropriate tactile sensation can be generated.
[0309] Furthermore, the operation unit sensor 254 may estimate the size and mass of the attached operation unit by comparison with a reference operation unit, instead of directly determining the size and mass by an application or calibration of the mobile terminal 60. For example, when an operation unit whose ID is registered in the operation unit parameters 54 and an operation unit whose ID is not registered are placed close to each other, two operation units are captured in the image data. The processor 18 obtains the ratio between the size of the operation unit whose ID is registered and the size of the operation unit whose ID is not registered, and estimates the size and mass of the operation unit whose ID is not registered by multiplying this ratio by the size and mass of the operation unit whose ID is registered.
[0310] The processor 18 is an example of an operation detection unit, the arithmetic function unit 12 is an example of a signal generation unit, and the tactile sense provision unit 30 is an example of a sensation provision unit.
[0311] [Additional Note for Mode 3] [Claim 1] An operation unit; an operation detection unit that detects an operation of the operation unit and generates an operation signal; a signal generating unit that generates a sensation providing signal based on the operation signal; a sensation providing unit that provides a sensation to an operator based on the sensation providing signal; A sensation control device comprising: an adjustment unit that adjusts at least one of the operation signal, the sensation presentation signal, and the sensation presentation based on physical characteristics of the operation unit. [Claim 2] The sensory control device according to claim 1, wherein the physical characteristics of the operating portion include at least one physical parameter of a mass, a diameter, a radius, or a total length of at least a portion of the operating portion. [Claim 3] An operation unit sensor that detects the attached operation unit is provided, The operation unit sensor identifies a physical characteristic of the operation unit by acquiring identification information of the operation unit, or The sensory control device according to claim 1, characterized in that the physical characteristics of the operation unit are identified from image data captured of the operation unit. [Claim 4] 2. The sensory control device according to claim 1, wherein the sensory providing unit stops generating the sensory providing signal when a physical characteristic of the operation unit satisfies a predetermined condition. [Claim 5] 2. The sensory control device according to claim 1, wherein the operation unit is a pressure-type operation unit that accepts a pressing operation. [Claim 6] 2. The sensory control device according to claim 1, wherein the operation unit is a slide operation unit that accepts a slide operation. [Claim 7] 2. The sensory control device according to claim 1, wherein the operation unit is a pivot operation unit that receives a tilt operation. [Claim 8] 2. The sensory control device according to claim 1, wherein the operation unit is a rotary operation unit that accepts rotational operation. [Claim 9] The sensory control device according to claim 1 , wherein the sensory presentation signal is correlated with a sensory parameter. [Claim 10] 2. The sensory control device according to claim 1, wherein the sensory presentation unit is a tactile presentation unit that presents a tactile sensation to an operator. [Claim 11] The sensory control device according to claim 1, characterized in that at least a part of the operation unit is detachable. [Claim 12] a torque sensor that detects a torque required when the operation unit is driven by an actuator; a calibration unit that estimates a mass of the operation unit from the torque detected by the torque sensor based on a relationship between torque and mass that is prepared in advance; The sensory control device according to claim 1, further comprising: [Claim 13] An acceleration sensor that detects a tilt of the operation unit; a mass correction unit that corrects a mass of the operation unit in accordance with the tilt detected by the acceleration sensor; The sensory control device according to claim 1, further comprising: [Claim 14] A sensory control method performed by a device having an operation unit, comprising: detecting an operation of the operation unit and generating an operation signal; generating a sensation providing signal based on the operation signal; providing a sensation to an operator based on the sensation providing signal; adjusting at least one of the operation signal, the sensation providing signal, and the sensation providing signal based on a physical characteristic of the operation unit; A method for controlling sensation, comprising: [Claim 15] A sensory control system including a communication device and a terminal device capable of communicating with each other, The terminal device An operation unit; an operation detection unit that detects an operation of the operation unit and generates an operation signal; a sensation providing unit that provides a sensation to an operator based on a sensation providing signal transmitted from the communication device, The communication device includes: a signal generating unit that generates the sensation providing signal based on the operation signal, The terminal device or the communication device is a sensory control system comprising an adjustment unit that adjusts at least one of the operation signal, the sensory presentation signal, and the sensory presentation based on the physical characteristics of the operation unit. [Aspect 4]
[0312] [Background technology] Conventionally, there are known operating tools that provide a sense of sensation by providing some kind of stimulus to a person. Here, the sense of sensation includes tactile sensation, auditory sensation by sound, and visual sensation by image display. The sense of sensation is adjusted by adjusting the signals that drive various operating tools.
[0313] A tactile system that considers a fingertip model to present a clicking sensation and the like is known (see, for example, Patent Document 4). Patent Document 4 discloses a technique for parameter evaluation by applying a response to a shear vibration generated by a fingertip while pressing a key to an approximation of the mass-spring-damper system of the fingertip.
[0314] [Summary of the Invention] [Problem to be solved by the invention] However, conventional technology does not take into account deformation of elastic bodies such as fingers in the direction of operation, such as buckling due to pressing operations, and therefore has a problem of narrowing the range of expressiveness of sensory presentation. In other words, although fingers contain elastic bodies such as skin and flesh, buckling phenomena caused by elastic bodies are not reflected in sensory presentation.
[0315] In view of the above-mentioned problems, the present aspect aims to provide a technology that further expands the range of expressiveness of sensory presentation.
[0316] [Effects of the invention] It is possible to provide technology that further expands the range of expressiveness of sensory presentation.
[0317] [Explanation of Aspect 4] In this embodiment, a tactile control system 1 that outputs a sensory stimulation signal based on physical parameters including dynamic characteristics and a sensory control method thereof will be described. Dynamic characteristics are physical characteristics including a time factor, and for example, physical characteristics change over time.
[0318] In this embodiment, the block diagram of FIG. 1, the hardware configuration diagram of the haptic control system 1 of FIG. 2, and other necessary explanations explained in the above embodiment 1 are assumed to be applicable.
[0319] Conventionally, the load-displacement curve when a user presses an operating tool such as a switch is based on the static characteristics when a rigid body is pressed, and does not include a time factor. Therefore, it is not possible to obtain information on the correspondence between the sensory parameters and the physical parameters in a state where the buckling phenomenon that occurs when a user actually presses the switch with his / her finger is reproduced.
[0320] In this embodiment, in order to approximate the situation when a user presses an operating tool with a finger, a finger model pressing tool is used to press the operating tool, which is provided with an elastic body (corresponding to the flesh and skin of the finger) integrated with a rigid body (corresponding to the bones of the finger) between the rigid body and the operating tool. By analyzing the position change [mm] of the operating tool and the two force sensor values [N] between the elastic body and the operating tool when the finger model pressing tool presses the operating tool, measurement evaluation was performed using the SD method in a configuration that takes into account the fingers of the human body. Since the new physical parameters obtained by this method include dynamic characteristics, correspondence information between the sensitivity parameters and the physical parameters is generated in a state where the buckling phenomenon that occurs when a user actually presses the operating tool with his or her finger is reproduced.
[0321] Specifically, the following correlation is obtained: Note that the buckling period T1, the fingertip impact period T3, and the fingertip vibration period T4 are the periods shown in Fig. 60(b) to be described later, and details will be given later. Correlation between physical parameters (movement distance of the operating tool during the fingertip impact period T3, change in the force sensor value during the fingertip impact period T3, fingertip impact period T3) and sensory parameters (sense of return) Correlation between physical parameters (position change during buckling period T1) and sensory parameters (feeling of being sucked in) Correlation between physical parameters (fingertip vibration period T4) and sensory parameters (feeling of fatigue) [Example of finger model pressing tool and operation tool configuration] Fig. 55 is a diagram for explaining the static characteristics obtained by a rigid pressing tool and the dynamic characteristics obtained by a finger model pressing tool 252 in which a rigid body and an elastic body are integrated. First, the load-displacement curve of the operating tool 250 by a pressing tool of a rigid body 253 can only express the static characteristics that do not include a time factor. The load-displacement curve 75 does not include the influence of the elastic body corresponding to the flesh part 257 of the finger, and therefore does not fully express the physical characteristics that contribute to the tactile sensation perceived by the operator.
[0322] Next, the pressing of the operating tool 250 by the finger model pressing tool 252 will be described. First, the flesh 257 of the finger is an elastic body that deforms with stress. In addition, there is also a bone 255 inside the finger that can be considered as a rigid body. As described later, the finger model pressing tool 252 is designed to have the characteristics of the flesh 257 and the bone 255. When the finger model pressing tool 252, which is an integral rigid body and elastic body, presses the operating tool 250, the operation reaction force, position change, etc. include dynamic characteristics with respect to time. In FIG. 55, the position change and two force sensor values A and B are shown as dynamic characteristics 270. The two force sensor values A and B detect the operation reaction force generated by the finger model pressing tool 252 against the operating tool 250. The two force sensor values A and B are measured by different force sensors, and are respectively arranged at the place where the flesh 257 of the finger contacts the operating tool 250 and at the rigid part inside the finger (corresponding to the bone 255). Details will be described in Fig. 59. As shown in dynamic characteristics 270, the pressing of operating tool 250 by finger model pressing tool 252 can capture the finger movement taking time into consideration, that is, the occurrence and change of sensation, and therefore a correlation close to the actual situation when a user presses with his / her finger can be obtained.
[0323] Fig. 56 is a diagram for explaining the relative positions of a finger and operating tool 250 when the finger is deformed. The upper part of Fig. 56 shows periods A to C that can be read from load-displacement curve 75. The lower part of Fig. 56 typically shows deformation of the flesh of the finger corresponding to periods A to C.
[0324] As shown in the lower part of FIG. 56, in period A, the position of button portion 56 of operating tool 250 gradually drops as the pushing force of the finger and the repulsive force are balanced.
[0325] In period B, deformation (buckling) of metal contact 57 of operating tool 250 occurs, and the repulsive force disappears. Button portion 56 falls downward while maintaining the downward force. The operation reaction force becomes a difference compared to period A. Therefore, the operation reaction force of the contact portion between the finger and the button is reduced.
[0326] In period C, the finger and button portion 56 collide again with metal contact 57. At this time, the maximum operation reaction force is again generated at the contact portion between the fingertip and button portion 56. The collision also causes vibration of button portion 56.
[0327] Fig. 57 is a diagram for explaining finger model pressing tool 252. As explained in Fig. 56, a finger is an elastic body in which flesh 257 deforms. Also, bones 255 that can be regarded as rigid bodies exist inside the finger. Thus, finger model pressing tool 252 having elastic body 59 in contact with button portion 56 and rigid body 58 pressing button portion 56 via elastic body 59 serves as an appropriate model when a finger presses operating tool 250.
[0328] [Generation of a sensory signal with a clicking sensation] Fig. 58 is a diagram for explaining the generation of a tactile signal that produces a clicking sensation. The clicking sensation refers to the reaction of an input device such as a button, or the feeling of pressing a switch. In the case of a mechanical switch, the clicking sensation is obtained by the resistance or deformation of the metal contact 57, etc. However, how the clicking sensation is generated varies depending on the button structure.
[0329] In addition, in the operating tool 250 in this embodiment in which the sensation providing signal is generated electrically, the clicking sensation is controlled by the current supplied to the actuator.
[0330] FIG. 58(a) shows the current value of the actuator versus time, and FIG. 58(b) shows the reaction force versus time. When the current value suddenly decreases in frame 283, the reaction force also suddenly decreases. A convex portion 284 in FIG. 58(b) corresponds to the time when the current value suddenly decreases. Therefore, when the user presses the operating tool 250 with his / her finger, a response (click sensation) similar to pressing a mechanical switch is obtained. The timing at which the current value suddenly decreases and the amount of reduction in current shown in FIG. 58(a) are merely examples and are adjusted as appropriate.
[0331] Fig. 59(a) shows a functional configuration diagram of a pressure type operating tool, and Fig. 59(b) shows a block diagram of the pressure type operating tool. The button portion 271 in Fig. 59 is an example of the operating device 33 in Fig. 2, and the VCM 263 is an example of the tactile sense providing unit 30 in Fig. 2. As shown in Fig. 59(a), two force sensors A and B are arranged on the finger model pressing tool 252. The force sensor A is arranged at a position where the elastic body 59 of the finger model pressing tool 252 and the button portion 271 contact each other, and the force sensor B is arranged inside the rigid body 58 of the finger model pressing tool 252. In this way, the buckling phenomenon can be monitored by the force sensor value A detected by the force sensor A.
[0332] Although the block diagram in Fig. 59(b) is merely an example of a pressing type operating tool, it will be briefly explained. The MCU circuit 262 is an example of the processor 18 in Fig. 2, and the position sensor 264 is an example of the position sensor 27 in Fig. 2. As shown in Fig. 59(b), the MCU circuit 262 outputs a current corresponding to the amount of operation (position change) by which the button part 271 of the operating tool 250 is pressed to the VCM (Voice Coil Motor) 263. The VCM 263 applies an artificial reaction force proportional to the current to the button part 271. The finger model pressing tool 252 presses the button part 271 from the opposite side to the VCM 263, so that an artificial reaction force is transmitted to the finger model pressing tool 252. The artificial reaction force is measured by the force sensors A and B.
[0333] [Dynamic characteristics obtained by finger model pressing tool] FIG. 60 is a diagram for explaining dynamic characteristics when the operating tool 250 is pressed by the finger model pressing tool 252. FIG. 60(a) is a load-displacement curve 75 shown for reference, and FIG. 60(b) is an example of dynamic characteristics 270 when the operating tool 250 is pressed by the finger model pressing tool 252. In FIG. 60(b), the horizontal axis represents time, and the vertical axis represents two force sensor values A and B and position change 211. The unit of time is [msec], and the unit of the force sensor values A and B is [N]. Note that the dynamic characteristics 270 vary greatly depending on the operating tool 250, and FIG. 60(b) is merely one example.
[0334] The dynamic characteristics (buckling period T1, fingertip drop period T2, fingertip collision period T3, fingertip vibration period T4) extracted from two force sensor values A and B over time and position change 211 over time will be described with reference to Fig. 60 and Fig. 61. Fig. 61 is a diagram for explaining the transition over time of the relative position of the finger model pressing tool 252 and the operating tool 250.
[0335] The buckling period T1 is the period from the peak of the force sensor value B to the peak of the position change 211. Although it is difficult to understand due to the scale, the force sensor value B is not constant and has a peak at the start of the buckling period T1. This peak will be explained in FIG. 62. The peak of the force sensor value B corresponds to the maximum value of the operation reaction force in the load-displacement curve 75. Therefore, the buckling period T1 is the period from when the operation reaction force becomes maximum to when the position change 211 becomes maximum. FIG. 61(a) shows the relative positions of the finger model pressing tool 252 and the button part 56 at the start of the buckling period T1. At the start of the buckling period T1, the maximum value of the operation reaction force in the load-displacement curve 75 is obtained, so that the elastic body 59 of the finger model pressing tool 252 is pressed down greatly. The arrow on the left of the button part 56 indicates the direction of the position change.
[0336] The fingertip drop period T2 is the period from the peak of the force sensor value B to the downward peak of the force sensor value A. As shown in the load-displacement curve 75, after the maximum value of the operation reaction force is obtained, the operation reaction force drops suddenly to give a clicking sensation. Therefore, the operation reaction force to the finger model pressing tool 252 decreases, and the elastic body 59 of the finger model pressing tool 252 starts to restore after the start point of the fingertip drop period T2. This causes the force sensor value A to decrease during the fingertip drop period T2. Therefore, the fingertip drop period T2 is the period from the maximum value of the operation reaction force is obtained to the elastic body of the finger model pressing tool 252 being restored to its maximum. FIG. 61(b) shows the relative positions of the finger model pressing tool 252 and the button portion 56 at the end point of the fingertip drop period T2. Comparing with FIG. 61(a), it can be seen that the elastic body 59 of the finger model pressing tool 252 has been restored.
[0337] The fingertip collision period T3 is the period from the downward peak of the force sensor value A to the upward peak of the force sensor value A. In the fingertip collision period T3, the force sensor value A increases rapidly because the finger model presser 252 continues to be pressed after the elastic body 59 of the finger model presser 252 has been restored to its maximum in FIG. 61(b). Therefore, the fingertip collision period T3 is the period from when the elastic body 59 of the finger model presser 252 has been restored to its maximum until the elastic body 59 is pressed to its maximum. FIG. 61(c) shows the relative positions of the finger model presser 252 and the button portion 56 at the end point of the fingertip collision period T3. Comparing with FIG. 61(b), it can be seen that the elastic body 59 of the finger model presser 252 is pressed.
[0338] The fingertip vibration period T4 is the period from the upward peak of the force sensor value A until the fluctuation of the force sensor value A falls within a certain value. Since the operation reaction force has already become small to produce a clicking sensation, even if the position change 211 continues to increase due to pressing, the force sensor A becomes suddenly small. After that, the position change 211 stops increasing (the finger model pressing tool 252 also stops moving), so the force sensor value B also becomes difficult to change, and the force sensor value A vibrates like chattering. Therefore, the fingertip vibration period T4 is the period until the most pressed elastic body is restored and stabilized. Figure 61 (d) shows the relative positions of the finger and the button part 56 at the end point of the fingertip vibration period T4. Comparing with Figure 61 (c), it can be seen that the elastic body 59 of the finger model pressing tool 252 has been restored.
[0339] The above buckling period T1, fingertip drop period T2, fingertip impact period T3, and fingertip vibration period T4 are examples of dynamic characteristics. In addition, the changes in the force sensor values A and B and the position change 211 can be extracted during each of the buckling period T1, fingertip drop period T2, fingertip impact period T3, and fingertip vibration period T4. In this embodiment, these can also be used as dynamic characteristics.
[0340] In this way, the dynamic characteristic may be a physical characteristic including a time change of at least one of the operation reaction force and the operation amount accompanying the operation of a predetermined operation tool 250. This physical characteristic is a physical characteristic that realizes the presentation of a sensation when the elastic body 59 of the finger model pressing tool 252 is brought into contact with the operation tool 250 and operated.
[0341] FIG. 62 is a diagram for explaining the dynamic characteristics in more detail along with the above-mentioned periods A to C. The upper left part of FIG. 62 is an overall view including the dynamic characteristics from the start to the end, and the lower right part of FIG. 62 is an enlarged view of the dynamic characteristics within frame 212 in the upper left part of FIG. 62. The lower right part of FIG. 62 shows the correspondence between the dynamic characteristics and periods A to C. The force sensor value A detected by force sensor A changes significantly due to the pressing and restoration of elastic body 59. The force sensor value B detected by force sensor B is less affected by the deformation of elastic body 59, and therefore changes only slightly.
[0342] The lower right part of Fig. 62 also shows the buckling period T1, fingertip drop period T2, fingertip impact period T3, and fingertip vibration period T4, which are as explained in Fig. 60. The peaks of force sensor B (the start points of the buckling period T1 and fingertip drop period T2), which were not clear in Fig. 60, are now clear.
[0343] [Dynamic characteristics correlated with sensory parameters] Among the dynamic characteristics described in Figures 60 and 62, there are some that are correlated with the sensory parameters. Appropriate dynamic characteristics that are correlated with the sensory parameters are the physical parameters of this embodiment.
[0344] The haptic control system 1 performs an evaluation using the SD method in order to evaluate appropriate dynamic characteristics correlated with the sensitivity parameters. For this purpose, a plurality of operating tools 250 with different dynamic characteristics are prepared.
[0345] Fig. 63 shows dynamic characteristics of a plurality of operating tools 250 with different dynamic characteristics when pressed by a finger model pressing tool 252. In this embodiment, 25 operating tools 250 were prepared for the purpose of explanation, and the dynamic characteristics were measured for each of the 25 operating tools 250. Fig. 63 shows the dynamic characteristics of four of the operating tools 250. In each of Fig. 63(a) to (d), the upper diagram shows dynamic characteristics 270 for the entire period (about 1 second) when pressed, and the lower diagrams show enlarged views of dynamic characteristics 270 before and after a buckling period T1, a fingertip dropping period T2, a fingertip collision period T3, and a fingertip vibration period T4.
[0346] <Determination of physical parameters correlated with sensory parameters> FIG. 64 is a flow chart for explaining the flow of determining physical parameters correlated with sensory parameters.
[0347] In step ST151, the haptic control system 1 measures the dynamic characteristics when the finger model pressing tool 252 presses each of the 25 operating tools 250.
[0348] Next, in step ST152, the input unit 4 receives the expression frequency for each of the 25 operating tools 250 using the SD method.
[0349] Next, in step ST153, the processor 101 obtains a pair of the dynamic characteristic and the expression frequency of each operation tool 250 for each affective parameter.
[0350] Next, in step ST154, the processor 101 obtains a correlation coefficient between the dynamic characteristic and the expression frequency for each affective parameter.
[0351] Next, in step ST155, the processor 101 determines a dynamic characteristic having a large absolute value of the correlation coefficient. The absolute value of the correlation coefficient may be, for example, 0.5 or more.
[0352] Next, in step ST156, the processor 101 applies the multiple regression analysis explained in relation to equation (5) to the physical parameters and the sensory parameters that are highly correlated with the sensory parameters, thereby creating the conversion model 15.
[0353] Fig. 65 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool 250 for a certain affective parameter acquired by the processor 101 in step ST153. In Fig. 65, the horizontal axis indicates "there is (is not) a sense of return" as an affective parameter, and the vertical axis indicates the buckling period T1. The buckling period T1 and the expression frequencies of "there is (is not) a sense of return" generally have an upward trend to the right. The correlation coefficient is 0.82.
[0354] Fig. 66 is a scatter diagram of pairs of dynamic characteristics and expression frequencies of each operating tool 250 for a certain affective parameter acquired by the processor 101 in step ST153. In Fig. 66, the horizontal axis indicates "feeling of being sucked in (not feeling like it is being sucked in)" as an affective parameter, and the vertical axis indicates position change during the fingertip impact period T3. There is a general downward trend between the position change during the fingertip impact period T3 and the expression frequency of "feeling of being sucked in (not feeling like it is being sucked in)". The correlation coefficient is 0.65.
[0355] Fig. 67 is a scatter diagram of pairs of dynamic characteristics of each operating tool 250 and expression frequency for a certain sensory parameter acquired by processor 101 in step ST153. In Fig. 67, the horizontal axis indicates "feeling of return (no feeling of return)" as a sensory parameter, and the vertical axis indicates the change in operation reaction force (force sensor value A) during fingertip vibration period T4. The change in operation reaction force during fingertip vibration period T4 and the expression frequency of "feeling of return (no feeling of return)" generally have an upward trend to the right. The correlation coefficient is 0.78.
[0356] The processor 101 correlates the sensitivity parameters and dynamic characteristics shown in Figures 65, 66, and 67 using the least squares method (an example of regression analysis) or the like. The least squares method estimates the strength of the correlation between the sensitivity parameters and the dynamic characteristics using a correlation coefficient.
[0357] Fig. 68 shows a list of correlation coefficients between each sensory parameter and each dynamic characteristic. In Fig. 68, the row headings are sensory parameters, and the column headings are dynamic characteristics of the operation tool 250. In Fig. 68, correlation coefficients of 0.5 or more are highlighted with diagonal lines. It can therefore be seen that dynamic characteristics with large correlation coefficients are suitable for physical parameters.
[0358] In this way, when each operating tool 250 is pressed by the finger model pressing tool 252, the physical parameters highly correlated with the sensory parameters are determined, and the processor 101 can apply the multiple regression analysis explained in Equation 5 to the physical parameters highly correlated with the sensory parameters and the sensory parameters to create the conversion model 15. The physical parameters P1 to Pn used in Equation 5 are those with large correlation coefficients determined in step ST154. The multiple regression analysis has been explained in Equation 5, Figs. 22 and 23 of the first embodiment. Therefore, the coefficient of determination B 11 ~B mn can be determined, and a transformation model 15 as shown in FIG.
[0359] [Haptic control system having a communication device (server) and a terminal device] Next, a haptic control system 2 having a communication device 70 (server) and a terminal device 80 will be described with reference to Fig. 69. Note that the block diagram of the haptic control system 2 may be the same as that in Fig. 20.
[0360] FIG. 69 is a sequence diagram showing how the communication device 70 (server) and the terminal device 80 communicate with each other to estimate the affective parameters of the attached operating tool 250. In FIG.
[0361] In step ST161, the communication device 70 and the terminal device 80 communicate with each other, and the finger model pressing tool 252 presses the 25 operating tools 250, thereby measuring the dynamic characteristics of each operating tool 250.
[0362] Next, in step ST162, the input unit 4 receives the expression frequency for each of the 25 operating tools 250 using the SD method.
[0363] Next, in step ST163, the terminal device 80 transmits the expression frequency to the communication device .
[0364] Next, in step ST164, the processor 14 obtains a pair of the dynamic characteristic and the expression frequency of each operation tool 250 for each affective parameter.
[0365] Next, in step ST165, the processor 14 obtains a correlation coefficient between the dynamic characteristic and the expression frequency for each affective parameter.
[0366] Next, in step ST166, the processor 14 determines a dynamic characteristic having a large absolute value of the correlation coefficient. The absolute value of the correlation coefficient may be, for example, 0.5 or more.
[0367] Next, in step ST167, the processor 14 applies the multiple regression analysis explained in Equation 5 to the physical parameters and the sensory parameters that are highly correlated with the sensory parameters, thereby creating the conversion model 15.
[0368] [Major Effects] As described above, the tactile control system 1 of this embodiment can extract dynamic characteristics correlated with the sensory parameters by pressing the operating tool 250 with the finger model pressing tool 252. Therefore, a conversion model that converts the sensory parameters into these dynamic characteristics can be created, so that a sensory presentation signal with desirable dynamic characteristics can be generated.
[0369] [others] For example, in the fourth embodiment, a pressure-type operating tool is described, but the present invention can be applied to a rotary operating tool that accepts rotational operation. In the case of a rotary operating tool, the rotation angle is the position change, and the resistance force against the rotation is the operation reaction force.
[0370] Moreover, while the finger model pressing tool 252 has been described as having only one type of elastic body 59, the finger model pressing tool 252 may have multiple types of elastic bodies with different elastic forces on the side that comes into contact with the button portion 56. The multiple types of elastic bodies with different elastic forces are, for example, elastic bodies corresponding to skin, elastic bodies corresponding to flesh, etc. Furthermore, the multiple types of elastic bodies with different elastic forces may be arranged in layers so that the closer they are to the rigid body 58, the greater the elastic force. In this way, a finger model pressing tool 252 can be constructed that exhibits dynamic characteristics closer to the human sense of touch.
[0371] The shape of the finger model pressing tool 252 may be a simple cube, or may imitate the shape of a finger. The finger shape may be different in size and shape, assuming the fingers of men, women, adults, children, and people of all races.
[0372] [Additional Note for Mode 4] [Claim 1] a receiving step of receiving an input of a sensitivity parameter indicating a degree of sensory expression when the operating tool is operated; a conversion step of converting the received sensory parameter into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in a physical property related to a sensory stimulus; and outputting a sensory stimulation signal based on the converted physical parameter. The sensory control method, wherein the physical property includes a dynamic property. [Claim 2] 2. The sensation control method according to claim 1, wherein the dynamic characteristic is a physical characteristic including a time change in at least one of an operation reaction force and an operation amount accompanying the operation of a predetermined operating tool. [Claim 3] The sensation control method according to claim 2, wherein the physical property is a physical property that realizes sensation presentation when a finger model pressing tool including a rigid body and an elastic body is brought into contact with the specified operating tool and operated. [Claim 4] The sensation control method according to claim 1 , wherein the physical parameter is a buckling period. [Claim 5] The sensation control method according to claim 1 , wherein the physical parameter is a fingertip drop duration. [Claim 6] The sensory control method of claim 1 , wherein the physical parameter is fingertip impact duration. [Claim 7] The sensation control method according to claim 1 , wherein the physical parameter is a fingertip vibration period. [Claim 8] The sensation control method according to claim 1 , wherein the physical parameter has a correlation with the sensory parameter. [Claim 9] 2. The sensation control method according to claim 1, wherein the operation tool is a pressure-type operation tool that accepts a pressing operation. [Claim 10] 2. The sensation control method according to claim 1, wherein the operating tool is a rotary operating tool that accepts a rotational operation. [Claim 11] an input unit that receives an input of a sensitivity parameter indicating a degree of sensory expression when the operating tool is operated; a conversion model for converting the sensory parameter received by the input unit into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in a physical characteristic related to a sensory stimulus; a sensation providing unit that outputs a sensory stimulation signal based on the physical parameters converted by the conversion model, The device wherein the physical characteristics include dynamic characteristics. [Claim 12] A sensory control system including a communication device and a terminal device capable of communicating with each other, the terminal device has an input unit that receives an input of an affective parameter indicating a degree of sensory expression when the operating tool is operated, the communication device has a conversion model for converting the sensory parameter transmitted from the terminal device into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in physical characteristics related to sensory stimuli; the terminal device includes a sensation providing unit that outputs a sensory stimulation signal based on the physical parameters converted by the conversion model; The sensory control system, wherein the physical characteristics include dynamic characteristics. [Claim 13] The device, an input unit that receives an input of a sensitivity parameter indicating a degree of sensory expression when the operating tool is operated; a conversion model for converting the sensory parameter received by the input unit into a physical parameter correlated with the sensory parameter among a plurality of types of physical parameters included in a physical characteristic related to a sensory stimulus; a sensory presentation unit that outputs a sensory stimulation signal based on the physical parameters converted by the conversion model; The program, wherein the physical characteristics include dynamic characteristics.
[0373] [others] Although the best mode for carrying out the present invention has been described above using various aspects, the present invention is not limited to these aspects, and various modifications and substitutions can be made without departing from the scope of the present invention. For example, the functions included in each component, step, etc. can be rearranged so as not to cause logical contradictions, and multiple components, steps, etc. can be combined into one or divided.
[0374] This application claims priority based on Patent Application No. 2021-084696 filed with the Japan Patent Office on May 19, 2021, Patent Application No. 2022-079095 filed with the Japan Patent Office on May 12, 2022, Patent Application No. 2022-079099 filed with the Japan Patent Office on May 12, 2022, and Patent Application No. 2022-079128 filed with the Japan Patent Office on May 13, 2022, and the entire contents of Patent Application No. 2021-084696, Patent Application No. 2022-079095, Patent Application No. 2022-079099, and Patent Application No. 2022-079128 are incorporated by reference into this application. [Explanation of symbols]
[0375] 1, 2 Haptic control system 3 Input / Output Devices 4 Input section 5 Display section 6, 10 Main control device 7, 14, 18, 41, 101 processors 8, 11 Storage section 9 Network 12, 13 Calculation function unit 15 Sensibility parameter-physical parameter conversion model 16 Sensitivity Database 20, 40 Tactile presentation device 21 Moving parts 24 Bobbin 25 Coil 26 Spring parts 27 Position Sensor 28 Acceleration Sensor 29 Variable operation range section 30, 43 Tactile display unit 31 York 31a Outer yoke 31b Centre York 32 Magnet 33, 42 Operating device 39 Actuator 43a Resistance torque generator 43b Rotational torque generator 45 Sensors 70 Communication Equipment 80 Terminal Equipment 100 Sensory Control System 102 Sensory presentation unit
Claims
1. A rotary operation unit; an operation detection unit that detects an operation of the rotary operation unit and generates an operation signal; a signal generating unit that generates a sensation providing signal based on the operation signal; a sensation providing unit that provides a sensation to an operator based on the sensation providing signal, The mass and size of the rotary operation unit are physical characteristics, and based on the mass and size, at least one of the operation signal, the sensation presentation signal, and the sensation presentation when operating the rotary operation unit is determined. A sensory control device comprising an adjustment unit that adjusts the torque required by the operator to rotate the rotary operating unit so that the reaction force of the rotation operation is approximately the same regardless of the diameter, so that the larger the diameter of the rotary operating unit, the greater the torque required by the operator to rotate the rotary operating unit, and so that the reaction force of the rotation operation is approximately the same regardless of the mass, so that the torque required by the operator to rotate the rotary operating unit is smaller as the mass of the rotary operating unit is greater.
2. The sensory control device of claim 1, wherein the mass and radius, diameter, or length of the maximum part of the rotary operating unit are physical characteristics, and at least one of the operation signal, the sensory presentation signal, and the sensory presentation when rotating the rotary operating unit is adjusted based on the mass and radius, diameter, or length of the maximum part.
3. The sensory control device according to claim 1 , wherein the physical characteristics of the rotary operating unit include at least one physical parameter of the mass of at least a portion of the rotary operating unit, and the diameter, the radius, or the total length.
4. An operation unit sensor that detects the attached operation unit is provided, The operation unit sensor identifies a physical characteristic of the rotary operation unit by acquiring identification information of the rotary operation unit, or The sensory control device according to claim 1 , wherein physical characteristics of the rotary operation unit are identified from image data obtained by photographing the rotary operation unit.
5. The sensory control device according to claim 1 , wherein the sensory providing unit stops generating the sensory providing signal when a physical characteristic of the rotary operation unit satisfies a predetermined condition.
6. The sensory control device according to claim 1 , wherein the sensory presentation signal is correlated with a sensory parameter.
7. The sensory control device according to claim 1 , wherein the sensory presentation unit is a tactile presentation unit that presents a tactile sensation to an operator.
8. The sensory control device according to claim 1 , wherein at least a portion of the rotary operation unit is detachable.
9. a torque sensor that detects a torque required when the rotary operation unit is driven by an actuator; a calibration unit that estimates a mass of the rotary operation unit from the torque detected by the torque sensor based on a previously prepared relationship between torque and mass; The sensory control device according to claim 1, further comprising:
10. The rotary operation unit is installed at a predetermined installation location, an acceleration sensor that detects a tilt of the rotary operation unit at the installation location; a mass correction unit that corrects an operation reaction force of the rotary operation unit based on an action of gravity corresponding to the inclination detected by the acceleration sensor; The sensory control device according to claim 1, further comprising:
11. A sensory control method performed by a device having a rotary operation unit, comprising: Detecting an operation of the rotary operation unit and generating an operation signal; generating a sensation providing signal based on the operation signal; providing a sensation to an operator based on the sensation providing signal; The mass and size of the rotary operation unit are physical characteristics, and based on the mass and size, at least one of the operation signal, the sensation presentation signal, and the sensation presentation when operating the rotary operation unit is determined. a step of adjusting the torque required by an operator to rotate the rotary operation unit so that the reaction force of the rotation operation is approximately the same regardless of the diameter, so that the torque required by an operator to rotate the rotary operation unit is larger as the diameter of the rotary operation unit increases, and so that the reaction force of the rotation operation is approximately the same regardless of the mass of the rotary operation unit so that the torque required by an operator to rotate the rotary operation unit is smaller as the mass of the rotary operation unit increases; A method for controlling sensation, comprising:
12. A sensory control system including a communication device and a terminal device capable of communicating with each other, The terminal device A rotary operation unit; an operation detection unit that detects an operation of the rotary operation unit and generates an operation signal; a sensation providing unit that provides a sensation to an operator based on a sensation providing signal transmitted from the communication device, The communication device includes: a signal generating unit that generates the sensation providing signal based on the operation signal, The terminal device or the communication device defines the mass and size of the rotary operation unit as physical characteristics, and based on the mass and size, generates at least one of the operation signal, the sense presentation signal, and the sense presentation when operating the rotary operation unit. a sensory control system including an adjustment unit that adjusts the torque required by an operator to rotate the rotary operating unit so that the reaction force to the rotation operation is approximately the same regardless of the diameter, so that the torque required by an operator to rotate the rotary operating unit is greater as the diameter of the rotary operating unit increases, and that adjusts the torque required by an operator to rotate the rotary operating unit so that the reaction force to the rotation operation is approximately the same regardless of the mass of the rotary operating unit.
Citation Information
Patent Citations
Alternating current zero voltage switch circuit
JP1981062425A
Tactile devices and technologies for quantifying their capabilities
JP2013519961A
Input device
JP2015132890A
Systems and methods for designing haptics using speech commands
JP2019220168A
Video Game Controllers
JP2020523068A
